# Coverage is not enough: Frequentist tests of simulation-based inference for primordial non-Gaussianity

**Authors:** Toka Alokda, Cristiano Porciani, Alexander Eggemeier  
**arXiv:** [2605.00980](https://arxiv.org/abs/2605.00980)  
**Date:** 2026-05-01  
**Summary page:** https://www.summarizepaper.com/en/paper/2605.00980v1/

## Key points

- Simulation-based inference (SBI) is valuable for analyzing complex, non-linear data without analytical likelihoods
- SBI reliability typically assessed using coverage-based diagnostics under prior predictive distribution, which may not provide constraints on posterior behavior at fixed parameter values
- Study focused on primordial non-Gaussianity parameterized by $f_\mathrm{NL}$ using simulations of dark matter halo field
- Comparison between SBI based on contrastive neural ratio estimation (CNRE) and likelihood-based inference (LBI) using statistical measures like power spectrum, bispectrum, and wavelet scattering transform (WST) coefficients across 1000 realizations
- SBI and LBI generally agreed well on posterior means and skewness but showed discrepancies in variance consistency and kurtosis values
- Incorporating WST coefficients enhanced constraints on $f_\mathrm{NL}$, highlighting the potential of higher-order statistics in refining cosmological parameter estimation
- Importance of validation strategies beyond standard coverage diagnostics to probe posterior shapes comprehensively for accurate cosmological information extraction

## Summary

In their paper titled "Coverage is not enough: Frequentist tests of simulation-based inference for primordial non-Gaussianity," authors Toka Alokda, Cristiano Porciani, and Alexander Eggemeier delve into the limitations of simulation-based inference (SBI) in the context of extracting cosmological information related to primordial non-Gaussianity. SBI has proven to be a valuable tool for analyzing complex, non-linear data where analytical likelihoods are not readily available. However, the reliability of SBI is typically evaluated using coverage-based diagnostics under the prior predictive distribution, which only assesses calibration in an averaged sense and does not provide constraints on posterior behavior at fixed parameter values - a crucial aspect for practical inference. To address these limitations, the authors conducted a comprehensive investigation focusing on primordial non-Gaussianity parameterized by $f_\mathrm{NL}$ using simulations of the dark matter halo field. They compared SBI based on contrastive neural ratio estimation (CNRE) with likelihood-based inference (LBI) utilizing various statistical measures such as the power spectrum, bispectrum, and wavelet scattering transform (WST) coefficients across 1000 realizations. The study revealed that SBI and LBI generally agreed well on posterior means and skewness, with some discrepancies observed in variance consistency across realizations. Notably, larger differences were found in kurtosis values, indicating discrepancies in posterior tails. These discrepancies were evident even for the power spectrum analysis - where Gaussian likelihood assumptions are most justified - and were particularly pronounced when considering combined power spectrum and bispectrum analyses. In such cases, SBI posteriors tended to be underconfident and could lead to weaker constraints compared to individual statistics despite passing coverage tests. Furthermore, incorporating WST coefficients further enhanced constraints on $f_\mathrm{NL}$, showcasing the potential of higher-order statistics in refining cosmological parameter estimation. The results underscored the importance of validation strategies that go beyond standard coverage diagnostics to probe posterior shapes more comprehensively. Overall, this study sheds light on the nuances of simulation-based inference in cosmological parameter estimation and emphasizes the significance of exploring alternative statistical approaches to improve accuracy and reliability in extracting cosmological information from complex datasets.