Accuracy stability checks

When developing new models, adding new approximations, changing numerical methods, or updating the Fortran calculation, it is good practice to check that the requested results are stable to tighter numerical accuracy settings. Small changes in the code can otherwise move spectra in ways that are hard to spot from a single run. The check_accuracy command compares a normal CAMB run with a higher-accuracy reference run and reports where any differences exceed configurable tolerances.

The same implementation is available from Python as the camb.check_accuracy module, and from the command line as:

camb check_accuracy inifiles/planck_2018.ini

This tests numerical stability with mostly fixed scale cuts. Also check lens-potential-accuracy is set high enough (effectively increasing kmax), though note that inaccuracies in the default non-linear halo model are often larger than numerical errors.

What it compares

The checker loads an input .ini file using camb.read_ini(), runs it once as requested, then runs a reference calculation with boosted accuracy settings. By default the reference uses:

AccuracyBoost = 2
lSampleBoost = 2
lAccuracyBoost = 2
IntTolBoost = 2
DoLateRadTruncation = True

It compares:

derived parameters, reported as fractional changes;
total lensed CMB spectra from get_total_cls(), so tensor contributions are included when present;
the lensing potential spectrum from get_lens_potential_cls();
matter power spectra when transfer functions are requested by the input parameters.

For TT and EE the comparison is fractional in each multipole range. For TE it uses Delta TE / sqrt(TT * EE), which avoids artificial problems at TE zero-crossings. The reported tables include maximum and rms errors, the tolerance, pass/fail status, and the multipole or grid point of the worst sample. CPU and wall times are reported for the standard run, reference run, and any candidate runs tested during boost searches.

Common uses

Basic stability check for a standard parameter file:

camb check_accuracy inifiles/planck_2018.ini

Check spectra only up to a realistic analysis scale, while forcing CAMB to calculate that far first:

camb check_accuracy inifiles/planck_2018.ini --set-for-lmax 4000

Use higher lensing-potential accuracy, useful for high-accuracy lensed spectra:

camb check_accuracy inifiles/planck_2018.ini --set-for-lmax 4000 --lens-potential-accuracy 4

Make diagnostic plots of the fractional differences:

camb check_accuracy inifiles/planck_2018.ini --plot-dir accuracy_plots

Calculate a fiducial CMB delta chi-squared using a Simons Observatory-like noise model:

camb check_accuracy inifiles/planck_2018.ini --set-for-lmax 4000 --chi2 --chi2-config so

Search for the smallest top-level boost settings that pass against the reference calculation:

camb check_accuracy inifiles/planck_2018.ini --find-minimal-boosts

Then try to identify which underlying component accuracy parameters are most important, keeping AccuracyBoost=1:

camb check_accuracy inifiles/planck_2018.ini --find-minimal-boosts --refine-accuracy-components

This component refinement varies IntTolBoost and the lower-level component accuracy settings affected by AccuracyBoost. It does not vary lSampleBoost or lAccuracyBoost; those remain top-level boost parameters handled by --find-minimal-boosts.

Assess the physical effect of changing lensing reference settings without changing the comparison run:

camb check_accuracy inifiles/planck_2018.ini --reference-lens-potential-accuracy 8

Options

The command-line option list below is generated from the current parser, so it stays aligned with camb check_accuracy --help.

usage: camb check_accuracy [-h] [--no-validate] [--lmax LMAX]
                           [--set-for-lmax SET_FOR_LMAX]
                           [--lens-margin LENS_MARGIN]
                           [--lens-potential-accuracy LENS_POTENTIAL_ACCURACY]
                           [--reference-lens-margin REFERENCE_LENS_MARGIN]
                           [--reference-lens-potential-accuracy REFERENCE_LENS_POTENTIAL_ACCURACY]
                           [--mpk-npoints MPK_NPOINTS] [--mpk-kmin MPK_KMIN]
                           [--mpk-tolerance MPK_TOLERANCE]
                           [--derived-tolerance DERIVED_TOLERANCE]
                           [--plot-dir PLOT_DIR] [--chi2]
                           [--chi2-config {planck,so,custom}]
                           [--chi2-lmin CHI2_LMIN] [--chi2-lmax CHI2_LMAX]
                           [--chi2-fields CHI2_FIELDS]
                           [--noise-muk-arcmin-t NOISE_MUK_ARCMIN_T]
                           [--noise-muk-arcmin-p NOISE_MUK_ARCMIN_P]
                           [--beam-fwhm-arcmin BEAM_FWHM_ARCMIN] [--fsky FSKY]
                           [--find-minimal-boosts] [--exhaustive-boost-search]
                           [--refine-accuracy-components]
                           [--component-search-size COMPONENT_SEARCH_SIZE]
                           [--search-grid SEARCH_GRID]
                           [--search-tolerance SEARCH_TOLERANCE]
                           [--max-search-runs MAX_SEARCH_RUNS]
                           [--accuracy-boost ACCURACY_BOOST]
                           [--l-sample-boost L_SAMPLE_BOOST]
                           [--l-accuracy-boost L_ACCURACY_BOOST]
                           [--int-tol-boost INT_TOL_BOOST]
                           [--do-late-rad-truncation | --no-do-late-rad-truncation]
                           ini_file

Check stability of one CAMB ini file against a boosted-accuracy calculation.

positional arguments:
  ini_file              CAMB ini file to load with camb.read_ini

options:
  -h, --help            show this help message and exit
  --no-validate         pass no_validate=True to camb.read_ini
  --lmax LMAX           maximum L to compare; default is common calculated
                        lmax
  --set-for-lmax SET_FOR_LMAX
                        call params.set_for_lmax before running; if --lmax is
                        unset, this is also used for comparison
  --lens-margin LENS_MARGIN
                        lens margin to use for both runs; can be set with or
                        without --set-for-lmax
  --lens-potential-accuracy LENS_POTENTIAL_ACCURACY
                        lens_potential_accuracy to use for both runs; can be
                        set with or without --set-for-lmax
  --reference-lens-margin REFERENCE_LENS_MARGIN
                        lens margin override for the boosted reference only
  --reference-lens-potential-accuracy REFERENCE_LENS_POTENTIAL_ACCURACY
                        lens_potential_accuracy override for the boosted
                        reference only
  --mpk-npoints MPK_NPOINTS
                        number of k samples for matter power comparison grid
  --mpk-kmin MPK_KMIN   minimum k/h for matter power comparison
  --mpk-tolerance MPK_TOLERANCE
  --derived-tolerance DERIVED_TOLERANCE
  --plot-dir PLOT_DIR   write plots of fractional errors to this directory
  --chi2                calculate a fiducial Gaussian CMB delta chi-squared
                        for standard vs boosted C_L
  --chi2-config {planck,so,custom}
                        noise configuration for --chi2
  --chi2-lmin CHI2_LMIN
  --chi2-lmax CHI2_LMAX
                        maximum L for --chi2; defaults to comparison lmax
  --chi2-fields CHI2_FIELDS
  --noise-muk-arcmin-t NOISE_MUK_ARCMIN_T
                        temperature white noise for --chi2
  --noise-muk-arcmin-p NOISE_MUK_ARCMIN_P
                        polarization white noise for --chi2
  --beam-fwhm-arcmin BEAM_FWHM_ARCMIN
                        Gaussian beam FWHM for --chi2
  --fsky FSKY           sky fraction for --chi2
  --find-minimal-boosts
                        search for the lowest-cost boosted settings that pass
                        against the high-accuracy run
  --exhaustive-boost-search
                        continue boost search after a passing single-parameter
                        path is found
  --refine-accuracy-components
                        after --find-minimal-boosts, try AccuracyBoost=1 with
                        underlying accuracy fields tweaked instead
  --component-search-size COMPONENT_SEARCH_SIZE
                        maximum number of component accuracy fields to try
                        together
  --search-grid SEARCH_GRID
                        optional comma-separated seed values for the four
                        boost parameters; the default search does not need
                        this
  --search-tolerance SEARCH_TOLERANCE
                        target precision for refining numeric boost values
  --max-search-runs MAX_SEARCH_RUNS
                        maximum candidate runs for boost search
  --accuracy-boost ACCURACY_BOOST
  --l-sample-boost L_SAMPLE_BOOST
  --l-accuracy-boost L_ACCURACY_BOOST
  --int-tol-boost INT_TOL_BOOST
  --do-late-rad-truncation, --no-do-late-rad-truncation

Programmatic use

For scripts and tests that already have a camb.model.CAMBparams object, use camb.check_accuracy.compare_params_accuracy() directly rather than writing a temporary .ini file. The returned camb.check_accuracy.AccuracyCheckResult contains the standard and reference run outputs, comparison tables, timings, and optional chi-squared summary.

For example:

import camb
from camb.check_accuracy import compare_params_accuracy

params = camb.read_ini("inifiles/planck_2018.ini")
result = compare_params_accuracy(params, set_for_lmax=4000)

if not result.comparison.passed:
    print(result.comparison.worst_failure)

API reference

Check numerical stability of CAMB parameters against a higher-accuracy reference run.

The command-line interface compares spectra and derived parameters from an input .ini file with a reference calculation using boosted accuracy settings. It can also plot fractional differences, estimate a fiducial CMB delta chi-squared, search for minimal top-level boosts, and refine which underlying component accuracy settings are most relevant.

Examples:

camb check_accuracy inifiles/params.ini
camb check_accuracy inifiles/params.ini --plot-dir accuracy_plots
camb check_accuracy inifiles/params.ini --find-minimal-boosts --refine-accuracy-components

camb.check_accuracy.compare_params_accuracy(params, *, reference_accuracy_settings: dict[str, float | bool] | None = None, comparator_accuracy_settings: dict[str, float | bool] | None = None, lmax: int | None = None, set_for_lmax: int | None = None, lens_margin: int | None = None, lens_potential_accuracy: float | None = None, reference_lens_margin: int | None = None, reference_lens_potential_accuracy: float | None = None, mpk_kmin: float = 0.0001, mpk_npoints: int = 500, derived_tolerance: float = 0.001, mpk_tolerance: float | None = None, chi2_config: NoiseConfig | None = None) → AccuracyCheckResult[source]

Compare a CAMBparams object against a higher-accuracy copy.

The input parameters are copied before modification. The returned result includes both CAMB runs, row-wise comparison statistics, and optionally a fiducial CMB delta chi-squared estimate.

class camb.check_accuracy.AccuracyCheckResult(standard: 'RunOutput', reference: 'RunOutput', comparison: 'ComparisonResult', chi2: 'ChiSquaredResult | None' = None)[source]

class camb.check_accuracy.SearchResult(settings: 'dict[str, float | bool]', comparison: 'ComparisonResult', run: 'RunOutput | None' = None, fastest: 'PassingCandidate | None' = None)[source]

class camb.check_accuracy.ComparisonResult(derived_rows: 'list[StatRow]', cl_rows: 'list[StatRow]', lensing_rows: 'list[StatRow]', mpk_rows: 'list[StatRow]')[source]

class camb.check_accuracy.RunOutput(label: 'str', params: 'object', results: 'object', cpu_time: 'float', wall_time: 'float', derived: 'dict[str, float]', lensed_cls: 'np.ndarray | None', lens_potential_cls: 'np.ndarray | None', matter_power: 'MatterPowerData | None')[source]

class camb.check_accuracy.NoiseConfig(name: 'str', noise_muK_arcmin_T: 'float', noise_muK_arcmin_P: 'float', fwhm_arcmin: 'float', fsky: 'float', lmin: 'int', lmax: 'int | None', fields: 'tuple[str, ...]')[source]