SACC Usage and Conventions

This guide explains how Firecrown uses the SACC format, the naming conventions that must be followed, and how to handle and fix common issues.

What is SACC?

SACC (Save All Correlations and Covariances) is a standardized format for storing measurements from cosmological surveys, including two-point correlation functions, SNIa observations, cluster counts, and other related cosmological probes. SACC files organize measurements from multiple tracers (e.g., tomographic bins) and provide a consistent interface for accessing data and metadata.

Note: This guide focuses on the naming conventions and best practices for two-point measurements specifically. While SACC supports diverse measurement types, the conventions described here are tailored to two-point correlations.

Key Features of SACC:

Tracers: Represent tomographic bins or other matter distribution tracers (e.g., galaxies in a redshift range, CMB convergence map)
Data Points: Individual measurements of correlations between pairs of tracers
Metadata: Including covariances, window functions, and measurement types
Data Types: Standardized strings that identify what kind of measurement is stored (e.g., galaxy_shear_xi_plus)

SACC Naming Convention

SACC defines a strict naming convention for the association between measurement types and tracers. This convention is essential for unambiguous interpretation of two-point measurements.

Important Notice: Firecrown will enforce this naming convention strictly in a future release. Currently, Firecrown provides compatibility features to allow loading older SACC files that do not fully comply with the convention. However, if you use such non-compliant files, you must be careful to understand the implications:

Non-compliant files may produce unexpected results if tracers and measurement types are misaligned
Auto-correction features are deprecated and will be removed
You should update your SACC files to comply with the convention
Future versions of Firecrown will reject non-compliant SACC files without an auto-correction option

Convention Rules

The fundamental rule is:

The order of measurement types in a SACC data type string must match the order of the tracers.

Example 1: Single Measurement Type (Same Type for Both Tracers)

When the two tracers have the same measurement type, the measurement type name appears only once in the data type string:

Consider the data type galaxy_shear_xi_plus measured for tracers bin_0 and bin_1:

Data Type:   galaxy_shear_xi_plus
Tracers:     (bin_0, bin_1)
Meaning:     Both bin_0 and bin_1 are galaxy shear measurements

Notice that “shear” appears only once in the data type string, even though both tracers are shear measurements. This is the naming convention for measurements between two tracers of the same type. Other examples include:

galaxy_density_xi - density auto-correlation
galaxy_shear_cl_ee - shear-shear correlation in harmonic space

In these cases, there is no ambiguity about which tracer corresponds to which measurement type, since both tracers have the same type.

Important: The tracer names (bin_0, bin_1) are not used to determine measurement types. The measurement type is determined entirely from the data type string and validated against the tracers involved. You could name tracers anything (e.g., alpha, beta) and the convention would still apply, both would still be shear measurements.

Example 2: Two Different Measurement Types

When the two tracers have different measurement types, both measurement type names appear in the data type string, in a specific order that must match the tracer order.

Consider the data type galaxy_shearDensity_cl measured for tracers bin_0 and bin_1:

Data Type:   galaxy_shearDensity_cl
Tracers:     (bin_0, bin_1)

The naming convention requires:

The first measurement type in the data type string (shear) corresponds to the first tracer (bin_0)
The second measurement type in the data type string (Density) corresponds to the second tracer (bin_1)

It would be an error to have them reversed (bin_1, bin_0), as SACC follows a canonical ordering (see next section).

Note: Again, the tracer names here (bin_0, bin_1) are arbitrary and chosen only to illustrate the concept. The measurement type determination is based on the data type string and the order of tracers in the measurement, not on their names.

Canonical Ordering in Firecrown

SACC proposes a canonical ordering of measurement types that determines the order in which tracers should appear in a measurement. This ordering is essential for unambiguous interpretation of data type strings:

CMB < Clusters < Galaxies

This ordering applies when combining measurements across different measurement type categories. For example:

CMB convergence measurements always come before cluster measurements
Cluster measurements always come before galaxy measurements
When two tracers have types from the same category, they are ordered according to the internal enum ordering within that category

Why Canonical Ordering Matters: The canonical ordering enables deterministic construction of data type strings. For example, if you have one CMB tracer and one galaxy tracer, the only valid data type string is cmbGalaxy_convergenceDensity_cl (CMB first), not galaxyCMB_densityConvergence_cl. Moreover, since the measurement type order in the data type string must match the tracer order (see convention examples above), the tracers must be ordered such that the first tracer’s type comes before the second tracer’s type in the canonical ordering.

Internal Enum Ordering: Within each measurement type category (CMB, Clusters, Galaxies), the measurement types are also ordered canonically. For example, within Galaxies:

Galaxies.SHEAR_E < Galaxies.SHEAR_T < … < Galaxies.COUNTS

This means:

When determining tracer order for cross-measurements (e.g., shear cross-correlation with counts), shear measurements always come before count measurements
CMB and cluster measurement types are ordered internally according to their positions in their respective enums
Firecrown applies this internal ordering automatically to ensure consistency

Firecrown’s Ordering Support: Firecrown provides utilities to help you determine the correct tracer order based on the canonical ordering. When creating or fixing SACC files, you can use Firecrown’s ordering functions to automatically compute the correct order of tracers for a given pair of measurement types.

Understanding Measurement Types

Firecrown recognizes the following measurement types:

Galaxy Measurements:

SHEAR_E or SHEAR_T: Weak lensing shear components
COUNTS: Galaxy number counts
PART_OF_XI_MINUS or PART_OF_XI_PLUS: Components of two-point shear correlation functions

CMB Measurements:

CONVERGENCE: CMB lensing convergence

Cluster Measurements:

COUNTS: Cluster abundance

Measurement type associations are stored in the SACC file via the data type strings. Firecrown uses the MEASURED_TYPE_STRING_MAP to decode which measurement types are involved in each measurement.

Single-Type vs. Two-Type Measurements

Measurement Type Sets: In Firecrown, measurements are organized into sets based on their conceptual role. For galaxies, these include:

GALAXY_SOURCE_TYPES: Shear measurements (SHEAR_E, SHEAR_T, PART_OF_XI_MINUS, PART_OF_XI_PLUS)
GALAXY_LENS_TYPES: Galaxy counts (COUNTS)
CMB_TYPES: CMB convergence (CONVERGENCE)
CLUSTER_TYPES: Cluster abundance

A tracer may have multiple measurement types as long as they belong to the same measurement set. For example, a galaxy source tracer might have both SHEAR_T and PART_OF_XI_MINUS measurements, since both are part of GALAXY_SOURCE_TYPES. However, mixing types from different sets (e.g., shear and counts) in the same tracer is not allowed by default.

Swap Detection and Auto-Correction (Deprecated): When Firecrown processes a SACC file, it may detect that a tracer was mistakenly assigned to the wrong measurement set. If tracers are reversed (e.g., a source tracer mixed with a lens tracer) in a measurement, Firecrown will:

Test if swapping the tracers would fix the issue
If a swap corrects the assignment, emit a DeprecationWarning and perform the swap
Apply the corrected measurement types to the tracers

Important: This auto-correction via tracer swapping is deprecated and will be removed in a future release. When strict enforcement is enabled:

If a tracer has mixed types from different sets and allow_mixed_types=False, Firecrown will raise an error instead of attempting to auto-correct
You should fix your SACC file to ensure tracers are properly ordered according to the measurement types in the data type string

Single-Type Measurements: When both tracers belong to the same measurement set and have compatible types (e.g., both are sources or both are lenses), there is typically no ambiguity.

Two-Type Measurements: When the two tracers belong to different measurement sets or have incompatible types (e.g., source paired with lens), the order of tracers must match the order of measurement types in the data type string, as explained in the convention examples above.

Handling Mixed-Type Measurements

What are Mixed-Type Measurements?: In the context of Firecrown, “mixed-type measurements” refers to a tracer being associated with measurement types from different measurement sets (e.g., both source types and lens types).

Within the same set is allowed: A tracer may have multiple measurement types as long as they belong to the same measurement set. For example:

A source tracer can have both SHEAR_T and PART_OF_XI_MINUS (both in GALAXY_SOURCE_TYPES)
A lens tracer can only have COUNTS (the only type in GALAXY_LENS_TYPES)

However, mixing types from different sets in the same tracer (e.g., shear and counts) is not permitted by default.

Use Cases for Mixed-Type Measurements: In specialized analyses, you might intentionally want to allow a single tracer to contain measurements from different sets, though this is unusual. For example, a tracer might contain both galaxy shear and galaxy count measurements in a joint analysis.

Using `allow_mixed_types` Parameter

When loading SACC data with extract_all_measured_types(), you can control whether mixed-type measurements (types from different sets) are permitted:

from firecrown.metadata_functions import extract_all_measured_types

# Strict mode (default): raises error if a tracer has types from different sets
measured_types = extract_all_measured_types(sacc_data, allow_mixed_types=False)

# Permissive mode: allows tracers to have types from different measurement sets
measured_types = extract_all_measured_types(sacc_data, allow_mixed_types=True)

By default, allow_mixed_types=False, which raises a ValueError if a tracer is found to have measurement types from different sets. This strict behavior helps identify potential inconsistencies and ensures proper separation between source and lens tracers.

Note: If allow_mixed_types=False and Firecrown detects reversed tracers that could be fixed by swapping, it will emit a DeprecationWarning and perform the swap. However, if swapping cannot fix the issue, an error will be raised. This auto-correction via tracer swapping is deprecated and will be removed in a future release.

Fixing SACC Convention Violations

Common Issues

The most common SACC convention violation occurs when tracers are misaligned with measurement types due to reversed tracer order:

Scenario: You have a CMB convergence tracer (cmb_bin_0) and a galaxy counts tracer (lens_bin_0), but they are reversed in one of your data type strings.

Correct:     (cmb_bin_0, lens_bin_0)  with data type "cmbGalaxy_convergenceDensity_cl"
Incorrect:   (lens_bin_0, cmb_bin_0)  with same data type "cmbGalaxy_convergenceDensity_cl"

The data type string cmbGalaxy_convergenceDensity_cl specifies that the measurement is CMB convergence paired with galaxy counts, and the canonical ordering requires CMB to come first (CMB < Galaxies). In the incorrect case, the tracers are reversed: the first tracer is galaxy counts, but the data type string claims it should be CMB convergence.

Detection and Auto-Correction (Deprecated): Firecrown can detect this error when other measurements involving these tracers provide evidence of the correct ordering. For example, if you have auto-correlations like:

cmb_bin_0 with cmb_bin_0 in a cmb_convergence_cl measurement
lens_bin_0 with lens_bin_0 in a galaxy_density_cl measurement

Then Firecrown can infer the correct types for each tracer and detect that the cross-measurement tracers are reversed. When this happens with allow_mixed_types=False, Firecrown will:

Emit a DeprecationWarning explaining the issue
Automatically swap the tracers to fix the ordering
Continue processing with the corrected assignment

Important: This automatic tracer swapping is deprecated and will be removed in a future release. You should fix your SACC file manually rather than relying on this auto-correction.

Deprecated Auto-Correction

When allow_mixed_types=False (the default), Firecrown will attempt to auto-correct simple convention violations by swapping tracer labels. This is done as a convenience for legacy SACC files that don’t follow the convention.

Important: This auto-correction behavior is deprecated and will be removed in a future release when Firecrown enforces strict SACC compliance. You should fix your SACC file now instead of relying on this auto-correction. Once strict enforcement is enabled, non-compliant SACC files will cause Firecrown to raise an error, and users will have to manually fix their files.

The auto-correction process works as follows:

Firecrown detects that assigning the measurement types in their original tracer order would create mixed-type measurements (measurements with types from different sets)
Firecrown tests whether swapping the tracers would resolve the issue
If swapping would fix the problem, Firecrown performs the swap and emits a DeprecationWarning with detailed diagnostic information
Processing continues with the corrected tracer assignment

When auto-correction is performed, Firecrown issues a DeprecationWarning structured as follows:

SACC Convention Violation Detected (DEPRECATED AUTO-FIX)

Firecrown detected an inconsistency in how measurement types are assigned to tracers.
Specifically, assigning measurement type 'SHEAR_E' to tracer 'lens0' and measurement
type 'COUNTS' to tracer 'src0' would create mixed-type measurements (multiple distinct
measurement types from different sets in the same tomographic bin).

The data type string 'galaxy_shearDensity_cl' follows the SACC naming convention, where
the order of measurement types in the string must match the order of tracers. However,
your SACC file/object appears to violate this convention.

AUTO-CORRECTION PERFORMED
Because allow_mixed_types=False (the default), Firecrown attempted to correct this by
swapping the tracer assignment, assuming the tracers were simply misaligned. This auto-
correction is a convenience feature for legacy SACC files that don't follow the
convention.

⚠️  DEPRECATION NOTICE ⚠️
This automatic correction will be REMOVED in a future release. Going forward, files
that violate the SACC convention will be interpreted as genuinely mixed-type
measurements and will either raise an error (if allow_mixed_types=False) or be
processed as-is (if allow_mixed_types=True).

RECOMMENDED ACTION
To future-proof your code, fix your SACC file to follow the naming convention. See the
documentation for detailed instructions:
    https://firecrown.readthedocs.io/en/latest/sacc_usage.html

Using Firecrown CLI for Quality Control

Firecrown provides command-line tools to detect and fix SACC convention violations automatically.

Step 1: Check Your SACC File

Use the view command with the --check flag to validate your SACC file:

firecrown sacc view data.sacc --check

This command will:

Display file contents (tracers, measurements, covariance)
Run quality checks for naming convention compliance
Detect tracer ordering issues
Report any violations with detailed diagnostics

Example Output:

──────────────────────────────────── Loading SACC file ─────────────────────────────────────
File: sacc_data.sacc
Allow mixed types: False
─────────────────────────────────────── SACC Summary ───────────────────────────────────────
╭────── SACC Summary ──────╮
│ Number of tracers: 9     │
│ Data points: 457         │
│ Covariance elements: 457 │
╰──────────────────────────╯
───────────────────────────────────────── Tracers ──────────────────────────────────────────
┏━━━━━━━┳━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┓
┃ Name  ┃ TypeSource ┃ z min-max (density) ┃ dndz mean, std  ┃ Measurements                ┃
┡━━━━━━━╇━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┩
│ lens0 │ default    │ 0.005-3.995         │ 0.240 +/- 0.044 │ {COUNTS}                    │
│ lens1 │ default    │ 0.005-3.995         │ 0.383 +/- 0.050 │ {COUNTS}                    │
│ ...   │ ...        │ ...                 │ ...             │ ...                         │
│ src0  │ default    │ 0.005-3.995         │ 0.390 +/- 0.250 │ {SHEAR_T, PART_OF_XI_MINUS, │
│       │            │                     │                 │ PART_OF_XI_PLUS}            │
└───────┴────────────┴─────────────────────┴─────────────────┴─────────────────────────────┘
...(bin combinations and summary omitted)...
─────────────────────────────────── SACC Quality Checks ────────────────────────────────────
⚠️  Found 20 tracer naming convention violation(s)
  • Tracers 'lens0' and 'src0' (data type: 'galaxy_shearDensity_xi_t')
  • Tracers 'lens0' and 'src1' (data type: 'galaxy_shearDensity_xi_t')
  • Tracers 'lens0' and 'src2' (data type: 'galaxy_shearDensity_xi_t')
  • Tracers 'lens0' and 'src3' (data type: 'galaxy_shearDensity_xi_t')
  ...(16 more violations)...

Recommendation: Use 'firecrown sacc transform --fix-ordering' to correct.

Step 2: Fix Ordering Issues

Use the transform command with --fix-ordering to automatically correct tracer ordering:

firecrown sacc transform data.sacc --fix-ordering --overwrite

This command will:

Read the SACC file
Detect tracer ordering violations
Swap tracers to match the data type string convention
Write the corrected file back (with --overwrite)
Display a summary of corrections

Example Output:

Detected input format: HDF5
Reading HDF5 file: sacc_data.sacc
Fixing tracer ordering for 20 unique corrections.
For data type galaxy_shearDensity_xi_t and tracers lens0, src0, 6 data points were flipped.
For data type galaxy_shearDensity_xi_t and tracers lens0, src1, 6 data points were flipped.
For data type galaxy_shearDensity_xi_t and tracers lens0, src2, 6 data points were flipped.
For data type galaxy_shearDensity_xi_t and tracers lens0, src3, 6 data points were flipped.
For data type galaxy_shearDensity_xi_t and tracers lens1, src0, 8 data points were flipped.
...(15 more corrections)...
For data type galaxy_shearDensity_xi_t and tracers lens4, src3, 11 data points were flipped.
Writing SaccFormat.HDF5 file: sacc_data.sacc

============================================================
✅ Transformation successful!
============================================================
Input:  sacc_data.sacc (HDF5, 1,679,790 bytes)
Output: sacc_data.sacc (HDF5, 1,653,400 bytes)
Size reduction: 1.6%

Step 3: Verify the Fix

Run the check again to confirm all issues are resolved:

firecrown sacc view data.sacc --check

Example Output:

──────────────────────────────────── Loading SACC file ─────────────────────────────────────
File: sacc_data.sacc
Allow mixed types: False
...(file contents omitted)...
─────────────────────────────────── SACC Quality Checks ────────────────────────────────────
✅ All quality checks passed!

Additional Options:

--output <path>: Specify output file (default: overwrites input if formats match)
--output-format {fits|hdf5}: Convert format while fixing
--input-format {fits|hdf5}: Force input format detection

Example: Fix and Convert Format:

firecrown sacc transform data.fits --fix-ordering --output-format hdf5 --output data.h5

How to Fix Your SACC File Manually

If you prefer to fix SACC convention violations manually, you need to reorder tracers in your data type entries so that they match the measurement type order in the data type string.

Manual Approach:

Identify which measurements have convention violations (look for DeprecationWarning messages)
Understand which measurement types correspond to which tracers
Reorder tracers in the affected data type entries to match the measurement type order in the data type string
Save the corrected SACC file

Example Fix:

Suppose you have this incorrect entry:

# Incorrect: tracers are (cmb, shear) but data type implies (shear, cmb)
sacc_data.add_data_point("cmbGalaxy_convergenceShear_cl_e",
                          ("shear_bin_0",   # First tracer (should be cmb)
                          "cmb_bin_0"),     # Second tracer (should be shear)
                          ...)

To fix it, swap the tracer order:

# Correct: tracers are (shear, cmb) matching the data type
sacc_data.add_data_point("cmbGalaxy_convergenceShear_cl_e",
                          ("cmb_bin_0",     # First tracer (cmb) ✓
                          "shear_bin_0"),   # Second tracer (shear) ✓
                          ...)

Programmatic Approach:

For large SACC files with many violations, a dedicated repair tool (to be provided in a future release) will automate this process. In the meantime, you can write a Python script to:

Load the SACC file
Identify violations using Firecrown’s extraction functions
Reorder tracer entries as needed
Save the corrected file

Best Practices

Follow the Convention from the Start

When creating SACC files, follow the naming convention from the beginning:
- Use clear, descriptive tracer names that indicate their measurement type
- Order tracers in data type entries to match the order in the data type string
- Test your SACC file with Firecrown using allow_mixed_types=False to validate compliance
- When writing a SACC file you can use Firecrown ordering support to determine automatically the correct order.

Use Descriptive Tracer Names

Make it obvious what each tracer represents:

Good:    shear_bin_0, cmb_convergence_bin_0, cluster_counts_bin_0
Unclear: tracer_0, tracer_1, bin_0

Document Your SACC File

Include comments in your code describing: - What each tracer represents - Why mixed-type measurements are used (if applicable) - Any deviations from the standard convention

Validate Early and Often

Use the CLI to validate your SACC files before using them in analyses:

# Check for quality issues
firecrown sacc view my_data.sacc --check

# Fix any detected issues
firecrown sacc transform my_data.sacc --fix-ordering --overwrite

# Verify the fix
firecrown sacc view my_data.sacc --check

Or programmatically:

import warnings
from firecrown.metadata_functions import extract_all_measured_types

# Catch deprecation warnings about convention violations
with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter("always", DeprecationWarning)
    measured_types = extract_all_measured_types(sacc_data, allow_mixed_types=False)

    if w:
        print("Convention violations detected:")
        for warning in w:
            print(f"  {warning.message}")

Automate Quality Checks in Your Pipeline

Include SACC validation in your data processing pipeline:

#!/bin/bash
# Example pipeline script

# Generate SACC file
python generate_sacc.py --output raw_data.sacc

# Validate and fix
firecrown sacc view raw_data.sacc --check || \
    firecrown sacc transform raw_data.sacc --fix-ordering --overwrite

# Final verification
firecrown sacc view raw_data.sacc --check

# Proceed with analysis
firecrown examples des_y1_3x2pt ./output --sacc raw_data.sacc

Command Reference

firecrown sacc view

Inspect and validate SACC files.

Syntax:

firecrown sacc view SACC_FILE [OPTIONS]

Options:

--check: Run quality checks for naming convention compliance
--plot-covariance: Display covariance matrix visualization
--allow-mixed-types: Allow tracers with mixed measurement types (default: False)
--help: Show command help

Examples:

# Basic inspection
firecrown sacc view data.sacc

# With quality checks
firecrown sacc view data.sacc --check

# Full inspection with visualization
firecrown sacc view data.sacc --check --plot-covariance

firecrown sacc transform

Transform SACC files: update internal format, fix ordering, or convert formats.

Syntax:

firecrown sacc transform SACC_FILE [OPTIONS]

Options:

--output PATH or -o PATH: Output file path (default: same as input)
--input-format {fits|hdf5} or -f: Force input format
--output-format {fits|hdf5} or -t: Output format (default: same as input)
--fix-ordering: Fix tracer ordering issues
--overwrite: Overwrite output file if it exists
--allow-mixed-types: Allow tracers with mixed measurement types (default: False)
--help: Show command help

Examples:

# Update internal format in-place
firecrown sacc transform data.sacc --overwrite

# Fix tracer ordering
firecrown sacc transform data.sacc --fix-ordering --overwrite

# Convert FITS to HDF5
firecrown sacc transform data.fits --output-format hdf5 -o data.h5

# Fix ordering and convert format
firecrown sacc transform data.fits --fix-ordering --output-format hdf5 -o data.h5

# Create backup before fixing
firecrown sacc transform data.sacc --fix-ordering -o data_fixed.sacc

Getting Help

For more information about SACC:

For Firecrown-specific questions:

firecrown.metadata_functions.extract_all_measured_types()
firecrown.metadata_functions.extract_all_tracers_inferred_galaxy_zdists()
firecrown.app.sacc.View - CLI view command
firecrown.app.sacc.Transform - CLI transform command
Firecrown GitHub Issues and Discussions

SACC Usage and Conventions

What is SACC?

SACC Naming Convention

Convention Rules

Example 1: Single Measurement Type (Same Type for Both Tracers)

Example 2: Two Different Measurement Types

Canonical Ordering in Firecrown

Understanding Measurement Types

Single-Type vs. Two-Type Measurements

Handling Mixed-Type Measurements

Using allow_mixed_types Parameter

Fixing SACC Convention Violations

Common Issues

Deprecated Auto-Correction

Using Firecrown CLI for Quality Control

How to Fix Your SACC File Manually

Best Practices

Command Reference

firecrown sacc view

firecrown sacc transform

Getting Help

See Also

Using `allow_mixed_types` Parameter