statfidelity

statfidelity audits whether AI-mediated answers preserve the statistical meaning of official data. It is designed for generative AI, retrieval-augmented generation (RAG), statistical agents, and model-context-protocol style tool workflows without depending on a particular model vendor.

Authors and maintainer

Authors, in package citation order:

  1. Hossein Hassani
  2. Steve MacFeely
  3. Leila Marvian Mashhad

Maintainer: Leila Marvian Mashhad.

Relationship to GSBPM 5.2

statfidelity is designed to complement, not replace, the Generic Statistical Business Process Model (GSBPM) 5.2. GSBPM is a process-level reference model for the production of official statistics, covering activities from specifying needs and design through collection, processing, analysis, dissemination, and evaluation. statfidelity addresses a narrower executable problem at the statistical-claim level: whether the multidimensional meaning of an official statistical object is preserved when it is retrieved, transformed, summarised, or communicated through generative AI, retrieval-augmented generation (RAG), or statistical agents.

The closest operational alignment is with GSBPM activities related to Analyse, Disseminate, Evaluate, and overarching Quality Management. The package does not implement the GSBPM production process. Instead, it provides claim-level controls that can sit within or alongside those activities:

A useful distinction is: GSBPM provides the process-level quality architecture; statfidelity provides an executable AI claim-level fidelity-control layer.

Official GSBPM 5.2 information: https://unece.org/statistics/gsbpm-v5.2

This relationship is implemented directly in the package:

# Embedded offline mapping of relevant GSBPM 5.2 activities
gsbpm52_map()

# Where a fidelity dimension or package function fits
gsbpm_align("geographic")
gsbpm_align("audit_stat_ai")

# After an audit, create a GSBPM-oriented quality evidence report
# report <- gsbpm_quality_report(aud)
# report

The mapping deliberately uses the terms direct and supporting, not “compliant” or “certified”. GSBPM 5.2 is a reference model for statistical business processes; statfidelity supplies executable evidence that can support selected GSBPM activities.

What statfidelity 0.1.0 includes

Core audit

library(statfidelity)

ref <- stat_reference(
  value = 10.4,
  indicator = "unemployment rate",
  geo = "Spain",
  time = "2025",
  unit = "percent",
  source = "Eurostat",
  dataset = "example_lfs",
  population = "labour force"
)

claim <- stat_claim(
  value = 10.4,
  indicator = "unemployment rate",
  geo = "Spain",
  time = "2025",
  unit = "%",
  source = "Eurostat",
  dataset = "example_lfs",
  population = "labour force"
)

aud <- audit_stat_ai(ref, claim)
aud
proof <- certify_claim(aud)
verify_proof(proof)

Official-data connectors

The package builds requests against the current official provider endpoints documented by Eurostat, the World Bank and OECD. Network access happens only when a fetch_*() function is explicitly called. Session-local caching is enabled by default; it can be disabled with cache = FALSE or redirected with cache_dir.

# Eurostat URL only (offline)
eurostat_url("demo_pjan", list(geo = "ES", time = "2025", sex = "T"))

# World Bank URL only (offline)
worldbank_url("SP.POP.TOTL", country = "FRA", start = 2020, end = 2025)

# OECD URL only (offline)
oecd_url("OECD.SDD.STES", "DSD_STES@DF_CLI", start_period = "2025")

Live retrieval examples:

# Requires internet access
# wb <- fetch_worldbank("SP.POP.TOTL", country = "FRA", start = 2025, end = 2025)
# ref <- as_stat_reference(wb, row = 1, unit = "persons")

Search across official catalogues

# Requires internet access
# hits <- search_official_stats(
#   "unemployment",
#   providers = c("worldbank", "ons", "unsd_sdg")
# )
# hits

Extend statfidelity to another official JSON API

# Session-local registration; no secret is written to disk.
register_stat_api(
  "my_nso",
  base_url = "https://api.example.gov/v1",
  search_path = "search",
  query_param = "q",
  api_key_name = "api_key",
  provider_label = "Example National Statistical Office"
)

# Optional key for the current R process only:
# set_stat_api_key("my_nso", "YOUR_KEY")
# search_stat_api("my_nso", "population")

Additional built-in endpoints:

# UNSD SDG metadata search and series data
# search_unsd_sdg("social protection")
# sdg <- fetch_unsd_sdg("SI_COV_SOCINS", page_size = 100)

# UK ONS full-text search
# search_ons("unemployment", content_type = "dataset")

# BLS public single-series retrieval
# bls <- fetch_bls("LNS14000000", latest = TRUE)

# IMF SDMX URL builder
imf_api_url("dataflow/IMF/all/latest", version = "3.0")

Provider documentation used by this package:

Transformation fidelity

ref <- stat_reference(
  -0.7, "unemployment-rate change", "Exampleland", "2024-2025",
  "percentage points", "Official provider",
  transformation = list(
    operation = "percentage point change",
    inputs = c(11.1, 10.4), result = -0.7
  )
)

claim <- stat_claim(
  -6.31, "unemployment-rate change", "Exampleland", "2024-2025",
  "percent", "Official provider",
  transformation = list(
    operation = "percent change",
    inputs = c(11.1, 10.4), result = -6.31
  )
)

audit_stat_ai(ref, claim)

Offline benchmark tools

head(officialstat_benchmark())
benchmark_from_reference(ref)

CRAN-oriented design

Live API calls are not required by tests or runnable examples. Parser tests use local fixtures. Provider requests are bounded, cached in the session by default, and only occur after an explicit user call. Before CRAN submission, build the source tarball with the current R release or R-patched and run R CMD check --as-cran on that built tarball.