Package {statfidelity}

Audit Statistical Fidelity of AI-Mediated Official Statistics

Description

Tools for representing official-statistics references and AI-generated claims, auditing multiple dimensions of statistical fidelity, applying critical-error rules, creating proof bundles, tracing provenance, retrieving bounded official data from supported providers, and generating robustness and red-team tests.

Details

The package is model-provider neutral and performs no mandatory network calls. Live provider access occurs only through explicit fetch functions. It can be used with outputs from generative AI, retrieval-augmented generation, or agentic systems after those outputs are represented as stat_claim objects. It complements process-level official-statistics quality frameworks such as GSBPM 5.2 by providing an executable claim-level fidelity-control layer rather than a replacement statistical production model. The package also provides offline GSBPM mapping and quality-report helpers via gsbpm52_map(), gsbpm_align(), and gsbpm_quality_report().

Author(s)

Hossein Hassani, Steve MacFeely, and Leila Marvian Mashhad

Convert Provider Data to an Official Statistical Reference

Description

Converts one canonical provider row into a structured official-statistics reference while preserving the source row and retrieval provenance.

Usage

as_stat_reference(x, row = 1L, indicator = NULL, geo = NULL, time = NULL,
  unit = NULL, source = NULL, dataset = NULL, population = NULL,
  adjustment = NULL, frequency = NULL, methodology = NULL, revision = NULL,
  quality_flag = NULL, uncertainty = NULL, transformation = NULL)

Arguments

Value

A stat_reference object.

Component-Level Statistical Fidelity Audits

Description

Compares individual semantic, numerical, temporal, geographic, unit, provenance, revision, and uncertainty dimensions of an AI-mediated claim against an official-statistics reference.

Usage

semantic_audit(reference, claim, aliases = NULL, minimum_similarity = 0.65)
numeric_audit(reference, claim, abs_tol = 1e-08, rel_tol = 1e-06)
temporal_audit(reference, claim)
geo_audit(reference, claim, aliases = NULL)
unit_audit(reference, claim)
source_audit(reference, claim, missing_score = 0.4)
revision_audit(reference, claim)
uncertainty_audit(reference, claim)

Arguments

Value

A stat_audit_component object containing score, status, critical flag, error code, expected value, observed value, and details.

Examples

ref <- stat_reference(10.4, "unemployment rate", "Spain", "2025",
                      "percent", "Eurostat")
clm <- stat_claim(10.4, "unemployment rate", "Spain", "2025",
                  "%", "Eurostat")
numeric_audit(ref, clm)
unit_audit(ref, clm)

Audit and Summarize Statistical Fidelity

Description

Runs the full ten-component audit and applies geometric composite scoring plus non-compensatory critical-error rules. The package also records the audit's intended alignment with relevant GSBPM 5.2 Analyse, Quality Management, and Metadata Management activities.

Usage

audit_stat_ai(reference, claim, indicator_aliases = NULL,
  geo_aliases = NULL, numeric_abs_tol = 1e-08,
  numeric_rel_tol = 1e-06, weights = NULL,
  pass_threshold = 0.90, weak_threshold = 0.75)

fidelity(components, weights = NULL, pass_threshold = 0.90,
  weak_threshold = 0.75)

audit_batch(references, claims, ids = NULL, ...)

weakest_link(x)

Arguments

Value

audit_stat_ai() returns a stat_fidelity_audit with ten component audits, the certification decision, and an embedded GSBPM 5.2 alignment record; audit_batch() returns a batch summary data frame; fidelity() returns summary scores; weakest_link() returns the lowest component score.

Examples

ref <- stat_reference(10.4, "unemployment rate", "Spain", "2025",
                      "percent", "Eurostat")
clm <- stat_claim(10.4, "unemployment rate", "Spain", "2025",
                  "%", "Eurostat")
aud <- audit_stat_ai(ref, clm)
aud
weakest_link(aud)

Official Statistics AI Benchmark Tools

Description

Creates adversarial statistical minimal pairs and loads the offline OfficialStatAI-Bench starter cases.

Usage

statistical_minimal_pairs(reference)
benchmark_from_reference(reference)
officialstat_benchmark()

Arguments

Value

A data frame for minimal pairs or the packaged starter benchmark; benchmark_from_reference() returns a list.

Examples

ref <- stat_reference(5.1, "unemployment rate", "Exampleland", "2025", "percent", "NSO")
statistical_minimal_pairs(ref)
head(officialstat_benchmark())

Audit Statistical Context and Revision Propagation

Description

Audits dataset and methodological context and measures revision propagation lag, including right-censored lag when an AI system has not yet reflected a revision.

Usage

context_audit(reference, claim, missing_score = 0.5,
  methodology_min_similarity = 0.35)
revision_propagation_lag(official_revised_at, ai_reflected_at = NULL,
  observed_at = Sys.time())

Arguments

Value

context_audit() returns a component audit. revision_propagation_lag() returns a one-row data frame.

Extract Candidate Statistical Claims from Text

Description

Provides a conservative built-in numerical extractor and a parser interface for advanced NLP or LLM-based claim extraction without tying the package to any model vendor.

Usage

extract_stat_claims(text, hints = list(), parser = NULL)

Arguments

Value

A list of stat_claim objects.

Examples

extract_stat_claims("The rate was 10.4% in 2025.",
  hints = list(indicator = "rate", geo = "Example", time = "2025"))

Align statfidelity with GSBPM 5.2

Description

Offline helpers that map statfidelity's AI-mediated claim-level quality controls to relevant phases, sub-processes and overarching activities in the Generic Statistical Business Process Model (GSBPM) version 5.2.

Usage

gsbpm52_map(relationship = c("all", "direct", "supporting"))

gsbpm_align(x)

gsbpm_quality_report(audit)

Arguments

Details

The mapping is intended to make the relationship to GSBPM 5.2 explicit and auditable. It does not claim that statfidelity replaces the GSBPM or that using the package constitutes official UNECE certification of GSBPM conformance. The principal direct relationships are with Analyse 6.2 (Validate outputs), 6.5 (Finalise outputs), Disseminate 7.2/7.5, Evaluate 8.1/8.2, and the overarching Quality and Metadata Management activities.

Value

gsbpm52_map() and gsbpm_align() return data frames. gsbpm_quality_report() returns a stat_gsbpm_report object containing component-level evidence, the audit decision, and mapped GSBPM activities.

Examples

gsbpm52_map("direct")
gsbpm_align("semantic")
gsbpm_align("audit_stat_ai")

Official-Statistics API Connectors

Description

Build URLs and retrieve bounded data from Eurostat, the World Bank Indicators API, and the OECD Data Explorer SDMX API.

Usage

eurostat_url(dataset, filters = list(), lang = "EN")
fetch_eurostat(dataset, filters = list(), lang = "EN", cache = TRUE,
  cache_dir = NULL, max_age = 3600, timeout = 30, max_cells = 1e6)
worldbank_url(indicator, country = "all", start = NULL, end = NULL,
  source = NULL, page = 1L, per_page = 1000L, metadata = FALSE)
fetch_worldbank(indicator, country = "all", start = NULL, end = NULL,
  source = NULL, per_page = 1000L, include_metadata = TRUE, cache = TRUE,
  cache_dir = NULL, max_age = 3600, timeout = 30)
oecd_url(agency, dataset, selection = "all", version = NULL,
  start_period = NULL, end_period = NULL, labels = TRUE)
fetch_oecd(agency, dataset, selection = "all", version = NULL,
  start_period = NULL, end_period = NULL, labels = TRUE, cache = TRUE,
  cache_dir = NULL, max_age = 3600, timeout = 30)

Arguments

Details

Network access occurs only after an explicit call to a fetch function. Tests use local fixtures rather than provider endpoints.

Value

URL builders return a character string. Fetchers return a stat_provider_data data frame.

Examples

eurostat_url("demo_pjan", list(geo = "ES", time = "2025"))
worldbank_url("SP.POP.TOTL", country = "FRA", start = 2020, end = 2025)
oecd_url("OECD.SDD.STES", "DSD_STES@DF_CLI", start_period = "2025")
## Not run: 
wb <- fetch_worldbank("SP.POP.TOTL", country = "FRA", start = 2025, end = 2025)

## End(Not run)

Write, Read, and Verify Statistical Proof Bundles

Description

Serializes Proof-Carrying Statistical Answers as canonical JSON and verifies their SHA-256 payload fingerprint.

Usage

write_proof(proof, path, pretty = TRUE)
read_proof(path)
verify_proof(proof)

Arguments

Value

Writing returns the path invisibly; reading returns a proof list; verification returns hash validity diagnostics.

Provenance Graphs and Proof-Carrying Statistical Answers

Description

Creates traceable relationships between source statistics, transformations, and AI claims, and packages audit evidence into a reproducible proof object.

Usage

claim_graph(reference, claim)
trace_claim(graph)
proof_bundle(audit)
certify_claim(audit, require_pass = FALSE)

Arguments

Value

A lightweight graph list, or a stat_proof carrying a SHA-256 fingerprint and audit status.

Examples

ref <- stat_reference(10.4, "unemployment rate", "Spain", "2025",
                      "percent", "Eurostat")
clm <- stat_claim(10.4, "unemployment rate", "Spain", "2025",
                  "%", "Eurostat")
aud <- audit_stat_ai(ref, clm)
g <- claim_graph(ref, clm)
trace_claim(g)
certify_claim(aud)

Search multiple official-statistics catalogues

Description

Runs explicit API/catalogue searches across supported official providers and combines ranked metadata results without treating unavailable providers as ground-truth failures.

Usage

search_official_stats(query,
  providers = c("worldbank", "ons", "unsd_sdg"), limit = 10L,
  strict = FALSE, cache = TRUE, cache_dir = NULL, max_age = 3600,
  timeout = 30)

Arguments

Value

A stat_search_results data frame. When strict = FALSE, provider errors are preserved in the provider_errors attribute.

Examples

# Requires internet access:
# search_official_stats("unemployment", limit = 5)

Additional official-statistics API connectors and search tools

Description

Adds metadata discovery and bounded retrieval for official World Bank, UNSD SDG, UK ONS, and U.S. BLS services. The IMF helper builds URLs for the current IMF SDMX 2.1 and 3.0 services while leaving dataset-specific resource paths to the caller.

Usage

search_worldbank(query, source = NULL, limit = 20L, per_page = 1000L,
  max_pages = 20L, cache = TRUE, cache_dir = NULL, max_age = 86400,
  timeout = 30)

ons_search_url(query, content_type = "dataset", limit = 20L, offset = 0L,
  filters = list())

search_ons(query, content_type = "dataset", limit = 20L, offset = 0L,
  filters = list(), cache = TRUE, cache_dir = NULL, max_age = 3600,
  timeout = 30)

unsd_sdg_series_url(all_releases = FALSE)

unsd_sdg_data_url(series_code, page = 1L, page_size = 1000L,
  release_code = NULL)

search_unsd_sdg(query, limit = 20L, all_releases = FALSE, cache = TRUE,
  cache_dir = NULL, max_age = 86400, timeout = 30)

fetch_unsd_sdg(series_code, page = 1L, page_size = 1000L, max_pages = 1L,
  release_code = NULL, cache = TRUE, cache_dir = NULL, max_age = 3600,
  timeout = 30)

bls_url(series_id, latest = FALSE)

fetch_bls(series_id, latest = FALSE, cache = TRUE, cache_dir = NULL,
  max_age = 3600, timeout = 30)

imf_api_url(path, version = c("3.0", "2.1"), query = list())

Arguments

Value

URL builders return character URLs. Search functions return stat_search_results. Data retrieval functions return stat_provider_data.

Examples

ons_search_url("unemployment", content_type = "dataset")
unsd_sdg_series_url()
unsd_sdg_data_url("SI_COV_SOCINS", page_size = 100)
bls_url("LNS14000000", latest = TRUE)
imf_api_url("dataflow/IMF/all/latest", version = "3.0")

# Live examples require network access and are not run automatically.
# search_worldbank("unemployment rate")
# search_ons("unemployment")
# search_unsd_sdg("social protection")
# fetch_bls("LNS14000000", latest = TRUE)

Extensible REST API support and session-only API keys

Description

Provides explicit GET-only access to additional official JSON APIs without scraping HTML. Custom registrations are session-local and never store secrets. API keys can be supplied directly or through environment variables named STATFIDELITY_<PROVIDER>_API_KEY.

Usage

stat_api_key(provider, key = NULL, required = FALSE)

set_stat_api_key(provider, key)

clear_stat_api_key(provider)

api_get_json(base_url, path = NULL, query = list(), provider = "custom",
  api_key = NULL, api_key_name = NULL, cache = TRUE, cache_dir = NULL,
  max_age = 3600, timeout = 30)

register_stat_api(name, base_url, search_path = NULL, query_param = "q",
  api_key_name = NULL, provider_label = name, overwrite = FALSE)

official_stat_providers(include_custom = TRUE)

fetch_stat_api(name, path, params = list(), parser = NULL, api_key = NULL,
  cache = TRUE, cache_dir = NULL, max_age = 3600, timeout = 30)

search_stat_api(name, query, params = list(), parser = NULL, api_key = NULL,
  cache = TRUE, cache_dir = NULL, max_age = 3600, timeout = 30)

Arguments

Value

API-key helpers return a key or invisibly return the environment-variable name. api_get_json() returns a stat_api_response. Registered fetch and search functions return parser output or a best-effort data frame.

Examples

official_stat_providers()

register_stat_api(
  "example_nso",
  base_url = "https://api.example.gov/v1",
  search_path = "search",
  query_param = "q"
)

# Network calls are intentionally not run in examples.
# search_stat_api("example_nso", "population")

Construct an AI-Mediated Statistical Claim

Description

Creates a structured representation of a statistical claim produced or mediated by an AI system.

Usage

stat_claim(value = NA_real_, indicator = NA_character_, geo = NA_character_,
  time = NA_character_, unit = NA_character_, source = NA_character_,
  dataset = NULL, population = NULL, adjustment = NULL, frequency = NULL,
  methodology = NULL, revision = NULL, quality_flag = NULL,
  uncertainty = NULL, transformation = NULL, answer = NULL,
  metadata = list())

Arguments

Value

An object of class stat_claim.

Examples

clm <- stat_claim(10.4, "unemployment rate", "Spain", "2025",
                  "%", "Eurostat")
clm

Construct an Official-Statistics Reference Object

Description

Creates a structured canonical representation of one official statistic used as ground truth for fidelity auditing.

Usage

stat_reference(value, indicator, geo, time, unit, source,
  dataset = NULL, population = NULL, adjustment = NULL,
  frequency = NULL, methodology = NULL, revision = NULL,
  quality_flag = NULL, uncertainty = NULL, transformation = NULL,
  retrieved_at = Sys.time(), source_url = NULL, version = NULL,
  metadata = list())

Arguments

Value

An object of class stat_reference.

Examples

ref <- stat_reference(10.4, "unemployment rate", "Spain", "2025",
                      "percent", "Eurostat")
ref

Prompt Robustness, Statistical Red-Teaming, and Model Stability

Description

Utilities for testing prompt sensitivity, generating official-statistics failure traps, and measuring cross-model answer stability.

Usage

perturb_prompt(prompt, n = 8L, include_original = TRUE)
redteam_stats(reference)
model_stability(results, item_col = "item", model_col = "model",
  answer_col = "answer")

Arguments

Value

Prompt variants, a red-team data frame, or a stability summary list.

Examples

perturb_prompt("What is the unemployment rate in Spain in 2025?", n = 3)
ref <- stat_reference(10.4, "unemployment rate", "Spain", "2025",
                      "percent", "Eurostat")
head(redteam_stats(ref))

Audit a Derived Statistical Transformation

Description

Checks whether the AI used the required statistical operation and, when possible, whether the derived result is numerically correct. Supported built-in operations include difference, percentage-point change, percent/percentage change, ratio, share, sum, and mean.

Usage

transformation_audit(reference, claim, abs_tol = 1e-08, rel_tol = 1e-06)

Arguments

Value

A stat_audit_component.

Examples

ref <- stat_reference(-0.7, "rate change", "A", "2024-2025",
  "percentage points", "Provider",
  transformation = list(operation = "percentage point change",
                        inputs = c(11.1, 10.4), result = -0.7))
clm <- stat_claim(-6.31, "rate change", "A", "2024-2025",
  "percent", "Provider",
  transformation = list(operation = "percent change",
                        inputs = c(11.1, 10.4), result = -6.31))
transformation_audit(ref, clm)