statwitness

statwitness provides model-aware behavioral validation and audit certificates for statistical analyses.

The package applies controlled transformations with known expected consequences, refits the analysis, and verifies that the fitted model responds correctly. Examples include reordering observations, changing measurement units, relabeling factors, reconstructing predictions, and refitting models with alternative optimizers.

These behavioral checks are combined with model-specific diagnostics for numerical stability, design adequacy, assumptions, and influential observations.

Audit domains

statwitness organizes checks into five domains.

Computational integrity

Verifies:

• analysis-sample reconstruction;
• model-matrix and prediction reconstruction;
• row-order invariance;
• factor- and grouping-label invariance;
• unit conversion;
• outcome translation;
• parameterization behavior.

Numerical stability

Evaluates:

• rank deficiency;
• numerical conditioning;
• convergence;
• singularity;
• boundary estimates;
• optimizer agreement.

Design adequacy

Screens:

• ANOVA cell balance and empty cells;
• repeated-measures completeness;
• random-effects design;
• potential random-slope requirements;
• survival-event information.

Assumption screening

Provides model-specific checks for:

• residual shape;
• unequal residual variance;
• GLM dispersion;
• numerical separation;
• proportional hazards.

Influence stability

Evaluates influential:

• observations;
• participants;
• clusters;
• raters;
• survival-model cases.

The same audit domains are used across model families, but the individual checks are model-specific. A check is run only when its expected behavior is justified for the fitted model.

Supported analyses

Version 0.1.0 supports:

• linear regression with stats::lm();
• classical and factorial ANOVA represented by lm() or aov();
• generalized linear models with stats::glm();
• linear mixed-effects models with lme4::lmer();
• generalized mixed-effects models with lme4::glmer();
• conditional mixed models fitted with glmmTMB;
• repeated-measures ANOVA fitted through afex::aov_ez();
• Cox proportional-hazards models with survival::coxph();
• parametric survival models with survival::survreg();
• custom fitting functions that return compatible model objects.

Packages such as lme4, glmmTMB, afex, and survival are required only for their corresponding model classes.

Installation

Install the released version from CRAN:

install.packages("statwitness")

Load the package:

library(statwitness)

General usage

The main interface is:

audit <- statwitness(
  formula,
  data = your_data,
  focus = "term_of_interest",
  audit_level = "standard"
)

Print a concise certificate:

audit

Display every individual check:

print(audit, details = TRUE)

Selecting the focus term

focus identifies the coefficient or omnibus model term treated as the primary result.

For example:

audit <- statwitness(
  outcome ~ treatment + age + sex,
  data = dataset,
  focus = "treatment"
)

For an interaction:

audit <- statwitness(
  outcome ~ treatment * sex,
  data = dataset,
  focus = "treatment:sex"
)

When focus refers to a multi-parameter term, such as an interaction between factors, statwitness evaluates it as an omnibus term rather than selecting a single dummy-coded coefficient.

The focus argument is optional. Specifying it is recommended whenever a model contains multiple predictors or interactions.

Mixed-effects models

audit <- statwitness(
  score ~ criterion * model + (1 | rater),
  data = dataset,
  focus = "criterion:model"
)

audit

For mixed-effects models, the standard audit can include:

• row-order invariance;
• analysis-sample reconstruction;
• fixed-effect prediction reconstruction;
• factor-label invariance;
• grouping-label invariance;
• outcome translation and scaling;
• fixed-effect design conditioning;
• convergence diagnostics;
• singularity screening;
• random-effect boundary checks;
• alternative-optimizer agreement;
• random-slope design screening;
• residual screening;
• leave-one-group-out influence analysis.

A possible certificate is:

Computational integrity:    PASSED
Numerical stability:        PASSED
Design adequacy:            REVIEW NEEDED
Assumption screening:       PASSED
Influence stability:        PASSED

Overall status:
PASSED WITH REVIEW ITEMS

A design review does not automatically mean that the fitted model is incorrect. It identifies an aspect of the model that requires scientific consideration.

Linear regression

dat <- transform(
  mtcars,
  transmission = factor(am)
)

audit <- statwitness(
  mpg ~ transmission + wt + hp,
  data = dat,
  focus = "transmission"
)

audit

The standard linear-model audit can include:

• row-order invariance;
• formula-order invariance;
• outcome translation and scaling;
• predictor rescaling;
• model-matrix reconstruction;
• prediction reconstruction;
• fixed-design rank and conditioning;
• residual-shape screening;
• unequal-variance screening;
• Cook’s-distance influence screening.

Logistic regression

dat <- transform(
  mtcars,
  high_mpg = mpg > median(mpg),
  transmission = factor(am)
)

audit <- statwitness(
  high_mpg ~ transmission + wt,
  data = dat,
  family = binomial(),
  focus = "transmission"
)

audit

Generalized linear-model audits can additionally include:

• binary-outcome recoding;
• factor-reference reversal;
• linear-predictor reconstruction;
• likelihood verification;
• Pearson-dispersion screening;
• numerical separation screening.

Factorial ANOVA

audit <- statwitness(
  breaks ~ wool * tension,
  data = warpbreaks,
  focus = "wool:tension",
  method = "aov"
)

audit

The ANOVA design audit can identify:

• empty cells;
• unbalanced cells;
• rank-deficient terms;
• sensitivity to sequential term order;
• contrast-dependent interpretations.

Repeated-measures ANOVA

Repeated-measures designs use a dedicated interface:

audit <- statwitness_repeated(
  data = dataset,
  outcome = "score",
  id = "rater",
  within = c("criterion", "model"),
  focus = "criterion:model",
  type = 3,
  correction = "GG"
)

audit

The repeated-measures audit can include:

• participant-condition completeness;
• duplicated participant-condition observations;
• participant-label invariance;
• row-order invariance;
• outcome translation and scaling;
• sphericity-correction handling;
• residual screening;
• leave-one-participant-out influence analysis.

Generalized mixed-effects models

audit <- statwitness(
  correct ~ criterion * model + (1 | rater),
  data = dataset,
  family = binomial(),
  focus = "criterion:model"
)

audit

Generalized mixed-model audits combine GLM-specific checks with:

• convergence diagnostics;
• singularity screening;
• random-effect boundary checks;
• grouping-label invariance;
• optimizer agreement;
• group-level influence analysis.

Cox proportional-hazards models

audit <- statwitness(
  survival::Surv(time, event) ~ treatment + age,
  data = patients,
  focus = "treatment"
)

audit

The Cox-model audit can include:

• event-coding validation;
• analysis-sample reconstruction;
• time-unit scaling;
• event-information screening;
• proportional-hazards diagnostics;
• DFBETA influence screening.

Audit levels

Three audit levels are available.

Core

statwitness(
  outcome ~ treatment,
  data = dataset,
  focus = "treatment",
  audit_level = "core"
)

Runs relatively fast computational-integrity checks.

Standard

statwitness(
  outcome ~ treatment,
  data = dataset,
  focus = "treatment",
  audit_level = "standard"
)

Adds routine numerical, design, assumption, and influence checks.

Thorough

statwitness(
  outcome ~ treatment,
  data = dataset,
  focus = "treatment",
  audit_level = "thorough"
)

Adds more computationally intensive checks, including broader optimizer comparisons and deletion-based influence analyses where supported.

The exact tests depend on the model class and analysis structure.

Previewing an audit

Use audit_plan() to see which checks are applicable before running the full audit:

audit_plan(
  score ~ criterion * model + (1 | rater),
  data = dataset,
  focus = "criterion:model",
  audit_level = "standard"
)

The plan reports:

• the selected checks;
• the domain of each check;
• checks that are not applicable;
• the reason each check was selected or skipped.

Interpreting a certificate

Possible domain results include:

• PASSED;
• WARNING;
• REVIEW NEEDED;
• FAILED;
• NOT EVALUATED.

A passing certificate means that the analysis behaved as expected under the registered checks that were performed.

It does not establish that:

• the study design is correct;
• the model answers the intended scientific question;
• the estimand is appropriate;
• confounding is absent;
• causal interpretation is justified;
• the source data are accurate;
• every possible assumption has been satisfied;
• the effect is practically important.

statwitness complements scientific judgment and substantive model evaluation; it does not replace them.

Reproducible certificates

Audit reports can include:

• a certificate identifier;
• a data fingerprint;
• an analysis fingerprint;
• the model formula;
• the model class;
• the selected focus term;
• the R and package versions;
• the checks performed;
• numerical tolerances used by the checks.

Fingerprints help determine whether two certificates were generated from the same supplied data and analysis configuration.

Citation

To cite statwitness in publications, run:

citation("statwitness")