Technical reference: algorithm functions

1. Two Z-Score Methods

2. CSD Method — sd.orig

The cleaning algorithm uses the Conditional Standard Deviation (CSD) method. This divides the difference from the median by a half-SD derived from the reference distribution, rather than the full LMS-adjusted SD. This is the same approach the CDC uses for its modified z-scores, described in:

> Kuczmarski RJ, Ogden CL, Guo SS, et al. 2000 CDC growth charts for the United States: Methods and development. National Center for Health Statistics. Vital Health Stat 11(246). 2002. PDF

If measurement < M:   sd.orig = (measurement - M) / csd_neg
If measurement >= M:  sd.orig = (measurement - M) / csd_pos

3. LMS Method (not used by the cleaning algorithm)

The cleaning algorithm computes no LMS z-score. No CDC-only Box-Cox z-score (z.orig) is calculated or returned by cleangrowth(). The standard LMS formula is used only in the standalone ext_bmiz() extended-BMI tool (§7):

If L != 0:  z = ((X / M)^L - 1) / (L * S)
If L == 0:  z = log(X / M) / S

If you need an age-blended LMS z-score for analysis (the conventional growth z-score, distinct from the CSD scores the algorithm uses), compute it separately — see §10.

4. Recentered Scores — tbc.sd and ctbc.sd

The cleaning algorithm works on recentered z-scores so that outlier detection is age-stable: the population-median CSD z-score drifts with age, so subtracting a per-agedays median (computed separately for each param and sex) puts every measurement on a flatter footing before trajectory checks. By default the median comes from a fixed precomputed file (see below); it can instead be turned off (sd.recenter = "none") or supplied directly by the caller.

tbc.sd = sd.orig - rcmedian.sd

ctbc.sd = sd.corr - rcmedian.sd

5. Gestational-Age-Corrected Scores — sd.corr

For preterm infants, growthcleanr applies a gestational-age correction using gestational-age reference values (see vignette("wrapper") → “Gestational-age reference values”).

The gestational-age correction writes a separate column (sd.corr); it does not overwrite sd.orig with a corrected score, so sd.orig remains the uncorrected reference CSD z-score. One exception: under the default, Child Step 2b rescores sd.orig 0.7 cm lower for height rows in the gestational-age window (vignette("wrapper"), Phase 8). PIV (Child Step 7) and CF rescue (Child Step 6) therefore read sd.orig directly as the uncorrected score. When no correction applies (term infants, older children), sd.corr = sd.orig.

6. EWMA (Exponentially Weighted Moving Average)

The EWMA is the central mechanism for detecting implausible values. It is computed from recentered CSD z-scores (tbc.sd).

weight_i = (5 + |agedays_current - agedays_i|) ^ exponent

7. Extended BMI Z-Scores — ext_bmiz()

The standalone ext_bmiz() function computes extended BMI z-scores for children 2–19.9 years, following the CDC’s extended BMI method.

For children with BMI at or below the 95th percentile, the extended z-score equals the standard LMS z-score. For children above the 95th percentile, a half-normal distribution with a fitted sigma parameter is used:

Males:   sigma = 0.3728 + 0.5196 * age_yr - 0.0091 * age_yr^2
Females: sigma = 0.8334 + 0.3712 * age_yr - 0.0011 * age_yr^2

For BMI <= 95th percentile:  ebp = 100 * pnorm(bz)        [standard LMS]
For BMI > 95th percentile:   ebp = 90 + 10 * pnorm((BMI - p95) / sigma)

8. Rounding

Intermediate z-score rounding has been removed in version 3. sd.orig_who, sd.orig_cdc, the blended sd.orig, tbc.sd, and ctbc.sd are all carried at full numeric precision throughout the cleaning pipeline. The previous Stata-style rounding to 0.001 (half away from zero) existed only for cross-platform Stata-vs-R validation and has been removed along with the round_stata() helper; it was never needed by the algorithm itself. Downstream consumers can round the returned columns to whatever precision they like.

9. Columns Returned by cleangrowth()

The full return-value contract — which columns are always returned, which are gated behind options (debug, cf_detail, corr_detail, etc.), and what each can contain — is documented in vignette("output") (the single source of truth).

The z-scores described in this document (sd.orig, tbc.sd, sd.corr, ctbc.sd, and the plus/minus perturbation variants) are not in the default output; they are returned only under debug = TRUE (or full_detail = TRUE), and on child rows only. See vignette("output") §7.

10. Key Distinctions for Downstream Use

11. Reference Table Summary

Unit-Error-Range and CSD-z Helper Reference (child + adult)

Single shared reference for the unit-error-range helper family and the small adult CSD-z reference it depends on. These helpers span both algorithms: the child algorithm and the adult algorithm each compute an informational unit_error_range output column through the same band-logic helper (.unit_error_range_flag()), and the adult algorithm additionally uses its CSD-z helpers inside Evil Twins (Adult Step 9Wa) for the UER3 unit-error split and the median-band |z| tiebreak. Because the family is cross-algorithm, it is documented here rather than duplicated in the child and adult narratives; both narratives’ Support Functions sections point here.

This document describes what the current R code does. It is a debugging aid and a long-term technical reference. References to code use function names rather than line numbers, because line numbers go stale with each edit.

What unit_error_range is

unit_error_range is a default-ON, informational logical output column (parameter unit_error_range = TRUE; see vignette("configuration") and vignette("output")). It never affects exclude and is not a member of exclude.levels. For each row it answers a single question:

Is this measurement’s magnitude consistent with a metric/imperial unit-conversion error (the value was recorded in the wrong unit system), AND is it outside the plausible z = −3 to +3 band?

TRUE flags values that look like they could be a unit error and are not already within the plausible range. It is purely a magnitude/range test — it does not check the rest of the trajectory, and it makes no exclusion decision. It is set to FALSE (not evaluated) for rows with the internal codes Exclude-Missing, Exclude-Not-Cleaned, and Exclude-Not-GC-Param (internal names; for the reported codes, see vignette("exclusion-codes")) and for rows with missing age (see the .compute_unit_error_range() dispatch below).

The same flag is also computed internally during cleaning — in child Evil Twins (Child Step 9) and adult Evil Twins (Adult Step 9Wa) — as one input to the UER3 unit-error-split selection logic. The output column and the internal use share one helper.

cleangrowth() also uses this flag for a systematic-unit-error warning: when more than 20% of a parameter group’s cleaned (evaluated) values are flagged TRUE, it emits a soft warning() that the whole column may be in the wrong unit (e.g. weight recorded in pounds but labeled WEIGHTKG). The warning also notes that a high share may instead reflect people with atypical growth (for example, extremely premature infants, severe obesity, some medical conditions), whose real values can fall in the unit-error range. The share is computed per parameter group (weight / height / head circumference) over the same evaluated rows the flag applies to. It is advisory only — no minimum sample size and no effect on exclude.

The shared band logic: .unit_error_range_flag(param, value, zn3, zp3)

Purpose. Direction-agnostic unit-error-range test. Given each row’s param, its metric measurement value, and the plausible-band endpoints zn3/zp3 (the measurements at recentered z = −3 and z = +3 for that row’s reference), it builds the two unit-error bands and the plausible carve-out and returns a logical vector.

The two unit-error bands. A value is “unit-error-shaped” if, had it actually been recorded in the other unit system, it would have landed in the plausible band. Two directions:

• range_im — measured imperial, recorded metric (the imperial number was dropped into a metric field).
• range_mi — measured metric, recorded imperial.

The band endpoints are the plausible endpoints scaled by the conversion factor for the param:

• Weight (WEIGHTKG): factor LB_PER_KG = 2.2046226218. A lb number sitting in a kg field inflates; range_im endpoints are zn3 * LB_PER_KG, zp3 * LB_PER_KG; range_mi endpoints are zn3 / LB_PER_KG, zp3 / LB_PER_KG.
• Height/HC (everything not WEIGHTKG): factor CM_PER_IN = 2.54. An inch number in a cm field deflates; range_im endpoints are zn3 / CM_PER_IN, zp3 / CM_PER_IN; range_mi endpoints are zn3 * CM_PER_IN, zp3 * CM_PER_IN.

Both unit-error bands are exclusive (value > pmin(...) & value < pmax(...), using pmin/pmax so the band orients correctly regardless of endpoint order).

The plausible carve-out range_3. The inclusive interval value >= zn3 & value <= zp3. A value already inside the plausible band is never flagged, even if it also happens to fall in a unit-error band.

Return. TRUE iff (in range_im OR in range_mi) AND NOT in range_3. NA endpoints (no reference lookup matched, e.g. HEADCM out of cleaning range) → FALSE (res[is.na(res)] <- FALSE).

Callers. Child Step 9 Evil Twins (builds the internal uer column), the child+adult output assembler .compute_unit_error_range(), and adult Evil Twins (evil_twins() in adult_support.R, as sp_uer).

Child side

Adult side

The adult algorithm computes no recentered z-scores. For both the unit-error band endpoints and the Evil Twins CSD work it uses a small fixed sex-specific NHANES reference with two-sided conditional SDs.

How the pieces fit

unit_error_range OUTPUT column (default ON):
  .compute_unit_error_range(dt, adult_cutpoint)         [child_clean.R]
      ├── child rows → .child_z3()  → zn3/zp3 from child_z3_reference.csv.gz
      ├── adult rows → .adult_z3()  → zn3/zp3 = M ± 3·SD   [adult_support.R]
      └── .unit_error_range_flag(param, v, zn3, zp3)       [adult_support.R]

INTERNAL use during cleaning (one input to the unit-error split):
  child Step 9 Evil Twins:  .child_z3() + .unit_error_range_flag()  → uer
  adult Step 9Wa Evil Twins: .adult_z3() + .unit_error_range_flag() → sp_uer
                             .adult_csd_z() → UER3 group split + |z| tiebreak

Two distinct uses of the adult CSD reference. The band endpoints (.adult_z3()) feed both the output column and the adult ET unit-error band; the per-value z (.adult_csd_z()) is ET-internal only. Both draw their constants from the single .adult_csd_ref() object.

Related: .compute_corrected_z() (GA-correction engine)

.compute_corrected_z() is not a unit-error helper — it is the Child Step 2b gestational-age corrected-z engine — but it is the other child CSD-z helper, so it is cross-referenced here. Its Support-Functions entry lives in vignette("child-algorithm") → Support Functions. In brief: it encapsulates the corrected-z chain (uncorrected WHO/CDC blend at chronological age, gestational-age reference CSD z at corrected age, corrected WHO/CDC z at cagedays, the 0.7 cm standing/recumbent offset (.gc_len_offset, shared with length.adjust) when agedays >= 730 and cagedays < 730 (see vignette("wrapper"), Phase 8, fact 6), the GA-reference-preferred rule ≤2y, and the final ≤2 / 2–4 blend / else fcase) so the same correction is applied to the actual measurement and to the ±5%-wt / ±1 cm plus/minus perturbations. Callers: cleangrowth() Step 2b preprocessing.

Notes for maintainers

• Verify after any reference-data change. If the NHANES source values or inst/extdata/child_z3_reference.csv.gz are regenerated, confirm the .adult_csd_ref() hard-coded values still match them, and re-run the unit-error-range tests (tests/testthat/test-child-unit-error-range.R, which unit-tests .unit_error_range_flag() and .compute_unit_error_range() directly).
• Magic numbers. LB_PER_KG = 2.2046226218 and CM_PER_IN = 2.54 are the only conversion constants; the ±3 band width is fixed (the “plausible” range definition).
• Single source of truth for the public column contract is vignette("output"); for the parameter, vignette("configuration"). This file documents the helpers, not the column’s place in the return contract.