Package {VIM}

Version:	7.3.0
Title:	Visualization and Imputation of Missing Values
Depends:	R (≥ 4.1.0),colorspace,grid
Imports:	grDevices, robustbase, stats, sp, vcd, nnet, e1071, methods, Rcpp, utils, graphics, laeken, ranger, MASS, Matrix, xgboost, data.table(≥ 1.9.4), mlr3, mlr3pipelines, R6, paradox (≥ 1.0.0), mlr3tuning (≥ 1.0.0), mlr3learners, future
Suggests:	dplyr, tinytest, knitr, mgcv, rmarkdown, reactable, covr, withr, pdist, enetLTS, robmixglm, stringr, glmnet, validate, ECOSolveR, cellWise, crmReg, mice, car, lgr, rpart, missRanger, tidyr
Description:	Provides methods for imputation and visualization of missing values. It includes graphical tools to explore the amount, structure and patterns of missing and/or imputed values, supporting exploratory data analysis and helping to investigate potential missingness mechanisms (details in Alfons, Templ and Filzmoser, <doi:10.1007/s11634-011-0102-y>). The quality of imputations can be assessed visually using a wide range of univariate, bivariate and multivariate plots. The package further provides several imputation methods, including efficient implementations of k-nearest neighbour and hot-deck imputation (Kowarik and Templ 2013, <doi:10.18637/jss.v074.i07>), iterative robust model-based multiple imputation (Templ 2011, <doi:10.1016/j.csda.2011.04.012>; Templ 2023, <doi:10.3390/math11122729>), and machine learning–based approaches such as robust GAM-based multiple imputation (Templ 2024, <doi:10.1007/s11222-024-10429-1>) as well as random forest and gradient boosting (XGBoost) imputation (Niederhametner et al., <doi:10.1177/18747655251339401>). General background and practical guidance on imputation are provided in the Springer book by Templ (2023) <doi:10.1007/978-3-031-30073-8>.
LazyData:	TRUE
ByteCompile:	TRUE
License:	GPL-2 | GPL-3 [expanded from: GPL (≥ 2)]
URL:	https://github.com/statistikat/VIM
BugReports:	https://github.com/statistikat/VIM/issues
Repository:	CRAN
LinkingTo:	Rcpp
Encoding:	UTF-8
VignetteBuilder:	knitr
Config/roxygen2/version:	8.0.0
RoxygenNote:	7.3.3
NeedsCompilation:	yes
Packaged:	2026-09-01 13:55:48 UTC; matthias
Author:	Matthias Templ [aut, cre], Alexander Kowarik [aut], Andreas Alfons [aut], Johannes Gussenbauer [aut], Nina Niederhametner [aut], Eileen Vattheuer [aut], Gregor de Cillia [aut], Bernd Prantner [ctb], Wolfgang Rannetbauer [aut]
Maintainer:	Matthias Templ <matthias.templ@gmail.com>
Date/Publication:	2026-09-02 10:20:02 UTC

The VIM Package: Visualization and Imputation of Missing Values

Description

VIM provides tools for visualization, imputation, and exploration of missing and multivariate data.

Details

Visualization and Imputation of Missing Values

This package introduces new tools for the visualization of missing or imputed values in , which can be used for exploring the data and the structure of the missing or imputed values. Depending on this structure, they may help to identify the mechanism generating the missing values or errors, which may have happened in the imputation process. This knowledge is necessary for selecting an appropriate imputation method in order to reliably estimate the missing values. Thus the visualization tools should be applied before imputation and the diagnostic tools afterwards.

Detecting missing values mechanisms is usually done by statistical tests or models. Visualization of missing and imputed values can support the test decision, but also reveals more details about the data structure. Most notably, statistical requirements for a test can be checked graphically, and problems like outliers or skewed data distributions can be discovered. Furthermore, the included plot methods may also be able to detect missing values mechanisms in the first place.

A graphical user interface available in the package VIMGUI allows an easy handling of the plot methods. In addition, VIM can be used for data from essentially any field.

This package includes advanced imputation methods, robust statistics, and tools for data preprocessing and diagnostics.

Author(s)

Matthias Templ, Andreas Alfons, Alexander Kowarik, Bernd Prantner

Maintainer: Matthias Templ matthias.templ@gmail.com

References

M. Templ (2023) Visualization and Imputation of Missing Values. Springer Publishing. Series in Computational Statistics. Cham. Switzerland. 463 pages. DOI: 10.1007/978-3-031-30073-8

A. Kowarik, M. Templ (2016) Imputation with R package VIM. Journal of Statistical Software, 74(7), 1-16.

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

M. Templ, A. Kowarik, P. Filzmoser (2011) Iterative stepwise regression imputation using standard and robust methods. Computational Statistics & Data Analysis, Vol. 55, pp. 2793-2806.

See Also

Useful links:

• https://github.com/statistikat/VIM

• Report bugs at https://github.com/statistikat/VIM/issues

Apply a weight function to standardized values

Description

Dispatches to huber_weight or tukey_weight depending on the method argument.

Usage

.apply_weight_fun(u, method = "huber", alpha = NULL)

Arguments

Value

numeric vector of weights in [0, 1]

Author(s)

Matthias Templ

Robust scale estimate via MAD

Description

Computes the median absolute deviation with a fallback for zero or near-zero MAD (constant columns). In that case the inter-quartile range scaled to match the normal distribution is used. If both are zero, returns 1 so that standardized values remain unchanged.

Usage

.robust_scale(x)

Arguments

Value

positive numeric scalar

Author(s)

Matthias Templ

QR-based weighted least squares with cell-derived row weights

Description

Solves \min_\beta \sum_i w_i (y_i - X_i \beta)^2 on the unweighted design, where the row weight w_i = w^{cellrow}_i \cdot w^{\psi}_i \cdot w^{resp}_i and w^{cellrow}_i is the geometric mean of the predictor cell weights of row i. Cell weights thus downweight the influence of rows with contaminated cells without distorting the design values, so the returned \beta is a valid coefficient for X \beta. Uses QR decomposition for numerical stability.

Usage

.weighted_qr_solve(X_int, y, w_cell_int, w_response, w_psi)

Arguments

Value

numeric (p+1)-vector of regression coefficients

Author(s)

Matthias Templ

Animals_na

Description

Average log brain and log body weights for 28 Species

Format

A data frame with 28 observations on the following 2 variables.

lbody

log body weight

lbrain

log brain weight

Details

The original data can be found in package MASS. 10 values on brain weight are set to be missing.

Source

P. J. Rousseeuw and A. M. Leroy (1987) Robust Regression and Outlier Detection. Wiley, p. 57.

References

Venables, W. N. and Ripley, B. D. (1999) Modern Applied Statistics with S-PLUS. Third Edition. Springer.

Templ, M. (2022) Visualization and Imputation of Missing Values. Springer Publishing. Upcoming book.

Examples

data(Animals_na)
aggr(Animals_na)

Synthetic subset of the Austrian structural business statistics data

Description

Synthetic subset of the Austrian structural business statistics (SBS) data, namely NACE code 52.42 (retail sale of clothing).

Details

The Austrian SBS data set consists of more than 320.000 enterprises. Available raw (unedited) data set: 21669 observations in 90 variables, structured according NACE revision 1.1 with 3891 missing values.

We investigate 9 variables of NACE 52.42 (retail sale of clothing).

From these confidential raw data set a non-confidential, close-to-reality, synthetic data set was generated.

Source

http://www.statistik.at

Examples

data(SBS5242)
aggr(SBS5242)

Aggregations for missing/imputed values

Description

Calculate or plot the amount of missing/imputed values in each variable and the amount of missing/imputed values in certain combinations of variables.

Print method for objects of class "aggr".

Summary method for objects of class "aggr".

Print method for objects of class "summary.aggr".

Usage

aggr(x, delimiter = NULL, plot = TRUE, ...)

## S3 method for class 'aggr'
plot(
  x,
  col = c("skyblue", "red", "orange"),
  bars = TRUE,
  numbers = FALSE,
  prop = TRUE,
  combined = FALSE,
  varheight = FALSE,
  only.miss = FALSE,
  border = par("fg"),
  sortVars = FALSE,
  sortCombs = TRUE,
  ylabs = NULL,
  axes = TRUE,
  labels = axes,
  cex.lab = 1.2,
  cex.axis = par("cex"),
  cex.numbers = par("cex"),
  gap = 4,
  ...
)

## S3 method for class 'aggr'
print(x, ..., digits = NULL)

## S3 method for class 'aggr'
summary(object, ...)

## S3 method for class 'summary.aggr'
print(x, ...)

Arguments

Details

Often it is of interest how many missing/imputed values are contained in each variable. Even more interesting, there may be certain combinations of variables with a high number of missing/imputed values.

If combined is FALSE, two separate plots are drawn for the missing/imputed values in each variable and the combinations of missing/imputed and non-missing values. The barplot on the left hand side shows the amount of missing/imputed values in each variable. In the aggregation plot on the right hand side, all existing combinations of missing/imputed and non-missing values in the observations are visualized. Available, missing and imputed data are color coded as given by col. Additionally, there are two possibilities to represent the frequencies of occurrence of the different combinations. The first option is to visualize the proportions or frequencies by a small bar plot and/or numbers. The second option is to let the cell heights be given by the frequencies of the corresponding combinations. Furthermore, variables may be sorted by the number of missing/imputed values and combinations by the frequency of occurrence to give more power to finding the structure of missing/imputed values.

If combined is TRUE, a small version of the barplot showing the amount of missing/imputed values in each variable is drawn on top of the aggregation plot.

The graphical parameter oma will be set unless supplied as an argument.

Value

for aggr, a list of class "aggr" containing the following components:

• x the data used.

• combinations a character vector representing the combinations of variables.

• count the frequencies of these combinations.

• percent the percentage of these combinations.

• missings a data.frame containing the amount of missing/imputed values in each variable.

• tabcomb the indicator matrix for the combinations of variables.

a list of class "summary.aggr" containing the following components:

• missings a data.frame containing the amount of missing or imputed values in each variable.

• combinations a data.frame containing a character vector representing the combinations of variables along with their frequencies and percentages.

Note

Some of the argument names and positions have changed with version 1.3 due to extended functionality and for more consistency with other plot functions in VIM. For back compatibility, the arguments labs and names.arg can still be supplied to ...{} and are handled correctly. Nevertheless, they are deprecated and no longer documented. Use ylabs and labels instead.

Author(s)

Andreas Alfons, Matthias Templ, modifications for displaying imputed values by Bernd Prantner

Matthias Templ, modifications by Andreas Alfons and Bernd Prantner

Matthias Templ, modifications by Andreas Alfons

Andreas Alfons, modifications by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

print.aggr(), summary.aggr()

aggr()

print.summary.aggr(), aggr()

summary.aggr(), aggr()

Other plotting functions: barMiss(), histMiss(), marginmatrix(), marginplot(), matrixplot(), mosaicMiss(), pairsVIM(), parcoordMiss(), pbox(), scattJitt(), scattMiss(), scattmatrixMiss(), spineMiss()

Examples

data(sleep, package="VIM")
## for missing values
a <- aggr(sleep)
a
summary(a)

## for imputed values
sleep_IMPUTED <- kNN(sleep)
a <- aggr(sleep_IMPUTED, delimiter="_imp")
a
summary(a)


data(sleep, package = "VIM")
a <- aggr(sleep, plot=FALSE)
a


data(sleep, package = "VIM")
summary(aggr(sleep, plot=FALSE))


data(sleep, package = "VIM")
s <- summary(aggr(sleep, plot=FALSE))
s

Alphablending for colors

Description

Convert colors to semitransparent colors.

Usage

alphablend(col, alpha = NULL, bg = NULL)

Arguments

Value

a vector containing the semitransparent colors.

Author(s)

Andreas Alfons

Examples

alphablend("red", 0.6)

Barplot with information about missing/imputed values

Description

Barplot with highlighting of missing/imputed values in other variables by splitting each bar into two parts. Additionally, information about missing/imputed values in the variable of interest is shown on the right hand side.

Usage

barMiss(
  x,
  delimiter = NULL,
  pos = 1,
  selection = c("any", "all"),
  col = c("skyblue", "red", "skyblue4", "red4", "orange", "orange4"),
  border = NULL,
  main = NULL,
  sub = NULL,
  xlab = NULL,
  ylab = NULL,
  axes = TRUE,
  labels = axes,
  only.miss = TRUE,
  miss.labels = axes,
  interactive = TRUE,
  ...
)

Arguments

Details

If more than one variable is supplied, the bars for the variable of interest are split according to missingness/number of imputed missings in the additional variables.

If only.miss=TRUE, the missing/imputed values in the variable of interest are visualized by one bar on the right hand side. If additional variables are supplied, this bar is again split into two parts according to missingness/number of imputed missings in the additional variables.

Otherwise, a small barplot consisting of two bars is drawn on the right hand side. The first bar corresponds to observed values in the variable of interest and the second bar to missing/imputed values. Since these two bars are not on the same scale as the main barplot, a second y-axis is plotted on the right (if axes=TRUE). Each of the two bars are again split into two parts according to missingness/number of imputed missings in the additional variables. Note that this display does not make sense if only one variable is supplied, therefore only.miss is ignored in that case.

If interactive=TRUE, clicking in the left margin of the plot results in switching to the previous variable and clicking in the right margin results in switching to the next variable. Clicking anywhere else on the graphics device quits the interactive session. When switching to a continuous variable, a histogram is plotted rather than a barplot.

Value

a numeric vector giving the coordinates of the midpoints of the bars.

Note

Some of the argument names and positions have changed with version 1.3 due to extended functionality and for more consistency with other plot functions in VIM. For back compatibility, the arguments axisnames, names.arg and names.miss can still be supplied to ...{} and are handled correctly. Nevertheless, they are deprecated and no longer documented. Use labels and miss.labels instead.

Author(s)

Andreas Alfons, modifications to show imputed values by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

spineMiss(), histMiss()

Other plotting functions: aggr(), histMiss(), marginmatrix(), marginplot(), matrixplot(), mosaicMiss(), pairsVIM(), parcoordMiss(), pbox(), scattJitt(), scattMiss(), scattmatrixMiss(), spineMiss()

Examples

data(sleep, package = "VIM")
## for missing values
x <- sleep[, c("Exp", "Sleep")]
barMiss(x)
barMiss(x, only.miss = FALSE)

## for imputed values
x_IMPUTED  <- kNN(sleep[, c("Exp", "Sleep")])
barMiss(x_IMPUTED, delimiter = "_imp")
barMiss(x_IMPUTED, delimiter = "_imp", only.miss = FALSE)

Breast cancer Wisconsin data set

Description

Dataset containing the original Wisconsin breast cancer data.

Format

A data frame with 699 observations on the following 11 variables.

ID

Sample ID

clump_thickness

as integer from 1 - 10

uniformity_cellsize

as integer from 1 - 10

uniformity_cellshape

as integer from 1 - 10

adhesion

as integer from 1 - 10

epithelial_cellsize

as integer from 1 - 10

bare_nuclei

as integer from 1 - 10, includes 16 missings

chromatin

as integer from 1 - 10

normal_nucleoli

as integer from 1 - 10

mitoses

as integer from 1 - 10

class

benign or malignant

References

The data downloaded and conditioned for R from the UCI machine learning repository, see https://archive.ics.uci.edu/ml/datasets/Breast+Cancer+Wisconsin+(Original) This breast cancer databases was obtained from the University of Wisconsin Hospitals, Madison from Dr. William H. Wolberg. If you publish results when using this database, then please include this information in your acknowledgements. Also, please cite one or more of: O. L. Mangasarian and W. H. Wolberg: "Cancer diagnosis via linear programming", SIAM News, Volume 23, Number 5, September 1990, pp 1 & 18. William H. Wolberg and O.L. Mangasarian: "Multisurface method of pattern separation for medical diagnosis applied to breast cytology", Proceedings of the National Academy of Sciences, U.S.A., Volume 87, December 1990, pp 9193-9196. O. L. Mangasarian, R. Setiono, and W.H. Wolberg: "Pattern recognition via linear programming: Theory and application to medical diagnosis", in: "Large-scale numerical optimization", Thomas F. Coleman and Yuying Li, editors, SIAM Publications, Philadelphia 1990, pp 22-30. K. P. Bennett & O. L. Mangasarian: "Robust linear programming discrimination of two linearly inseparable sets", Optimization Methods and Software 1, 1992, 23-34 (Gordon & Breach Science Publishers).

Examples

data(bcancer)
aggr(bcancer)

Backgound map

Description

Plot a background map.

Usage

bgmap(map, add = FALSE, ...)

Arguments

Author(s)

Andreas Alfons

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Vol. 6, pp. 29-47. DOI: 10.1007/s11634-011-0102-y.

See Also

growdotMiss(), mapMiss()

Examples

data(kola.background, package = "VIM")
bgmap(kola.background)

Bootstrap resampling with robust strategies

Description

Returns bootstrap row indices based on the chosen strategy. Strategies adapted from imputeRobust (Templ 2024).

Usage

bootstrap_resample(
  n,
  strategy = "stratified",
  weights = NULL,
  residuals = NULL,
  alpha = 0.75,
  best_subset = NULL
)

Arguments

Value

Integer vector of length n with bootstrap row indices

Brittleness index data set

Description

A plastic product is produced in three parallel reactors (TK104, TK105, or TK107). For each row in the dataset, we have the same batch of raw material that was split, and fed to the 3 reactors. These values are the brittleness index for the product produced in the reactor. A simulated data set.

Format

A data frame with 23 observations on the following 3 variables.

TK104

Brittleness for batches of raw material in reactor 104

TK105

Brittleness for batches of raw material in reactor 105

TK107

Brittleness for batches of raw material in reactor 107

Source

https://openmv.net/info/brittleness-index

Examples

data(brittleness)
aggr(brittleness)

Build a GAM formula with automatic smooth terms

Description

Constructs a formula for mgcv::gam() by wrapping numeric predictors with sufficient unique values in s() terms and keeping factors as linear terms.

Usage

build_gam_formula(target, features, data, min_unique = 4L, default_k = 10L)

Arguments

Value

A formula object suitable for mgcv::gam()

Cell-weighted Iteratively Reweighted Least Squares

Description

Performs robust regression where each predictor cell (i, k) carries its own weight, rather than collapsing to a single row-level weight. This is the key building block for all three cellwise-robust imputation methods (cellM, cellIRMI, cellEM).

Usage

cellIRWLS(
  X,
  y,
  w_cell = NULL,
  w_response = NULL,
  maxit = 50,
  eps = 1e-06,
  method = "tukey",
  alpha = NULL,
  init = "s-estimator",
  damping = TRUE
)

Arguments

Value

A list with components:

coefficients

named numeric vector of regression coefficients (including intercept)

fitted

numeric n-vector of fitted values

residuals

numeric n-vector of residuals

weights

numeric n-vector of final combined row-level weights (product of psi-weight and response-weight)

w_total

the n \times (p+1) matrix of final total weights (including the intercept column)

sigma

final robust scale estimate

converged

logical indicating convergence

iterations

number of iterations performed

Algorithm

1. Initialize coefficients with an S-estimator (robustbase::lmrob.S) when init = "s-estimator" and n > 2p, otherwise cell-weighted OLS.

2. Iterate until convergence:

   1. Compute residuals r = y - X \beta.

   2. Estimate robust scale \sigma = \mathrm{MAD}(r).

   3. Compute psi-weights w^{\psi}_i from standardized residuals r_i / \sigma.

   4. Form row weights w^{row}_i = w^{\psi}_i \cdot w^{resp}_i.

   5. Construct weighted design matrix with cell weights entering linearly: \tilde{X}_{ik} = \sqrt{w^{row}_i} \cdot w^{cell}_{ik} \cdot X_{ik}, and weighted response \tilde{y}_i = \sqrt{w^{row}_i} \cdot y_i.

   6. Solve via QR decomposition: \beta = (\tilde{X}^T \tilde{X})^{-1} \tilde{X}^T \tilde{y}.

3. Return coefficients, residuals, combined weights.

Author(s)

Matthias Templ

Compute per-cell contamination weights

Description

For each continuous column, standardize by median and MAD, then apply a robust weight function (Huber or Tukey bisquare) to obtain a weight in [0, 1] per cell. Categorical (factor, character, logical) columns receive weight 1.

Usage

cellWeights(X, method = "huber", alpha = NULL)

Arguments

Value

an n \times p numeric matrix of weights

Author(s)

Matthias Templ

Compute cell weights from regression residuals

Description

Given a vector of residuals and a robust scale estimate, standardize and apply a robust weight function. This is used inside the IRWLS loop to compute psi-weights from the current fit residuals.

Usage

cellWeightsFromResiduals(residuals, sigma, method = "huber", alpha = NULL)

Arguments

Value

numeric n-vector of weights in [0, 1]

Author(s)

Matthias Templ

Compute per-cell weights using MCD-based conditional residuals

Description

For each continuous cell (i,j), computes the conditional expectation E(x_{ij} | x_{i,-j}) under a robust Gaussian model (estimated via MCD), standardizes the residual, and applies a weight function. This captures multivariate outlier structure that univariate standardization misses.

Usage

cellWeightsMCD(X, method = "tukey", alpha = NULL)

Arguments

Value

an n x p numeric matrix of weights in [0, 1]

Author(s)

Matthias Templ

C-horizon of the Kola data with missing values

Description

This data set is the same as in package mvoutlier, except that values below the detection limit are coded as NA.

Format

A data frame with 606 observations on the following 110 variables.

*ID

a numeric vector

XCOO

a numeric vector

YCOO

a numeric vector

Ag

a numeric vector

Ag_INAA

a numeric vector

Al

a numeric vector

Al2O3

a numeric vector

As

a numeric vector

As_INAA

a numeric vector

Au_INAA

a numeric vector

B

a numeric vector

Ba

a numeric vector

Ba_INAA

a numeric vector

Be

a numeric vector

Bi

a numeric vector

Br_IC

a numeric vector

Br_INAA

a numeric vector

Ca

a numeric vector

Ca_INAA

a numeric vector

CaO

a numeric vector

Cd

a numeric vector

Ce_INAA

a numeric vector

Cl_IC

a numeric vector

Co

a numeric vector

Co_INAA

a numeric vector

EC

a numeric vector

Cr

a numeric vector

Cr_INAA

a numeric vector

Cs_INAA

a numeric vector

Cu

a numeric vector

Eu_INAA

a numeric vector

F_IC

a numeric vector

Fe

a numeric vector

Fe_INAA

a numeric vector

Fe2O3

a numeric vector

Hf_INAA

a numeric vector

Hg

a numeric vector

Hg_INAA

a numeric vector

Ir_INAA

a numeric vector

K

a numeric vector

K2O

a numeric vector

La

a numeric vector

La_INAA

a numeric vector

Li

a numeric vector

LOI

a numeric vector

Lu_INAA

a numeric vector

wt_INAA

a numeric vector

Mg

a numeric vector

MgO

a numeric vector

Mn

a numeric vector

MnO

a numeric vector

Mo

a numeric vector

Mo_INAA

a numeric vector

Na

a numeric vector

Na_INAA

a numeric vector

Na2O

a numeric vector

Nd_INAA

a numeric vector

Ni

a numeric vector

Ni_INAA

a numeric vector

NO3_IC

a numeric vector

P

a numeric vector

P2O5

a numeric vector

Pb

a numeric vector

pH

a numeric vector

PO4_IC

a numeric vector

Rb

a numeric vector

S

a numeric vector

Sb

a numeric vector

Sb_INAA

a numeric vector

Sc

a numeric vector

Sc_INAA

a numeric vector

Se

a numeric vector

Se_INAA

a numeric vector

Si

a numeric vector

SiO2

a numeric vector

Sm_INAA

a numeric vector

Sn_INAA

a numeric vector

SO4_IC

a numeric vector

Sr

a numeric vector

Sr_INAA

a numeric vector

SUM_XRF

a numeric vector

Ta_INAA

a numeric vector

Tb_INAA

a numeric vector

Te

a numeric vector

Th

a numeric vector

Th_INAA

a numeric vector

Ti

a numeric vector

TiO2

a numeric vector

U_INAA

a numeric vector

V

a numeric vector

W_INAA

a numeric vector

Y

a numeric vector

Yb_INAA

a numeric vector

Zn

a numeric vector

Zn_INAA

a numeric vector

ELEV

a numeric vector

*COUN

a numeric vector

*ASP

a numeric vector

TOPC

a numeric vector

LITO

a numeric vector

Al_XRF

a numeric vector

Ca_XRF

a numeric vector

Fe_XRF

a numeric vector

K_XRF

a numeric vector

Mg_XRF

a numeric vector

Mn_XRF

a numeric vector

Na_XRF

a numeric vector

P_XRF

a numeric vector

Si_XRF

a numeric vector

Ti_XRF

a numeric vector

Note

For a more detailed description of this data set, see the help file chorizon in package mvoutlier.

Source

Kola Project (1993-1998)

References

Reimann, C., Filzmoser, P., Garrett, R.G. and Dutter, R. (2008) Statistical Data Analysis Explained: Applied Environmental Statistics with R. Wiley.

Examples

data(chorizonDL, package = "VIM")
summary(chorizonDL)

HCL and RGB color sequences

Description

Compute color sequences by linear interpolation based on a continuous color scheme between certain start and end colors. Color sequences may thereby be computed in the HCL or RGB color space.

Usage

colSequence(p, start, end, space = c("hcl", "rgb"), ...)

colSequenceRGB(p, start, end, fixup = TRUE, ...)

colSequenceHCL(p, start, end, fixup = TRUE, ...)

Arguments

Value

A character vector containing hexadecimal strings of the form "#RRGGBB".

Author(s)

Andreas Alfons

References

Zeileis, A., Hornik, K., Murrell, P. (2009) Escaping RGBland: Selecting colors for statistical graphics. Computational Statistics & Data Analysis, 53 (9), 1259–1270.

See Also

colorspace::hex(), colorspace::sequential_hcl()

Examples

p <- c(0, 0.3, 0.55, 0.8, 1)

## HCL colors
colSequence(p, c(0, 0, 100), c(0, 100, 50))
colSequence(p, polarLUV(L=90, C=30, H=90), c(0, 100, 50))

## RGB colors
colSequence(p, c(1, 1, 1), c(1, 0, 0), space="rgb")
colSequence(p, RGB(1, 1, 0), "red")

Colic horse data set

Description

This is a modified version of the original training data set taken from the UCI repository, see reference. The modifications are only related to having appropriate levels for factor variables. This data set is about horse diseases where the task is to determine, if the lesion of the horse was surgical or not.

Format

A training data frame with 300 observations on the following 31 variables.

surgery

yes or no

age

1 equals an adult horse, 2 is a horse younger than 6 months

hospitalID

ID

temp_rectal

rectal temperature

pulse

heart rate in beats per minute

respiratory_rate

a normal rate is between 8 and 10

temp_extreme

temperature of extremities

pulse_peripheral

factor with four categories

capillayr_refill_time

a clinical judgement. The longer the refill, the poorer the circulation. Possible values are 1 = < 3 seconds and 2 = >= 3 seconds

pain

a subjective judgement of the horse's pain level

peristalsis

an indication of the activity in the horse's gut. As the gut becomes more distended or the horse becomes more toxic, the activity decreases

abdominal_distension

An animal with abdominal distension is likely to be painful and have reduced gut motility. A horse with severe abdominal distension is likely to require surgery just tio relieve the pressure

nasogastric_tube

This refers to any gas coming out of the tube. A large gas cap in the stomach is likely to give the horse discomfort

nasogastric_reflux

posible values are 1 = none, 2 = > 1 liter, 3 = < 1 liter. The greater amount of reflux, the more likelihood that there is some serious obstruction to the fluid passage from the rest of the intestine

nasogastric_reflux_PH

scale is from 0 to 14 with 7 being neutral. Normal values are in the 3 to 4 range

rectal_examination

Rectal examination. Absent feces probably indicates an obstruction

abdomen

abdomen. possible values 1 = normal, 2 = other, 3 = firm feces in the large intestine, 4 = distended small intestine, 5 = distended large intestine

cell_volume

packed cell volume. normal range is 30 to 50. The level rises as the circulation becomes compromised or as the animal becomes dehydrated.

protein

total protein. Normal values lie in the 6-7.5 (gms/dL) range. The higher the value the greater the dehydration

abdominocentesis_appearance

Abdominocentesis appearance. A needle is put in the horse's abdomen and fluid is obtained from the abdominal cavity

abdomcentesis_protein

abdomcentesis total protein. The higher the level of protein the more likely it is to have a compromised gut. Values are in gms/dL

outcome

What eventually happened to the horse?

surgical_lesion

retrospectively, was the problem (lesion) surgical?

lesion_type1

type of lesion

lesion_type2

type of lesion

lesion_type3

type of lesion

cp_data

temp_extreme_ordered

temperature of extremities (ordered)

mucous_membranes_col

mucous membranes. A subjective measurement of colour

mucous_membranes_group

different recodings of mucous membrances

Source

https://archive.ics.uci.edu/ml/datasets/Horse+Colic Creators: Mary McLeish & Matt Cecile, Department of Computer Science, University of Guelph, Guelph, Ontario, Canada N1G 2W1 Donor: Will Taylor

Examples

data(colic)
aggr(colic)

Subset of the collision data

Description

Subset of the collision data from December 20. to December 31. 2018 from NYCD.

Details

Each record represents a collision in NYC by city, borough, precinct and cross street.

Source

https://data.cityofnewyork.us/Public-Safety/NYPD-Motor-Vehicle-Collisions/h9gi-nx95

Examples

data(collisions)
aggr(collisions)

Colored map with information about missing/imputed values

Description

Colored map in which the proportion or amount of missing/imputed values in each region is coded according to a continuous or discrete color scheme. The sequential color palette may thereby be computed in the HCL or the RGB color space.

Usage

colormapMiss(
  x,
  region,
  map,
  imp_index = NULL,
  prop = TRUE,
  polysRegion = 1:length(x),
  range = NULL,
  n = NULL,
  col = c("red", "orange"),
  gamma = 2.2,
  fixup = TRUE,
  coords = NULL,
  numbers = TRUE,
  digits = 2,
  cex.numbers = 0.8,
  col.numbers = par("fg"),
  legend = TRUE,
  interactive = TRUE,
  ...
)

colormapMissLegend(
  xleft,
  ybottom,
  xright,
  ytop,
  cmap,
  n = 1000,
  horizontal = TRUE,
  digits = 2,
  cex.numbers = 0.8,
  col.numbers = par("fg"),
  ...
)

Arguments

Details

The proportion or amount of missing/imputed values in x of each region is coded according to a continuous or discrete color scheme in the color range defined by col. In addition, the proportions or numbers can be shown as labels in the regions.

If interactive is TRUE, clicking in a region displays more detailed information about missing/imputed values on the console. Clicking outside the borders quits the interactive session.

Value

colormapMiss returns a list with the following components:

• nmiss a numeric vector containing the number of missing/imputed values in each region.

• nobs a numeric vector containing the number of observations in each region.

• pmiss a numeric vector containing the proportion of missing values in each region.

• prop a logical indicating whether the proportion of missing/imputed values have been used rather than the total amount.

• range the range of the proportion or amount of missing/imputed values corresponding to the color range.

• n either a positive integer giving the number of equally spaced cut-off points for a discretized color scheme, or NULL for a continuous color scheme.

• start the start color of the color scheme.

• end the end color of the color scheme.

• space a character string giving the color space (either "rgb" for RGB colors or "hcl" for HCL colors).

• gamma numeric; the display gamma value (see colorspace::hex()).

• fixup a logical indicating whether the colors have been corrected to valid RGB values (see colorspace::hex()).

Note

Some of the argument names and positions have changed with versions 1.3 and 1.4 due to extended functionality and for more consistency with other plot functions in VIM. For back compatibility, the arguments cex.text and col.text can still be supplied to ...{} and are handled correctly. Nevertheless, they are deprecated and no longer documented. Use cex.numbers and col.numbers instead.

Author(s)

Andreas Alfons, modifications to show imputed values by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

colSequence(), growdotMiss(), mapMiss()

Complete model diagnostics from learner predictions when the raw model does not expose them directly

Description

Complete model diagnostics from learner predictions when the raw model does not expose them directly

Usage

complete_model_info(info, learner = NULL, task = NULL)

Arguments

Value

Model info list with residuals/scale filled where possible

Count number of infinite or missing values

Description

Count the number of infinite or missing values in a vector.

Usage

countInf(x)

Arguments

Value

countInf returns the number of infinite values in x. countNA returns the number of missing values in x.

Author(s)

Andreas Alfons

Examples

data(sleep, package="VIM")
countInf(log(sleep$Dream))
countNA(sleep$Dream)

Synthetic Pima Indians Diabetes Data

Description

A synthetic replacement for the Pima Indians Diabetes data: 768 observations on 9 variables, a binary target (Outcome) and medical predictors such as the number of pregnancies, plasma glucose, BMI, insulin level and age. Every value is synthetic; no row corresponds to a real person.

Format

A data frame with 768 observations on the following 9 variables.

Pregnancies

Number of times pregnant (0 is a valid value)

Glucose

Plasma glucose concentration at 2 hours in an oral glucose tolerance test

BloodPressure

Diastolic blood pressure (mm Hg)

SkinThickness

Triceps skin fold thickness (mm)

Insulin

2-Hour serum insulin (mu U/ml)

BMI

Body mass index (weight in kg/(height in m)^2)

DiabetesPedigreeFunction

Diabetes pedigree function

Age

Age in years

Outcome

Diabetes (yes or no)

Details

The original data were collected by the National Institute of Diabetes and Digestive and Kidney Diseases from female patients of Pima Indian heritage aged at least 21. The team that collected them has stated that the data should not have been distributed and has asked for redistribution to stop, while a large body of teaching material and package examples depends on them. This synthetic version exists so that such material keeps working without real patient records being circulated.

The schema follows PimaIndiansDiabetes2 from package mlbench: physically impossible values (a zero glucose level, blood pressure, skin fold thickness, insulin level or BMI) are coded as NA, whereas 0 pregnancies is a valid value. Column names, types and the levels of Outcome are those VIM has used since version 6, so existing code runs unchanged; only the values differ from the real data shipped before VIM 7.3.0.

Missing values in the original are not scattered at random: BloodPressure, SkinThickness and Insulin are frequently missing together, and the pattern is age-related. The pattern itself was synthesised as a variable and the missing values imposed from it, so that complete-case and imputation examples behave as they did before (placing missing values independently would produce the joint pattern about five times instead of the observed 19 to 24).

Results computed on these data resemble published analyses of the original without matching them: a logistic regression reaches an AUC of 0.868 against 0.862 on the original, and a model fitted only on the synthetic data reaches 0.854 on the real records. Individual correlations are less reliable – most come out slightly weaker than in the original, but the association between number of pregnancies and age is considerably stronger (0.74 against 0.54). Use the data for teaching, examples and method demonstrations; any result computed on them describes the synthesis, not the Pima population, and must not be read as evidence about diabetes.

Author(s)

Matthias Templ matthias.templ@fhnw.ch (synthetic version)

Source

Synthetic. Generated by Matthias Templ from PimaIndiansDiabetes2 (package mlbench) with the synvey package – sequential robust synthesis with a random-forest backend and the missingness pattern treated as a structural variable (synvey 0.1.0, R 4.5.2, seed 20260804):

SynthDiabetes <- synvey::robsynth(PimaIndiansDiabetes2,
                                  method      = "robust_rf",
                                  missingness = "structural",
                                  coarsen     = "age",
                                  target      = "sample",
                                  seed        = 20260804)$synth

This configuration was selected from a comparison of five synthesis packages under two designs for the missingness mechanism. Candidates reproducing any original record verbatim were excluded; the data set contains no such record. synvey is not on CRAN; it is available from https://github.com/matthias-da/synvey. For VIM the columns were renamed to the historical names (pregnant, glucose, pressure, triceps, insulin, mass, pedigree, age, diabetes became Pregnancies, Glucose, BloodPressure, SkinThickness, Insulin, BMI, DiabetesPedigreeFunction, Age, Outcome), counts are stored as integers and the Outcome levels neg/pos as no/yes. The original data are neither distributed with VIM nor needed to use this data set.

References

Smith, J.W., Everhart, J.E., Dickson, W.C., Knowler, W.C., & Johannes, R.S. (1988). Using the ADAP learning algorithm to forecast the onset of diabetes mellitus. In Proceedings of the Symposium on Computer Applications and Medical Care (pp. 261–265). IEEE Computer Society Press. https://pmc.ncbi.nlm.nih.gov/articles/PMC2245318/ (The study behind the original data; the original data are not redistributed here.)

Examples

data(diabetes)
summary(diabetes)

# missingness is structured rather than scattered
aggr(diabetes)

Error performance measures

Description

Various error measures evaluating the quality of imputations

Usage

evaluation(x, y, m, vartypes = "guess", where = NULL)

nrmse(x, y, m)

pfc(x, y, m)

msecov(x, y)

msecor(x, y)

Arguments

Details

This function has been mainly written for procudures that evaluate imputation or replacement of rounded zeros. The ni parameter can thus, e.g. be used for expressing the number of rounded zeros.

Value

the error measures value

Author(s)

Matthias Templ

References

M. Templ, A. Kowarik, P. Filzmoser (2011) Iterative stepwise regression imputation using standard and robust methods. Computational Statistics & Data Analysis, Vol. 55, pp. 2793-2806.

Examples

data(iris)
iris_orig <- iris_imp <- iris
iris_imp$Sepal.Length[sample(1:nrow(iris), 10)] <- NA
iris_imp$Sepal.Width[sample(1:nrow(iris), 10)] <- NA
iris_imp$Species[sample(1:nrow(iris), 10)] <- NA
m <- is.na(iris_imp)
iris_imp <- kNN(iris_imp, imp_var = FALSE)
evaluation(iris_orig, iris_imp, m = m, vartypes = c(rep("numeric", 4), "factor"))
msecov(iris_orig[, 1:4], iris_imp[, 1:4])

Extract model diagnostics for bootstrap strategies

Description

Drills into an mlr3 learner or raw model to retrieve residuals, scale estimate, and robustness weights needed for bootstrap resampling.

Usage

extract_model_info(model, method = "robust")

Arguments

Value

List with components: residuals (numeric vector or NULL), scale (numeric scalar or NULL), weights (numeric vector or NULL)

Food consumption

Description

The relative consumption of certain food items in European and Scandinavian countries.

Format

A data frame with 16 observations on the following 21 variables.

Details

The numbers represent the percentage of the population consuming that food type.

Source

https://openmv.net/info/food-consumption

Examples

data(food)
str(food)
aggr(food)

Missing value gap statistics

Description

Computes the average missing value gap of a vector.

Usage

gapMiss(x, what = mean)

Arguments

Details

The length of each sequence of missing values (gap) in a vector is calculated and the mean gap is reported

Value

The gap statistics

Author(s)

Matthias Templ based on a suggestion and draft from Huang Tian Yuan.

Examples

v <- rnorm(20)
v[3] <- NA
v[6:9] <- NA
v[13:17] <- NA
v
gapMiss(v)
gapMiss(v, what = median)
gapMiss(v, what = function(x) mean(x, trim = 0.1))
gapMiss(v, what = var)

Computes the extended Gower distance of two data sets

Description

The function gowerD is used by kNN to compute the distances for numerical, factor ordered and semi-continous variables.

Usage

gowerD(
  data.x,
  data.y = data.x,
  weights = rep(1, ncol(data.x)),
  numerical = colnames(data.x),
  factors = vector(),
  orders = vector(),
  mixed = vector(),
  levOrders = vector(),
  mixed.constant = rep(0, length(mixed)),
  returnIndex = FALSE,
  nMin = 1L,
  returnMin = FALSE,
  methodStand = "range"
)

Arguments

Details

Numerical and semi-continuous (mixed) variables are standardised by their range (or IQR, see methodStand) before the distance is computed; for mixed variables the point mass (mixed.constant) is excluded from the range estimate and scaled along with the data.

Missing values in a distance variable are replaced by a sentinel (the pooled column maximum plus one) after standardisation. Two records both missing the same variable therefore have distance 0 on it, and a missing-vs-observed pair contributes up to about 2 (rather than a variable's nominal maximum of 1). This sentinel convention departs from Gower's (1971) omit-and-renormalise rule and biases nearest-neighbour selection towards records that share the same missingness pattern.

The distance computation is parallelised with OpenMP where R was built with OpenMP support. The number of threads is getOption("VIM.ncores"); if the option is unset, at most 2 threads are used while the package is checked (⁠R CMD check⁠ sets ⁠_R_CHECK_LIMIT_CORES_⁠; CRAN policy allows at most two cores) and OpenMP's default – typically all cores, or OMP_NUM_THREADS – otherwise. Results do not depend on the number of threads.

returnIndex=FALSE: a numerical matrix n x m with the computed distances returnIndex=TRUE: a named list with "ind" containing the requested indices and "mins" the computed distances

Examples

data(sleep)
# all variables used as numerical
gowerD(sleep)

# split in numerical an
gowerD(sleep, numerical = c("BodyWgt", "BrainWgt", "NonD", "Dream", "Sleep", "Span", "Gest"),
  orders = c("Pred","Exp","Danger"), levOrders = c(5,5,5))

# as before but only returning the index of the closest observation
gowerD(sleep, numerical = c("BodyWgt", "BrainWgt", "NonD", "Dream", "Sleep", "Span", "Gest"),
  orders = c("Pred","Exp","Danger"), levOrders = c(5,5,5), returnIndex = TRUE)

Growing dot map with information about missing/imputed values

Description

Map with dots whose sizes correspond to the values in a certain variable. Observations with missing/imputed values in additional variables are highlighted.

Usage

growdotMiss(
  x,
  coords,
  map,
  pos = 1,
  delimiter = NULL,
  selection = c("any", "all"),
  log = FALSE,
  col = c("skyblue", "red", "skyblue4", "red4", "orange", "orange4"),
  border = par("bg"),
  alpha = NULL,
  scale = NULL,
  size = NULL,
  exp = c(0, 0.95, 0.05),
  col.map = grey(0.5),
  legend = TRUE,
  legtitle = "Legend",
  cex.legtitle = par("cex"),
  cex.legtext = par("cex"),
  ncircles = 6,
  ndigits = 1,
  interactive = TRUE,
  ...
)

Arguments

Details

The smallest dots correspond to the 10\ the 99\ defining the shape of the exponential function. Missings/imputed missings in the variable of interest will be drawn as rectangles.

If interactive=TRUE, detailed information for an observation can be printed on the console by clicking on the corresponding point. Clicking in a region that does not contain any points quits the interactive session.

Note

The function was renamed to growdotMiss in version 1.3. bubbleMiss is a (deprecated) wrapper for growdotMiss for back compatibility with older versions. However, due to extended functionality, some of the argument positions have changed.

The code is based on (removed from CRAN) bubbleFIN from package StatDA.

Author(s)

Andreas Alfons, Matthias Templ, Peter Filzmoser, Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

bgmap(), mapMiss(), colormapMiss()

Examples

data(chorizonDL, package = "VIM")
data(kola.background, package = "VIM")
coo <- chorizonDL[, c("XCOO", "YCOO")]
## for missing values
x <- chorizonDL[, c("Ca","As", "Bi")]
growdotMiss(x, coo, kola.background, border = "white")

## for imputed values
x_imp <- kNN(chorizonDL[,c("Ca","As","Bi" )])
growdotMiss(x_imp, coo, kola.background, delimiter = "_imp", border = "white")

Histogram with information about missing/imputed values

Description

Histogram with highlighting of missing/imputed values in other variables by splitting each bin into two parts. Additionally, information about missing/imputed values in the variable of interest is shown on the right hand side.

Usage

histMiss(
  x,
  delimiter = NULL,
  pos = 1,
  selection = c("any", "all"),
  breaks = "Sturges",
  right = TRUE,
  col = c("skyblue", "red", "skyblue4", "red4", "orange", "orange4"),
  border = NULL,
  main = NULL,
  sub = NULL,
  xlab = NULL,
  ylab = NULL,
  axes = TRUE,
  only.miss = TRUE,
  miss.labels = axes,
  interactive = TRUE,
  ...
)

Arguments

Details

If more than one variable is supplied, the bins for the variable of interest will be split according to missingness/number of imputed missings in the additional variables.

If only.miss=TRUE, the missing/imputed values in the variable of interest are visualized by one bar on the right hand side. If additional variables are supplied, this bar is again split into two parts according to missingness/number of imputed missings in the additional variables.

Otherwise, a small barplot consisting of two bars is drawn on the right hand side. The first bar corresponds to observed values in the variable of interest and the second bar to missing/imputed values. Since these two bars are not on the same scale as the main barplot, a second y-axis is plotted on the right (if axes=TRUE). Each of the two bars are again split into two parts according to missingness/number of imputed missings in the additional variables. Note that this display does not make sense if only one variable is supplied, therefore only.miss is ignored in that case.

If interactive=TRUE, clicking in the left margin of the plot results in switching to the previous variable and clicking in the right margin results in switching to the next variable. Clicking anywhere else on the graphics device quits the interactive session. When switching to a categorical variable, a barplot is produced rather than a histogram.

Value

a list with the following components:

• breaks the breakpoints.

• counts the number of observations in each cell.

• missings the number of highlighted observations in each cell.

• mids the cell midpoints.

Note

Some of the argument names and positions have changed with version 1.3 due to extended functionality and for more consistency with other plot functions in VIM. For back compatibility, the arguments axisnames and names.miss can still be supplied to ...{} and are handled correctly. Nevertheless, they are deprecated and no longer documented. Use miss.labels instead.

Author(s)

Andreas Alfons, Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

spineMiss(), barMiss()

Other plotting functions: aggr(), barMiss(), marginmatrix(), marginplot(), matrixplot(), mosaicMiss(), pairsVIM(), parcoordMiss(), pbox(), scattJitt(), scattMiss(), scattmatrixMiss(), spineMiss()

Examples

data(tao, package = "VIM")
## for missing values
x <- tao[, c("Air.Temp", "Humidity")]
histMiss(x)
histMiss(x, only.miss = FALSE)

## for imputed values
x_IMPUTED <- kNN(tao[, c("Air.Temp", "Humidity")])
histMiss(x_IMPUTED, delimiter = "_imp")
histMiss(x_IMPUTED, delimiter = "_imp", only.miss = FALSE)

Hot-Deck Imputation

Description

Implementation of the popular Sequential, Random (within a domain) hot-deck algorithm for imputation.

Usage

hotdeck(
  data,
  variable = NULL,
  ord_var = NULL,
  domain_var = NULL,
  makeNA = NULL,
  NAcond = NULL,
  impNA = TRUE,
  donorcond = NULL,
  imp_var = TRUE,
  imp_suffix = "imp"
)

Arguments

Value

the imputed data set.

Note

If the sequential hotdeck does not lead to a suitable, a random donor in the group will be used.

Author(s)

Alexander Kowarik

References

A. Kowarik, M. Templ (2016) Imputation with R package VIM. Journal of Statistical Software, 74(7), 1-16.

See Also

Other imputation methods: impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

data(sleep)
sleepI <- hotdeck(sleep)
sleepI2 <- hotdeck(sleep,ord_var="BodyWgt",domain_var="Pred")

# Usage of donorcond in a simple example
sleepI3 <- hotdeck(
  sleep,
  variable = c("NonD", "Dream", "Sleep", "Span", "Gest"),
  ord_var = "BodyWgt", domain_var = "Pred",
  donorcond = list(">4", "<17", ">1.5", "%between%c(8,13)", ">5")
)

set.seed(132)
nRows <- 1e3
# Generate a data set with nRows rows and several variables
x <- data.frame(
  x = rnorm(nRows), y = rnorm(nRows),
  z = sample(LETTERS, nRows, replace = TRUE),
  d1 = sample(LETTERS[1:3], nRows, replace = TRUE),
  d2 = sample(LETTERS[1:2], nRows, replace = TRUE),
  o1 = rnorm(nRows), o2 = rnorm(nRows), o3 = rnorm(100)
)
origX <- x
x[sample(1:nRows,nRows/10), 1] <- NA
x[sample(1:nRows,nRows/10), 2] <- NA
x[sample(1:nRows,nRows/10), 3] <- NA
x[sample(1:nRows,nRows/10), 4] <- NA
xImp <- hotdeck(x,ord_var = c("o1", "o2", "o3"), domain_var = "d2")

Huber weight function

Description

Computes Huber weights for standardized values. Returns 1 for values within the tuning constant and k / abs(u) for values outside.

Usage

huber_weight(u, k = 1.345)

Arguments

Value

numeric vector of weights in [0, 1]

Author(s)

Matthias Templ

Iterative EM PCA imputation

Description

Greedy algorithm for EM-PCA including robust methods

Usage

impPCA(
  x,
  method = "classical",
  m = 1,
  eps = 0.5,
  k = ncol(x) - 1,
  maxit = 100,
  boot = FALSE,
  verbose = TRUE
)

Arguments

Value

the imputed data set. If boot = FALSE this is a data.frame. If boot = TRUE this is a list where each list element contains a data.frame.

Author(s)

Matthias Templ

References

Serneels, Sven and Verdonck, Tim (2008). Principal component analysis for data containing outliers and missing elements. Computational Statistics and Data Analysis, Elsevier, vol. 52(3), pages 1712-1727

See Also

Other imputation methods: hotdeck(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

data(Animals, package = "MASS")
Animals$brain[19] <- Animals$brain[19] + 0.01
Animals <- log(Animals)
colnames(Animals) <- c("log(body)", "log(brain)")
Animals_na <- Animals
probs <- abs(Animals$`log(body)`^2)
probs <- rep(0.5, nrow(Animals))
probs[c(6,16,26)] <- 0
set.seed(1234)
Animals_na[sample(1:nrow(Animals), 10, prob = probs), "log(brain)"] <- NA
w <- is.na(Animals_na$`log(brain)`)
impPCA(Animals_na)
impPCA(Animals_na, method = "mcd")
impPCA(Animals_na, boot = TRUE, m = 10)
impPCA(Animals_na, method = "mcd", boot = TRUE)[[1]]
plot(`log(brain)` ~ `log(body)`, data = Animals, type = "n", ylab = "", xlab="")
mtext(text = "impPCA robust", side = 3)
points(Animals$`log(body)`[!w], Animals$`log(brain)`[!w])
points(Animals$`log(body)`[w], Animals$`log(brain)`[w], col = "grey", pch = 17)
imputed <- impPCA(Animals_na, method = "mcd", boot = TRUE)[[1]]
colnames(imputed) <- c("log(body)", "log(brain)")
points(imputed$`log(body)`[w], imputed$`log(brain)`[w], col = "red", pch = 20, cex = 1.4)
segments(x0 = Animals$`log(body)`[w], x1 = imputed$`log(body)`[w], y0 = Animals$`log(brain)`[w],
y1 = imputed$`log(brain)`[w], lty = 2, col = "grey")
legend("topleft", legend = c("non-missings", "set to missing", "imputed values"),
pch = c(1,17,20), col = c("black","grey","red"), cex = 0.7)
mape <- round(100* 1/sum(is.na(Animals_na$`log(brain)`)) * sum(abs((Animals$`log(brain)` -
imputed$`log(brain)`) / Animals$`log(brain)`)), 2)
s2 <- var(Animals$`log(brain)`)
nrmse <- round(sqrt(1/sum(is.na(Animals_na$`log(brain)`)) * sum(abs((Animals$`log(brain)` -
imputed$`log(brain)`) / s2))), 2)
text(x = 8, y = 1.5, labels = paste("MAPE =", mape))
text(x = 8, y = 0.5, labels = paste("NRMSE =", nrmse))

Cellwise-robust EM imputation for mixed data

Description

EM algorithm with latent contamination indicators that jointly estimates clean distribution parameters and identifies cellwise outliers. Each continuous cell has a posterior probability of being clean vs contaminated. The clean distribution is modeled as multivariate normal for continuous variables, with categorical variables handled via conditional multinomial logistic regression.

Usage

imputeCellEM(
  data,
  maxit_em = 100,
  eps_em = 0.005,
  gamma_init = 3,
  eps_init = 0.1,
  uncert = "conditional",
  conditioning = "weighted",
  trust_min = 0.5,
  trace = FALSE
)

Arguments

Details

The algorithm proceeds as follows:

1. Initialization. Missing values are filled by initialise (medians for continuous, modes for categorical). Initial location and scale are estimated robustly (median and MAD). Cell weights are initialized to 1.

2. E-step (for each continuous variable j, each observation i):

   • Compute the conditional mean and variance of x_{ij} given the other continuous variables, using the current \mu and \Sigma.

   • For observed cells: compute the posterior probability that the cell is clean vs contaminated, yielding cell weight w_{ij}.

   • For missing cells: impute from the conditional distribution (with optional PMM).

3. M-step:

   • Update \mu: cell-weighted mean.

   • Update \Sigma: pairwise cell-weighted covariance using w_{ij} \cdot w_{ik} (not row-level min).

   • Update contamination rates: \varepsilon_j = 1 - \mathrm{mean}(w_{ij}) over observed cells.

   • Update contamination scale \gamma_j from weighted variance of contaminated cells.

   • For categorical variables: fit weighted multinomial logistic with row weights derived from continuous cell weights.

4. Convergence. Check relative change in estimated parameters (\mu, \Sigma); stop when below eps_em or after maxit_em iterations. The observed-data log-likelihood (sum of per-variable conditional mixture log-likelihoods) is tracked for diagnostics.

This method differs from cellGMM (Zaccaria et al., 2025) in using a single clean component rather than a mixture of clean clusters, and in supporting mixed continuous + categorical data.

The pseudo_loglik component is a composite (pseudo) log-likelihood: the sum of per-variable conditional mixture log-likelihoods, not the proper observed-data joint log-likelihood. It is useful for monitoring convergence but should not be compared across models or used for model selection criteria such as AIC/BIC.

This implementation is an ECM (Expectation Conditional Maximization) variant rather than a pure EM algorithm, because the conditional variance in the E-step is computed using the updated Sigma from the current M-step rather than the Sigma from the previous iteration. As a result, strict log-likelihood monotonicity is not guaranteed, but is observed empirically in practice.

Value

A list with components:

Note

Model uncertainty via bootstrap (Rubin's combining rules for multiple imputation) is not yet implemented. The current version provides single imputation with stochastic uncertainty (PMM or residual draw). For valid multiple imputation, call the function repeatedly with different seeds and combine using Rubin's rules.

Author(s)

Matthias Templ

References

A.P. Dempster, N.M. Laird, D.B. Rubin (1977) Maximum Likelihood from Incomplete Data via the EM Algorithm. Journal of the Royal Statistical Society: Series B, 39(1), 1–38.

C.F.J. Wu (1983) On the Convergence Properties of the EM Algorithm. The Annals of Statistics, 11(1), 95–103.

J. Raymaekers, P.J. Rousseeuw (2024) The cellwise minimum covariance determinant estimator. Journal of the American Statistical Association, 119(548), 2610–2621.

G. Zaccaria, L.A. Garcia-Escudero, F. Greselin, A. Mayo-Iscar (2025) Cellwise outlier detection in heterogeneous populations. Technometrics, 67(4), 643–654.

M. Templ, A. Kowarik, P. Filzmoser (2011) Iterative stepwise regression imputation using standard and robust methods. Computational Statistics & Data Analysis, Vol. 55, pp. 2793–2806.

See Also

imputeCellIRMI, imputeCellM, initialise, irmi

Other imputation methods: hotdeck(), impPCA(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

data(sleep, package = "VIM")
result <- imputeCellEM(sleep)
head(result$data_imputed)

# Inspect estimated contamination rates
result$epsilon

# Cell weight matrix (1 = clean, low = likely contaminated)
image(result$cellweights, main = "Cell weights")

# Log-likelihood trace
plot(result$pseudo_loglik, type = "b", xlab = "Iteration",
     ylab = "Pseudo log-likelihood")

# With predictive mean matching for imputation
result2 <- imputeCellEM(sleep, uncert = "pmm", trace = TRUE)

# Mixed data example
data(testdata)
result3 <- imputeCellEM(testdata$wna)

Cellwise-robust iterative regression imputation for mixed data

Description

Extends IRMI (Templ, Kowarik, and Filzmoser, 2011) with cellwise contamination handling. Each conditional regression uses a cell-weighted IRWLS engine where per-cell weights in the design matrix downweight contaminated cells without discarding entire observations.

Usage

imputeCellIRMI(
  data,
  method = "tukey",
  alpha = NULL,
  maxit = 100,
  maxit_irwls = 50,
  eps = 0.005,
  eps_irwls = 1e-06,
  uncert = "pmm",
  weight_update = "multivariate",
  init_weights = "ddc",
  hard_threshold = 0.5,
  trace = FALSE
)

Arguments

Details

The algorithm works iteratively: in each outer iteration, every variable with missing values is used as response in a conditional regression on all remaining variables. For continuous responses, the custom cellIRWLS() engine fits a weighted regression where each cell in the design matrix receives its own weight reflecting potential cellwise contamination. For categorical responses, a weighted multinomial model is used. After each regression, cell weights for the response variable are updated from the residuals.

The algorithm proceeds as follows:

1. Missing values are initialised using initialise.

2. Initial cell weights are computed with cellWeights() on all continuous variables in the initialised data.

3. Outer loop (up to maxit iterations):

   • For each variable j with missing values:

     • Form predictor matrix X (all other variables) and response y (variable j).

     • If j is continuous: fit cellIRWLS(X, y, w_cell, w_response) and impute missing values in j using the fitted model plus uncertainty.

     • If j is categorical: fit nnet::multinom() with row weights derived from the cell weight matrix and impute by sampling from predicted probabilities.

     • Update cell weights for j from residuals via cellWeightsFromResiduals().

   • Check convergence: relative change in imputed values falls below eps.

Value

A list with components:

Author(s)

Matthias Templ

References

Templ, M., Kowarik, A. and Filzmoser, P. (2011). Iterative stepwise regression imputation using standard and robust methods. Computational Statistics & Data Analysis, 55(10), 2793–2806.

See Also

imputeCellM, imputeCellEM, initialise, irmi

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

data(sleep, package = "VIM")
result <- imputeCellIRMI(sleep)
head(result$data_imputed)
image(result$cellweights, main = "Cell weights")

# With Huber weights (less aggressive downweighting)
result2 <- imputeCellIRMI(sleep, method = "huber", trace = TRUE)

# Mixed data example
data(testdata)
result3 <- imputeCellIRMI(testdata$wna)

Cellwise M-estimation imputation

Description

Impute missing values using a cell-weighted M-estimation approach. Each cell in the predictor matrix receives its own weight reflecting potential cellwise contamination, so that contaminated predictor cells are downweighted without discarding entire observations.

Usage

imputeCellM(
  formula,
  data,
  method = "tukey",
  alpha = NULL,
  maxit_irwls = 50,
  eps_irwls = 1e-06,
  uncert = "pmm",
  value_back = "all",
  maxit = 10,
  eps = 0.005,
  trace = FALSE
)

Arguments

Details

The function has two interfaces: with a model formula, a single response variable is imputed from the specified predictors; with a data.frame (or matrix) as first argument, all variables with missing values are imputed by chained equations, i.e. each such variable is regressed on all remaining variables and the sweeps are iterated until the imputed values stabilise.

The formula interface is a lightweight single-response alternative to imputeCellIRMI. It fits one cell-weighted IRWLS regression using cellIRWLS() and imputes the missing values in the response variable. This is appropriate when only one variable needs imputation and a specific model formula is desired.

The data.frame interface runs the same per-variable machinery as a chained-equations algorithm: missing values are initialised (median/mode), then each variable with missing values in turn is used as response in a formula containing all remaining variables. Sweeps use deterministic predictions and are iterated until the relative change of the imputed values falls below eps (or maxit is reached); the requested uncert step is applied once after convergence.

For categorical response variables, a weighted multinomial model via multinom is fitted instead. Categorical predictors are not subject to the cellwise contamination model (their cell weights are always 1).

Value

If value_back = "ymiss", a named vector of imputed values (for rows that were originally missing) is returned. Otherwise, a list with components:

data_imputed

the imputed data.frame (same structure as input)

cellweights

n x p matrix of final cell weights (1 = clean, 0 = fully downweighted). Categorical columns always have weight 1.

converged

logical; always TRUE for single-formula imputation, convergence of the sweeps for the data.frame interface

iterations

integer; always 1L for single-formula imputation, the number of sweeps for the data.frame interface

Note

Model uncertainty via bootstrap (Rubin's combining rules for multiple imputation) is not yet implemented. The current version provides single imputation with stochastic uncertainty (PMM or residual draw). For valid multiple imputation, call the function repeatedly with different seeds and combine using Rubin's rules.

Author(s)

Matthias Templ

References

M. Templ, A. Kowarik, P. Filzmoser (2011) Iterative stepwise regression imputation using standard and robust methods. Computational Statistics & Data Analysis, Vol. 55, pp. 2793-2806.

See Also

imputeCellIRMI, imputeRobust

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

data(sleep, package = "VIM")
# Impute Dream using BodyWgt and BrainWgt as predictors
result <- imputeCellM(Dream ~ BodyWgt + BrainWgt, data = sleep)
head(result)

# Return only imputed values
impvals <- imputeCellM(Dream ~ BodyWgt + BrainWgt, data = sleep,
                       value_back = "ymiss")

# Huber weights (less aggressive downweighting)
result2 <- imputeCellM(Dream ~ BodyWgt + BrainWgt, data = sleep,
                       method = "huber")

# Chained-equations interface: impute all variables with missings
result3 <- imputeCellM(sleep)
head(result3$data_imputed)

Cellwise MCD-based imputation for mixed data

Description

Extends the cellwise MCD approach (Raymaekers & Rousseeuw 2024) to mixed continuous + categorical data. Uses MCD for robust covariance estimation of the continuous block, computes cellwise weights from conditional residuals, then imputes missing values via conditional expectations (continuous) and weighted multinomial regression (categorical). Iterates until convergence.

Usage

imputeCellMCD(
  data,
  maxit = 50,
  eps = 0.005,
  method = "tukey",
  alpha = NULL,
  mcd_alpha = 0.75,
  hard_threshold = 0.5,
  mcd_observed = "all",
  init_method = "median",
  uncert = "conditional",
  m = 1L,
  boot = FALSE,
  trace = FALSE
)

Arguments

Value

A list with components:

Note

Multiple imputation with proper parameter-uncertainty propagation is not yet implemented for this method (see boot). m > 1 returns repeated imputations whose between-imputation variability comes only from the stochastic uncertainty step (uncert) and estimator randomness, so downstream Rubin pooling understates total variance. Treat the current m > 1 output as approximate.

Author(s)

Matthias Templ

References

Raymaekers, J. and Rousseeuw, P.J. (2024). The cellwise minimum covariance determinant estimator. Journal of the American Statistical Association, 119(548), 2610–2621.

See Also

imputeCellIRMI, imputeCellM, imputeCellEM

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

## Not run: 
data(sleep, package = "VIM")
result <- imputeCellMCD(sleep)
head(result$data_imputed)

# Inspect cell weights
image(result$cellweights, main = "Cell weights")

# With pairwise robust covariance
result2 <- imputeCellMCD(sleep, mcd_observed = "pairwise", trace = TRUE)

## End(Not run)

Cell-weighted MM imputation for mixed data (Path A)

Description

Uses a cell-weighted MM-estimator for each variable regression: S-step on unweighted data (high breakdown), M-step with cell weights (fine-grained downweighting). This preserves the S-estimator's breakdown guarantee while incorporating cellwise information.

Usage

imputeCellMM(
  data,
  maxit = 50,
  eps = 0.005,
  alpha_weight = 4.685,
  init_weights = "mcd",
  uncert = "pmm",
  trace = FALSE
)

Arguments

Value

list with data_imputed, cellweights, converged, iterations

Author(s)

Matthias Templ

Cellwise-robust regression imputation for mixed data

Description

IRMI-style imputation using cellwise-robust regression as the inner engine. Three engines are available: CRM (Filzmoser et al. 2020), Shooting S (Öllerer et al. 2016), and a cellwise-weighted MM hybrid.

Usage

imputeCellReg(
  data,
  engine = "crm",
  maxit = 50,
  eps = 0.005,
  uncert = "pmm",
  trace = FALSE
)

Arguments

Details

The function cycles through all variables with missing values (IRMI framework), fitting a cellwise-robust regression of each variable on all others. The engine argument selects the regression method:

"crm"

CRM (Cellwise Robust M-regression) from the crmReg package. Uses SPADIMO for cellwise outlier detection within each regression. Requires crmReg.

"cellwise-mm"

Hybrid: compute cell weights via MCD conditional residuals, then fit MM-estimation (lmrob) with row weights derived from cell weights. The MM-estimator provides high breakdown point.

"shooting-s"

Shooting S-estimator (Öllerer et al. 2016). Iterates between cellwise detection and S-estimation. Implemented from the published algorithm.

Categorical variables are imputed via weighted multinomial logistic regression, with row weights derived from the continuous cell weights.

Value

A list with components:

Author(s)

Matthias Templ

References

P. Filzmoser, S. Höppner, I. Ortner, S. Serneels, S. Van Aelst (2020) Cellwise robust M regression. Computational Statistics and Data Analysis, 147, 106944.

V. Öllerer, A. Alfons, C. Croux (2016) The shooting S-estimator for robust regression. Computational Statistics, 31(3), 829–844.

Unified cellwise-robust imputation dispatcher

Description

Convenience wrapper that dispatches to one of the three cellwise-robust imputation methods: imputeCellIRMI, imputeCellM, or imputeCellEM.

Usage

imputeCellwise(data, method = c("cellIRMI", "cellM", "cellEM"), ...)

Arguments

Value

The return value of the dispatched function. See the documentation of the individual methods for details.

Note

Model uncertainty via bootstrap (Rubin's combining rules for multiple imputation) is not yet implemented. The current version provides single imputation with stochastic uncertainty (PMM or residual draw). For valid multiple imputation, call the function repeatedly with different seeds and combine using Rubin's rules.

Author(s)

Matthias Templ

See Also

imputeCellIRMI, imputeCellM, imputeCellEM

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

data(sleep, package = "VIM")
result <- imputeCellwise(sleep, method = "cellIRMI")
head(result$data_imputed)

Robust imputation

Description

Multiple imputation using classical and robust methods accounting for model and imputation uncertainty.

Usage

imputeRobust(
  form,
  data,
  boot = TRUE,
  robustboot = "stratified",
  method = "MM",
  takeAll = TRUE,
  alpha = 0.75,
  uncert = "pmm",
  family = "Gaussian",
  value_back = "all"
)

Arguments

Details

Complex formulas can be provided for each variable in your data set.

Value

Imputed data set.

See Also

initialise lmrob gam pdist

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

## Not run: 
if(interactive()){
 #EXAMPLE1
 }

## End(Not run)

FUNCTION_TITLE

Description

FUNCTION_DESCRIPTION

Usage

imputeRobustChain(
  formulas = vector("list", ncol(data)),
  data,
  boot = TRUE,
  robustboot = TRUE,
  method = "lts",
  multinom.method = "multinom",
  takeAll = TRUE,
  eps = 0.5,
  maxit = 4,
  alpha = 0.5,
  uncert = "pmm",
  familiy = "Gaussian",
  value_back = "matrix",
  trace = FALSE
)

Arguments

Details

DETAILS

Value

OUTPUT_DESCRIPTION

See Also

initialise outCoDa

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

## Not run: 
if(interactive()){
 #EXAMPLE1
 }

## End(Not run)

Initialization of missing values

Description

Rough estimation of missing values in a vector according to its type.

Usage

initialise(x, mixed, method = "kNN", mixed.constant = NULL)

Arguments

Details

Missing values are imputed with the mean for vectors of class "numeric", with the median for vectors of class "integer", and with the mode for vectors of class "factor". Hence, x should be prepared in the following way: assign class "numeric" to numeric vectors, assign class "integer" to ordinal vectors, and assign class "factor" to nominal or binary vectors.

Value

the initialized vector.

Note

The function is used internally by some imputation algorithms.

Author(s)

Matthias Templ, modifications by Andreas Alfons

Inject imputation uncertainty into predictions

Description

Adds stochastic noise to point predictions to properly reflect imputation uncertainty. Called after model prediction, before value assignment.

Usage

inject_uncertainty(
  preds,
  method = "none",
  scale = NULL,
  residuals = NULL,
  y_obs = NULL,
  score_obs = NULL,
  score_miss = NULL,
  pmm_k = 5L,
  pmm_k_method = "random",
  X_obs = NULL,
  X_miss = NULL
)

Arguments

Value

Numeric vector of adjusted predictions (same length as preds)

Iterative robust model-based imputation (IRMI)

Description

In each step of the iteration, one variable is used as a response variable and the remaining variables serve as the regressors.

Usage

irmi(
  x,
  eps = 5,
  maxit = 100,
  mixed = NULL,
  mixed.constant = NULL,
  count = NULL,
  step = FALSE,
  robust = FALSE,
  takeAll = TRUE,
  noise = TRUE,
  noise.factor = 1,
  force = FALSE,
  robMethod = "lmrob",
  force.mixed = TRUE,
  mi = 1,
  addMixedFactors = FALSE,
  trace = FALSE,
  init.method = "kNN",
  modelFormulas = NULL,
  multinom.method = "multinom",
  imp_var = TRUE,
  imp_suffix = "imp"
)

Arguments

Details

The method works sequentially and iterative. The method can deal with a mixture of continuous, semi-continuous, ordinal and nominal variables including outliers.

A full description of the method can be found in the mentioned reference.

Value

the imputed data set.

Note

Deprecated: irmi() is considered legacy functionality. For new projects, please use vimpute() instead, which provides a more modern, flexible and robust imputation framework.

Author(s)

Matthias Templ, Alexander Kowarik

References

M. Templ, A. Kowarik, P. Filzmoser (2011) Iterative stepwise regression imputation using standard and robust methods. Computational Statistics & Data Analysis, Vol. 55, pp. 2793-2806.

A. Kowarik, M. Templ (2016) Imputation with R package VIM. Journal of Statistical Software, 74(7), 1-16.

M. Templ (2023) Visualization and Imputation of Missing Values. Springer Publishing. Series in Computational Statistics. Cham. Switzerland. 463 pages. DOI: 10.1007/978-3-031-30073-8

See Also

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

data(sleep)
irmi(sleep)

data(testdata)
imp_testdata1 <- irmi(testdata$wna, mixed = testdata$mixed)

# mixed.constant != 0 (-10)
testdata$wna$m1[testdata$wna$m1 == 0] <- -10
testdata$wna$m2 <- log(testdata$wna$m2 + 0.001)
imp_testdata2 <- irmi(
  testdata$wna,
  mixed = testdata$mixed,
  mixed.constant = c(-10,log(0.001))
)
imp_testdata2$m2 <- exp(imp_testdata2$m2) - 0.001

#example with fixed formulas for the variables with missing
form = list(
  NonD  = c("BodyWgt", "BrainWgt"),
  Dream = c("BodyWgt", "BrainWgt"),
  Sleep = c("BrainWgt"           ),
  Span  = c("BodyWgt"            ),
  Gest  = c("BodyWgt", "BrainWgt")
)
irmi(sleep, modelFormulas = form, trace = TRUE)

# Example with ordered variable
td <- testdata$wna
td$c1 <- as.ordered(td$c1)
irmi(td)

k-Nearest Neighbour Imputation

Description

k-Nearest Neighbour Imputation based on a variation of the Gower Distance for numerical, categorical, ordered and semi-continous variables.

Usage

kNN(
  data,
  variable = colnames(data),
  k = 5,
  dist_var = colnames(data),
  weights = NULL,
  numFun = median,
  catFun = maxCat,
  makeNA = NULL,
  NAcond = NULL,
  impNA = TRUE,
  donorcond = NULL,
  mixed = vector(),
  mixed.constant = NULL,
  trace = FALSE,
  imp_var = TRUE,
  imp_suffix = "imp",
  addRF = FALSE,
  onlyRF = FALSE,
  addRandom = FALSE,
  useImputedDist = TRUE,
  weightDist = FALSE,
  methodStand = "range",
  ordFun = medianSamp
)

Arguments

Details

Numerical and semi-continuous (mixed) distance variables are standardised by their range (or IQR, see methodStand) so that each variable contributes on a comparable scale; for mixed variables the point mass (mixed.constant) is excluded from the range estimate.

Missing values in the distance variables (dist_var) are handled with a sentinel: after standardisation an NA is replaced by the maximum of the pooled column plus one. As a consequence two records that are both missing the same distance variable have distance 0 on that variable (they are treated as identical there), while a record missing the variable and one observing it contribute up to about twice a normal variable's share. In effect kNN is drawn towards donors that share the recipient's pattern of missingness in the distance variables; under a MAR mechanism this can select unrepresentative donors. This departs from Gower's (1971) convention of omitting non-comparable variables and renormalising the remaining weights. To limit the effect, prefer dist_var variables with few or no missing values.

Value

the imputed data set.

Author(s)

Alexander Kowarik, Statistik Austria

References

A. Kowarik, M. Templ (2016) Imputation with R package VIM. Journal of Statistical Software, 74(7), 1-16.

See Also

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

data(sleep)
kNN(sleep)
library(laeken)
kNN(sleep, numFun = weightedMean, weightDist=TRUE)

Background map for the Kola project data

Description

Coordinates of the Kola background map.

Source

Kola Project (1993-1998)

References

Reimann, C., Filzmoser, P., Garrett, R.G. and Dutter, R. (2008) Statistical Data Analysis Explained: Applied Environmental Statistics with R. Wiley, 2008.

Examples

data(kola.background, package = "VIM")
bgmap(kola.background)

Synthetic Austrian Structural Business Survey data

Description

A synthetic, non-confidential enterprise-level data set inspired by the Austrian Leistungs- und Strukturstatistik from reporting years 2021 onward.

Format

A data frame with 10000 observations and 38 variables.

Details

The variables cover the main dimensions described in the Statistik Austria standard documentation: economic activity, NUTS2 region, data source, employment structure, turnover, intermediate consumption, gross value added, personnel costs, gross operating surplus, and tangible investment.

The monetary variables are expressed in thousand euros. The observations are generated synthetically and do not contain real enterprise records. Companion validation rules are available as lse_synthetic_rules.

Source

Statistik Austria, Standard-Dokumentation zur Leistungs- und Strukturstatistik ab 2021, https://www.statistik.at/fileadmin/shared/QM/Standarddokumentationen/U/std_u_ls_ab_2021.pdf

Examples

data(lse_synthetic)
str(lse_synthetic)

Validation rules for the synthetic LSE data

Description

Accounting, additional edit, and combined validation rules for lse_synthetic. The rules are stored as plain data frames of rule text so that the data set does not depend on the validate package; build the corresponding validate::validator objects with validate::validator(.data = lse_synthetic_rules$edit) before passing them to vimpute(method = "restricted").

Format

A named list with three data frames: accounting (36 rules), extra (56 rules), and edit (92 rules, the combination of both). Each has the columns

name

rule name (V01, V02, ...)

rule

the rule as R code, as accepted by validate::validator(.data = )

Examples

data(lse_synthetic_rules)
names(lse_synthetic_rules)
head(lse_synthetic_rules$edit)
if (requireNamespace("validate", quietly = TRUE)) {
  edit_rules <- validate::validator(.data = lse_synthetic_rules$edit)
  length(edit_rules)
}

Generate MCAR/MAR/MNAR missingness in complete data

Description

Amputation generator for simulation studies (the variable-wise counterpart of mice::ampute()): takes complete data, sets a controlled share of cells to NA under a chosen missingness mechanism, and returns the amputed data together with an attr(., "where") indicator matrix that plugs directly into evaluation(), nrmse() and pfc():

Usage

makeMissing(
  data,
  prop = 0.1,
  mechanism = c("MCAR", "MAR", "MNAR"),
  vars = NULL,
  weights = NULL,
  seed = NULL
)

Arguments

Details

  amp <- makeMissing(dat, prop = 0.2, mechanism = "MAR")
  imp <- vimpute(amp)
  evaluation(dat, imp, m = attr(amp, "where"))

Mechanisms (applied per target variable, each receiving exactly round(prop * nrow(data)) missing cells):

"MCAR"

cells are drawn uniformly at random.

"MAR"

the probability of a cell going missing grows with a weighted score of the other (observed) variables: rows are drawn with probabilities plogis(z) where z is the standardized weighted sum of the driver columns. Default drivers: all numeric columns except the target, with equal weights; use weights to select drivers and directions (negative weights make low driver values predictive of missingness).

"MNAR"

as "MAR", but the score is the target's own standardized values, so high values are more often missing (a weights entry named after the target flips or scales the direction). Numeric targets only.

With several target variables the scores are computed on the complete input, so each target's mechanism conditions on the pre-amputation values of its drivers.

Value

The amputed data, classed like the input (data.frame in, data.frame out; data.table in, data.table out), with attributes "where" (logical indicator matrix of the introduced missings), "mechanism" and "prop".

Author(s)

Matthias Templ

See Also

evaluation(), vimpute(), kNN()

Examples

data(sleep)
complete_rows <- na.omit(sleep[, c("BodyWgt", "BrainWgt", "Sleep", "Span")])
amp <- makeMissing(complete_rows, prop = 0.2, mechanism = "MAR",
                   vars = "Sleep", seed = 1)
colSums(is.na(amp))
imp <- kNN(amp, imp_var = FALSE)
evaluation(complete_rows, imp, m = attr(amp, "where"))

Map with information about missing/imputed values

Description

Map of observed and missing/imputed values.

Usage

mapMiss(
  x,
  coords,
  map,
  delimiter = NULL,
  selection = c("any", "all"),
  col = c("skyblue", "red", "orange"),
  alpha = NULL,
  pch = c(19, 15),
  col.map = grey(0.5),
  legend = TRUE,
  interactive = TRUE,
  ...
)

Arguments

Details

If interactive=TRUE, detailed information for an observation can be printed on the console by clicking on the corresponding point. Clicking in a region that does not contain any points quits the interactive session.

Author(s)

Matthias Templ, Andreas Alfons, modifications by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

bgmap(), bubbleMiss(), colormapMiss()

Examples

data(chorizonDL, package = "VIM")
data(kola.background, package = "VIM")
coo <- chorizonDL[, c("XCOO", "YCOO")]
## for missing values
x <- chorizonDL[, c("As", "Bi")]
mapMiss(x, coo, kola.background)

## for imputed values
x_imp <- kNN(chorizonDL[, c("As", "Bi")])
mapMiss(x_imp, coo, kola.background, delimiter = "_imp")

Marginplot Matrix

Description

Create a scatterplot matrix with information about missing/imputed values in the plot margins of each panel.

Usage

marginmatrix(
  x,
  delimiter = NULL,
  col = c("skyblue", "red", "red4", "orange", "orange4"),
  alpha = NULL,
  ...
)

Arguments

Details

marginmatrix uses pairsVIM() with a panel function based on marginplot().

The graphical parameter oma will be set unless supplied as an argument.

Author(s)

Andreas Alfons, modifications by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

marginplot(), pairsVIM(), scattmatrixMiss()

Other plotting functions: aggr(), barMiss(), histMiss(), marginplot(), matrixplot(), mosaicMiss(), pairsVIM(), parcoordMiss(), pbox(), scattJitt(), scattMiss(), scattmatrixMiss(), spineMiss()

Examples

data(sleep, package = "VIM")
## for missing values
x <- sleep[, 1:5]
x[,c(1,2,4)] <- log10(x[,c(1,2,4)])
marginmatrix(x)

## for imputed values
x_imp <- kNN(sleep[, 1:5])
x_imp[,c(1,2,4)] <- log10(x_imp[,c(1,2,4)])
marginmatrix(x_imp, delimiter = "_imp")

Scatterplot with additional information in the margins

Description

In addition to a standard scatterplot, information about missing/imputed values is shown in the plot margins. Furthermore, imputed values are highlighted in the scatterplot.

Usage

marginplot(
  x,
  delimiter = NULL,
  col = c("skyblue", "red", "red4", "orange", "orange4"),
  alpha = NULL,
  pch = c(1, 16),
  cex = par("cex"),
  numbers = TRUE,
  cex.numbers = par("cex"),
  zeros = FALSE,
  xlim = NULL,
  ylim = NULL,
  main = NULL,
  sub = NULL,
  xlab = NULL,
  ylab = NULL,
  ann = par("ann"),
  axes = TRUE,
  frame.plot = axes,
  ...
)

Arguments

Details

Boxplots for available and missing/imputed data, as well as univariate scatterplots for missing/imputed values in one variable are shown in the plot margins.

Imputed values in either of the variables are highlighted in the scatterplot.

Furthermore, the frequencies of the missing/imputed values can be displayed by a number (lower left of the plot). The number in the lower left corner is the number of observations that are missing/imputed in both variables.

Note

Some of the argument names and positions have changed with versions 1.3 and 1.4 due to extended functionality and for more consistency with other plot functions in VIM. For back compatibility, the argument cex.text can still be supplied to ...{} and is handled correctly. Nevertheless, it is deprecated and no longer documented. Use cex.numbers instead.

Author(s)

Andreas Alfons, Matthias Templ, modifications by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

scattMiss()

Other plotting functions: aggr(), barMiss(), histMiss(), marginmatrix(), matrixplot(), mosaicMiss(), pairsVIM(), parcoordMiss(), pbox(), scattJitt(), scattMiss(), scattmatrixMiss(), spineMiss()

Examples

data(tao, package = "VIM")
data(chorizonDL, package = "VIM")
## for missing values
marginplot(tao[,c("Air.Temp", "Humidity")])
marginplot(log10(chorizonDL[,c("CaO", "Bi")]))

## for imputed values
marginplot(kNN(tao[,c("Air.Temp", "Humidity")]), delimiter = "_imp")
marginplot(kNN(log10(chorizonDL[,c("CaO", "Bi")])), delimiter = "_imp")

Fast matching/imputation based on categorical variable

Description

Suitable donors are searched based on matching of the categorical variables. The variables are dropped in reversed order, so that the last element of 'match_var' is dropped first and the first element of the vector is dropped last.

Usage

matchImpute(
  data,
  variable = colnames(data)[!colnames(data) %in% match_var],
  match_var,
  imp_var = TRUE,
  imp_suffix = "imp"
)

Arguments

Details

The method works by sampling values from the suitable donors.

Value

the imputed data set.

Author(s)

Johannes Gussenbauer, Alexander Kowarik

See Also

hotdeck()

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), medianSamp(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

data(sleep,package="VIM")
imp_data <- matchImpute(sleep,variable=c("NonD","Dream","Sleep","Span","Gest"),
  match_var=c("Exp","Danger"))

data(testdata,package="VIM")
imp_testdata1 <- matchImpute(testdata$wna,match_var=c("c1","c2","b1","b2"))

dt <- data.table::data.table(testdata$wna)
imp_testdata2 <- matchImpute(dt,match_var=c("c1","c2","b1","b2"))

Matrix plot

Description

Create a matrix plot, in which all cells of a data matrix are visualized by rectangles. Available data is coded according to a continuous color scheme, while missing/imputed data is visualized by a clearly distinguishable color.

Usage

matrixplot(
  x,
  delimiter = NULL,
  sortby = NULL,
  col = c("red", "orange"),
  fixup = TRUE,
  xlim = NULL,
  ylim = NULL,
  main = NULL,
  sub = NULL,
  xlab = NULL,
  ylab = NULL,
  axes = TRUE,
  labels = axes,
  xpd = NULL,
  interactive = TRUE,
  ...
)

Arguments

Details

In a matrix plot, all cells of a data matrix are visualized by rectangles. Available data is coded according to a continuous color scheme. To compute the colors via interpolation, the variables are first scaled to the interval between 0 and 1. Missing/imputed values can then be visualized by a clearly distinguishable color. It is thereby possible to use colors in the HCL or RGB color space. A simple way of visualizing the magnitude of the available data is to apply a greyscale, which has the advantage that missing/imputed values can easily be distinguished by using a color such as red/orange. Note that -Inf and Inf are always assigned the begin and end color, respectively, of the continuous color scheme.

Additionally, the observations can be sorted by the magnitude of a selected variable. If interactive is TRUE, clicking in a column redraws the plot with observations sorted by the corresponding variable. Clicking anywhere outside the plot region quits the interactive session.

Note

This is a much more powerful extension to the function imagmiss in the former CRAN package dprep.

iimagMiss is deprecated and may be omitted in future versions of VIM. Use matrixplot instead.

Author(s)

Andreas Alfons, Matthias Templ, modifications by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

Other plotting functions: aggr(), barMiss(), histMiss(), marginmatrix(), marginplot(), mosaicMiss(), pairsVIM(), parcoordMiss(), pbox(), scattJitt(), scattMiss(), scattmatrixMiss(), spineMiss()

Examples

data(sleep, package = "VIM")
## for missing values
x <- sleep[, -(8:10)]
x[,c(1,2,4,6,7)] <- log10(x[,c(1,2,4,6,7)])
matrixplot(x, sortby = "BrainWgt")

## for imputed values
x_imp <- kNN(sleep[, -(8:10)])
x_imp[,c(1,2,4,6,7)] <- log10(x_imp[,c(1,2,4,6,7)])
matrixplot(x_imp, delimiter = "_imp", sortby = "BrainWgt")

Aggregation function for a factor variable

Description

The function maxCat chooses the level with the most occurrences and random if the maximum is not unique.

Usage

maxCat(x, weights = NULL)

Arguments

Aggregation function for a ordinal variable

Description

The function medianSamp chooses the level as the median or randomly between two levels.

Usage

medianSamp(x, weights = NULL)

Arguments

See Also

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), rangerImpute(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Midastouch: PMM with covariate-distance-weighted donor selection

Description

For each missing value, finds k nearest donors using a combined score: closeness in predicted value (score) AND closeness in covariate space (Mahalanobis distance). Donors closer in covariate space are upweighted.

Usage

midastouch_donors(
  y_obs,
  X_obs,
  X_miss,
  score_obs = NULL,
  score_miss = NULL,
  k = 5L
)

Arguments

Details

Based on Siddique & Belin (2008), "Multiple imputation using an iterative hot-deck with distance-based donor selection", Statistics in Medicine.

Value

Numeric vector of length n_miss with imputed values drawn from donors

Mosaic plot with information about missing/imputed values

Description

Create a mosaic plot with information about missing/imputed values.

Usage

mosaicMiss(
  x,
  delimiter = NULL,
  highlight = NULL,
  selection = c("any", "all"),
  plotvars = NULL,
  col = c("skyblue", "red", "orange"),
  labels = NULL,
  miss.labels = TRUE,
  ...
)

Arguments

Details

Mosaic plots are graphical representations of multi-way contingency tables. The frequencies of the different cells are visualized by area-proportional rectangles (tiles). Additional tiles are be used to display the frequencies of missing/imputed values. Furthermore, missing/imputed values in a certain variable or combination of variables can be highlighted in order to explore their structure.

Value

An object of class "structable" is returned invisibly.

Note

This function uses the highly flexible strucplot framework of package vcd.

Author(s)

Andreas Alfons, modifications by Bernd Prantner

References

Meyer, D., Zeileis, A. and Hornik, K. (2006) The strucplot framework: Visualizing multi-way contingency tables with vcd. Journal of Statistical Software, 17 (3), 1–48.

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

spineMiss(), vcd::mosaic()

Other plotting functions: aggr(), barMiss(), histMiss(), marginmatrix(), marginplot(), matrixplot(), pairsVIM(), parcoordMiss(), pbox(), scattJitt(), scattMiss(), scattmatrixMiss(), spineMiss()

Examples

data(sleep, package = "VIM")
## for missing values
mosaicMiss(sleep, highlight = 4, 
    plotvars = 8:10, miss.labels = FALSE)

## for imputed values
mosaicMiss(kNN(sleep), highlight = 4, 
    plotvars = 8:10, delimiter = "_imp", miss.labels = FALSE)

Constructor for vimmi objects

Description

Constructor for vimmi objects

Usage

new_vimmi(
  data,
  imp,
  where,
  m,
  nmis,
  method,
  boot,
  uncert,
  call,
  tuning_log = NULL,
  chain = NULL,
  seed = NULL,
  model_error = NULL
)

Arguments

Value

A vimmi object

Out-of-bag predictions of a fitted learner, when it exposes them

Description

ranger stores the out-of-bag predictions of the training rows on the fitted object, in training-row order and at no extra cost. They estimate the true predictive spread to within ~1% where the in-sample predictions understate it about twofold. Returns NULL whenever they are unavailable, misaligned, or too sparse to estimate a scale from, so callers fall back to their own path.

Usage

oob_predictions(learner, n)

Arguments

Value

Numeric vector of length n (possibly with non-finite entries for rows that were in-bag in every tree), or NULL

Overimputation: calibration diagnostic for an imputation model

Description

Answers "is my imputation model well calibrated?" without ground truth (the analogue of Amelia::overimpute(), model-agnostic): the observed cells of one variable are treated as missing – fold by fold, so every observed cell is overimputed exactly once – and imputed with draws multiple imputations via vimpute(). For each cell the observed value is compared with the mean and a level interval of its draws; a well-calibrated model covers roughly level of the observed values. Any genuinely missing cells stay missing and are imputed as part of the model, exactly as they would be in a real run.

Usage

overimpute(data, variable, ..., draws = 5, folds = 5, level = 0.9, seed = NULL)

## S3 method for class 'vimpute_overimpute'
print(x, ...)

## S3 method for class 'vimpute_overimpute'
plot(x, ...)

Arguments

Details

The ... arguments are passed on to vimpute(), so the diagnostic runs for any method, spec, or grammar configuration – e.g. overimpute(dat, "y", method = "robust") or overimpute(dat, "y", spec = list(y = vs_ranger(num.trees = 300))). Note the cost: folds x draws imputation runs; lower folds and draws (the defaults are modest) or use a fast method for large data. For a stochastic spread of the draws keep an uncertainty source switched on (the default uncert = "pmm", or boot/uncert); with purely deterministic settings all draws coincide and the intervals collapse.

Value

A data.frame of class vimpute_overimpute with one row per observed cell: row (row index in data), observed, mean, lower, upper (quantiles of the draws at level), and covered. Attributes: variable, level, draws, folds. print() reports the empirical coverage; plot() draws the observed-vs-imputed calibration plot with intervals and the 45-degree line.

Author(s)

Matthias Templ

See Also

vimpute(), makeMissing(), evaluation()

Examples

## Not run: 
data(sleep)
ov <- overimpute(sleep, "Sleep", method = "robust", sequential = FALSE,
                 draws = 5, folds = 5, seed = 1)
print(ov)   # empirical coverage of the 90% intervals
plot(ov)

## End(Not run)

Scatterplot Matrices

Description

Create a scatterplot matrix.

Usage

pairsVIM(
  x,
  ...,
  delimiter = NULL,
  main = NULL,
  sub = NULL,
  panel = points,
  lower = panel,
  upper = panel,
  diagonal = NULL,
  labels = TRUE,
  pos.labels = NULL,
  cex.labels = NULL,
  font.labels = par("font"),
  layout = c("matrix", "graph"),
  gap = 1
)

Arguments

Details

This function is the workhorse for marginmatrix() and scattmatrixMiss().

The graphical parameter oma will be set unless supplied as an argument.

A panel function should not attempt to start a new plot, since the coordinate system for each panel is set up by pairsVIM.

Note

The code is based on graphics::pairs(). Starting with version 1.4, infinite values are no longer removed before passing the x and y vectors to the panel functions.

Author(s)

Andreas Alfons, modifications by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

marginmatrix(), scattmatrixMiss()

Other plotting functions: aggr(), barMiss(), histMiss(), marginmatrix(), marginplot(), matrixplot(), mosaicMiss(), parcoordMiss(), pbox(), scattJitt(), scattMiss(), scattmatrixMiss(), spineMiss()

Examples

data(sleep, package = "VIM")
x <- sleep[, -(8:10)]
x[,c(1,2,4,6,7)] <- log10(x[,c(1,2,4,6,7)])
pairsVIM(x)

Parallel coordinate plot with information about missing/imputed values

Description

Parallel coordinate plot with adjustments for missing/imputed values. Missing values in the plotted variables may be represented by a point above the corresponding coordinate axis to prevent disconnected lines. In addition, observations with missing/imputed values in selected variables may be highlighted.

Usage

parcoordMiss(
  x,
  delimiter = NULL,
  highlight = NULL,
  selection = c("any", "all"),
  plotvars = NULL,
  plotNA = TRUE,
  col = c("skyblue", "red", "skyblue4", "red4", "orange", "orange4"),
  alpha = NULL,
  lty = par("lty"),
  xlim = NULL,
  ylim = NULL,
  main = NULL,
  sub = NULL,
  xlab = NULL,
  ylab = NULL,
  labels = TRUE,
  xpd = NULL,
  interactive = TRUE,
  ...
)

Arguments

Details

In parallel coordinate plots, the variables are represented by parallel axes. Each observation of the scaled data is shown as a line. Observations with missing/imputed values in selected variables may thereby be highlighted. However, plotting variables with missing values results in disconnected lines, making it impossible to trace the respective observations across the graph. As a remedy, missing values may be represented by a point above the corresponding coordinate axis, which is separated from the main plot by a small gap and a horizontal line, as determined by plotNA. Connected lines can then be drawn for all observations. Nevertheless, a caveat of this display is that it may draw attention away from the main relationships between the variables.

If interactive is TRUE, it is possible switch between this display and the standard display without the separate level for missing values by clicking in the top margin of the plot. In addition, the variables to be used for highlighting can be selected interactively. Observations with missing/imputed values in any or in all of the selected variables are highlighted (as determined by selection). A variable can be added to the selection by clicking on a coordinate axis. If a variable is already selected, clicking on its coordinate axis removes it from the selection. Clicking anywhere outside the plot region (except the top margin, if missing/imputed values exist) quits the interactive session.

Note

Some of the argument names and positions have changed with versions 1.3 and 1.4 due to extended functionality and for more consistency with other plot functions in VIM. For back compatibility, the arguments colcomb and xaxlabels can still be supplied to ...{} and are handled correctly. Nevertheless, they are deprecated and no longer documented. Use highlight and labels instead.

Author(s)

Andreas Alfons, Matthias Templ, modifications by Bernd Prantner

References

Wegman, E. J. (1990) Hyperdimensional data analysis using parallel coordinates. Journal of the American Statistical Association 85 (411), 664–675.

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

pbox()

Other plotting functions: aggr(), barMiss(), histMiss(), marginmatrix(), marginplot(), matrixplot(), mosaicMiss(), pairsVIM(), pbox(), scattJitt(), scattMiss(), scattmatrixMiss(), spineMiss()

Examples

data(chorizonDL, package = "VIM")
## for missing values
parcoordMiss(chorizonDL[,c(15,101:110)], 
    plotvars=2:11, interactive = FALSE)
legend("top", col = c("skyblue", "red"), lwd = c(1,1), 
    legend = c("observed in Bi", "missing in Bi"))

## for imputed values
parcoordMiss(kNN(chorizonDL[,c(15,101:110)]), delimiter = "_imp" ,
    plotvars=2:11, interactive = FALSE)
legend("top", col = c("skyblue", "orange"), lwd = c(1,1), 
    legend = c("observed in Bi", "imputed in Bi"))

Parallel boxplots with information about missing/imputed values

Description

Boxplot of one variable of interest plus information about missing/imputed values in other variables.

Usage

pbox(
  x,
  delimiter = NULL,
  pos = 1,
  selection = c("none", "any", "all"),
  col = c("skyblue", "red", "red4", "orange", "orange4"),
  numbers = TRUE,
  cex.numbers = par("cex"),
  xlim = NULL,
  ylim = NULL,
  main = NULL,
  sub = NULL,
  xlab = NULL,
  ylab = NULL,
  axes = TRUE,
  frame.plot = axes,
  labels = axes,
  interactive = TRUE,
  ...
)

Arguments

Details

This plot consists of several boxplots. First, a standard boxplot of the variable of interest is produced. Second, boxplots grouped by observed and missing/imputed values according to selection are produced for the variable of interest.

Additionally, the frequencies of the missing/imputed values can be represented by numbers. If so, the first line corresponds to the observed values of the variable of interest and their distribution in the different groups, the second line to the missing/imputed values.

If interactive=TRUE, clicking in the left margin of the plot results in switching to the previous variable and clicking in the right margin results in switching to the next variable. Clicking anywhere else on the graphics device quits the interactive session.

Value

a list as returned by graphics::boxplot().

Note

Some of the argument names and positions have changed with version 1.3 due to extended functionality and for more consistency with other plot functions in VIM. For back compatibility, the arguments names and cex.text can still be supplied to ...{} and are handled correctly. Nevertheless, they are deprecated and no longer documented. Use labels and cex.numbers instead.

Author(s)

Andreas Alfons, Matthias Templ, modifications by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

parcoordMiss()

Other plotting functions: aggr(), barMiss(), histMiss(), marginmatrix(), marginplot(), matrixplot(), mosaicMiss(), pairsVIM(), parcoordMiss(), scattJitt(), scattMiss(), scattmatrixMiss(), spineMiss()

Examples

data(chorizonDL, package = "VIM")
## for missing values
pbox(log(chorizonDL[, c(4,5,8,10,11,16:17,19,25,29,37,38,40)]))

## for imputed values
pbox(kNN(log(chorizonDL[, c(4,8,10,11,17,19,25,29,37,38,40)])),
     delimiter = "_imp")

Diagnostic plots for a vimmi object

Description

Draws mice-style multiple-imputation diagnostics for a vimmi object, for every numeric variable with imputed cells:

"chains"

convergence trace plots of the sequential (FCS) imputation: the mean and the standard deviation of the imputed values against the iteration number, one line per imputation. Chains that mix well fluctuate around a common level without trends; systematic drift suggests increasing nseq. Requires chain statistics (stored by vimpute() since VIM 7.3.0).

"density"

the density of the observed values (blue, bold) overlaid with the density of each imputation's imputed values (red, thin) – the analogue of mice::densityplot(). Imputed densities that deviate wildly from the observed one can flag model misfit (or genuine MAR shifts).

"strip"

every value as a point: column 0 holds the observed values (blue), columns 1..m the imputed values of each imputation (red) – the analogue of mice::stripplot().

Usage

## S3 method for class 'vimmi'
plot(x, y = c("chains", "density", "strip"), ...)

Arguments

Value

x, invisibly

See Also

vimmi, vimpute

Examples

## Not run: 
result <- vimpute(sleep, method = "ranger", m = 5, seed = 1)
plot(result)             # convergence chains
plot(result, "density")  # observed vs imputed densities
plot(result, "strip")    # observed vs imputed values

## End(Not run)

Score-based PMM donor selection

Description

For each missing value, finds k nearest observed values based on model score (predicted value) distance, then aggregates using the chosen method.

Usage

pmm_donor_selection(
  y_obs,
  score_obs,
  score_miss,
  k = 1L,
  agg_method = "random"
)

Arguments

Value

Numeric vector of length(score_miss) with imputed values

Predicted donor scores for true PMM

Description

Scores the observed rows of a target with the trained learner so that PMM donors are matched on their predicted values (Little 1988), not on their observed values. For a ranger fit without bootstrap resampling the honest out-of-bag predictions are used; otherwise the trained learner predicts the observed rows (aregImpute-style when the fit is a bootstrap refit). Returns NULL when the rows cannot be scored.

Usage

pmm_observed_scores(
  learner,
  data_temp,
  obs_idx,
  feature_cols,
  target_col,
  factor_levels,
  method_var,
  boot,
  lhs_transformation = NULL
)

Transformation and standardization

Description

This function is used by the VIM GUI for transformation and standardization of the data.

Usage

prepare(
  x,
  scaling = c("none", "classical", "MCD", "robust", "onestep"),
  transformation = c("none", "minus", "reciprocal", "logarithm", "exponential", "boxcox",
    "clr", "ilr", "alr"),
  alpha = NULL,
  powers = NULL,
  start = 0,
  alrVar
)

Arguments

Details

Transformation:

"none": no transformation is used.

"logarithm": compute the the logarithm (to the base 10).

"boxcox": apply a Box-Cox transformation. Powers may be specified or calculated with the function car::powerTransform().

Standardization:

"none": no standardization is used.

"classical": apply a z-Transformation on each variable by using function scale().

"robust": apply a robustified z-Transformation by using median and MAD.

Value

Transformed and standardized data.

Author(s)

Matthias Templ, modifications by Andreas Alfons

See Also

scale(), car::powerTransform()

Examples

data(sleep, package = "VIM")
x <- sleep[, c("BodyWgt", "BrainWgt")]
prepare(x, scaling = "robust", transformation = "logarithm")

Pulp lignin content

Description

Pulp quality by lignin content remaining

Format

A data frame with 301 observations on the following 23 variables.

Details

Pulp quality is measured by the lignin content remaining in the pulp: the Kappa number. This data set is used to understand which variables in the process influence the Kappa number, and if it can be predicted accurately enough for an inferential sensor application. Variables with a number at the end have been lagged by that number of hours to line up the data.

Source

https://openmv.net/info/kamyr-digester

References

K. Walkush and R.R. Gustafson. Application of feedforward neural networks and partial least squares regression for modelling Kappa number in a continuous Kamyr digester", Pulp and Paper Canada, 95, 1994, p T7-T13.

Examples

data(pulplignin)
str(pulplignin)
aggr(pulplignin)

Random Forest Imputation

Description

Impute missing values based on random-forest models via vimpute().

Usage

rangerImpute(
  formula,
  data,
  imp_var = TRUE,
  imp_suffix = "imp",
  ...,
  verbose = FALSE,
  median = FALSE
)

Arguments

Value

the imputed data set.

See Also

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), regressionImp(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Examples

data(sleep)
rangerImpute(Dream+NonD~BodyWgt+BrainWgt,data=sleep)

Register GAM-based mlr3 learners for vimpute

Description

Creates and registers four custom mlr3 learners: regr.gam_imp, classif.gam_imp, regr.robgam_imp, classif.robgam_imp. Called automatically by vimpute() when method includes "gam" or "robgam".

Usage

register_gam_learners()

Register an imputation method for vimpute()

Description

vimpute() resolves its method argument through a package-level method registry. The built-in methods ("ranger", "xgboost", "regularized", "robust", "gam", "robgam", "restricted") are pre-registered; this function adds (or, with overwrite = TRUE, replaces) a user-defined method backed by any pair of mlr3 learners – e.g. regr.rpart/classif.rpart from mlr3 itself, or learners from mlr3extralearners such as lightgbm – without modifying VIM. After registration the new name can be used anywhere the built-in method names work: as a global method, in a per-variable method list, and in method-keyed learner_params.

Usage

register_vimpute_method(
  name,
  learner,
  packages = character(),
  setup = NULL,
  defaults = NULL,
  search_space = NULL,
  supports_formula = FALSE,
  fallback = "robust",
  validate = NULL,
  overwrite = FALSE
)

Arguments

Details

Uncertainty handling for registered methods: PMM (uncert = "pmm", the default, and pmm = TRUE) and uncert = "midastouch" work with any method because they only use the method's predictions. uncert = "normalerror"/"resid" and boot = TRUE derive residuals from training predictions when the model object does not expose them.

Value

Invisibly, the registered method name.

See Also

vimpute_methods(), unregister_vimpute_method(), vimpute()

Other vimpute method registry: unregister_vimpute_method(), vimpute_methods()

Examples

# a CART method backed by mlr3's rpart learners -- one call, no VIM patching
register_vimpute_method("cart",
  learner  = list(regr = "regr.rpart", classif = "classif.rpart"),
  packages = "rpart")
"cart" %in% vimpute_methods()

data(sleep)
res <- vimpute(sleep[, c("Sleep", "Dream", "Span")], method = "cart",
               sequential = FALSE)

unregister_vimpute_method("cart")

Regression Imputation (via vimpute)

Description

Impute missing values based on a regression / classification model.

Usage

regressionImp(
  formula,
  data,
  family = "AUTO",
  robust = FALSE,
  imp_var = TRUE,
  imp_suffix = "imp",
  mod_cat = FALSE
)

Arguments

Details

By default (lm() for numeric responses and glm() for binary responses, with any number of predictors) the imputation is deterministic and matches a plain regression fit. When the design is rank-deficient (collinear predictors or p \ge n) or the response is a multi-level factor, it falls back to the regularized (glmnet) path via vimpute() with method = "regularized". If robust = TRUE, vimpute() with method = "robust" (lmrob() / glmrob()) is used.

Value

the imputed data set (same class as data)

See Also

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), sampleCat(), vimmi, vimpute(), xgboostImpute()

Rug representation of missing/imputed values

Description

Add a rug representation of missing/imputed values in only one of the variables to scatterplots.

Usage

rugNA(
  x,
  y,
  ticksize = NULL,
  side = 1,
  col = "red",
  alpha = NULL,
  miss = NULL,
  lwd = 0.5,
  ...
)

Arguments

Details

If side is 1 or 3, the rug representation consists of values available in x but missing/imputed in y. Else if side is 2 or 4, it consists of values available in y but missing/imputed in x.

Author(s)

Andreas Alfons, modifications by Bernd Prantner

Examples

data(tao, package = "VIM")
## for missing values
x <- tao[, "Air.Temp"]
y <- tao[, "Humidity"]
plot(x, y)
rugNA(x, y, side = 1)
rugNA(x, y, side = 2)

## for imputed values
x_imp <- kNN(tao[, c("Air.Temp","Humidity")])
x <- x_imp[, "Air.Temp"]
y <- x_imp[, "Humidity"]
miss <- x_imp[, c("Air.Temp_imp","Humidity_imp")]
plot(x, y)
rugNA(x, y, side = 1, col = "orange", miss = miss)
rugNA(x, y, side = 2, col = "orange", miss = miss)

Random aggregation function for a factor variable

Description

The function sampleCat samples with probabilites corresponding to the occurrence of the level in the NNs.

Usage

sampleCat(x, weights = NULL)

Arguments

See Also

Other imputation methods: hotdeck(), impPCA(), imputeCellEM(), imputeCellIRMI(), imputeCellM(), imputeCellMCD(), imputeCellwise(), imputeRobust(), imputeRobustChain(), irmi(), kNN(), matchImpute(), medianSamp(), rangerImpute(), regressionImp(), vimmi, vimpute(), xgboostImpute()

Bivariate jitter plot

Description

Create a bivariate jitter plot.

Usage

scattJitt(
  x,
  delimiter = NULL,
  col = c("skyblue", "red", "red4", "orange", "orange4"),
  alpha = NULL,
  cex = par("cex"),
  col.line = "lightgrey",
  lty = "dashed",
  lwd = par("lwd"),
  numbers = TRUE,
  cex.numbers = par("cex"),
  main = NULL,
  sub = NULL,
  xlab = NULL,
  ylab = NULL,
  axes = TRUE,
  frame.plot = axes,
  labels = c("observed", "missing", "imputed"),
  ...
)

Arguments

Details

The amount of observed and missing/imputed values is visualized by jittered points. Thereby the plot region is divided into up to four regions according to the existence of missing/imputed values in one or both variables. In addition, the amount of observed and missing/imputed values can be represented by a number.

Note

Some of the argument names and positions have changed with version 1.3 due to extended functionality and for more consistency with other plot functions in VIM. For back compatibility, the argument cex.text can still be supplied to ...{} and is handled correctly. Nevertheless, it is deprecated and no longer documented. Use cex.numbers instead.

Author(s)

Matthias Templ, modifications by Andreas Alfons and Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

Other plotting functions: aggr(), barMiss(), histMiss(), marginmatrix(), marginplot(), matrixplot(), mosaicMiss(), pairsVIM(), parcoordMiss(), pbox(), scattMiss(), scattmatrixMiss(), spineMiss()

Examples

data(tao, package = "VIM")
## for missing values
scattJitt(tao[, c("Air.Temp", "Humidity")])

## for imputed values
scattJitt(kNN(tao[, c("Air.Temp", "Humidity")]), delimiter = "_imp")

Scatterplot with information about missing/imputed values

Description

In addition to a standard scatterplot, lines are plotted for the missing values in one variable. If there are imputed values, they will be highlighted.

Usage

scattMiss(
  x,
  delimiter = NULL,
  side = 1,
  col = c("skyblue", "red", "orange", "lightgrey"),
  alpha = NULL,
  lty = c("dashed", "dotted"),
  lwd = par("lwd"),
  quantiles = c(0.5, 0.975),
  inEllipse = FALSE,
  zeros = FALSE,
  xlim = NULL,
  ylim = NULL,
  main = NULL,
  sub = NULL,
  xlab = NULL,
  ylab = NULL,
  interactive = TRUE,
  ...
)

Arguments

Details

Information about missing values in one variable is included as vertical or horizontal lines, as determined by the side argument. The lines are thereby drawn at the observed x- or y-value. In case of imputed values, they will additionally be highlighted in the scatterplot. Supplementary, percentage coverage ellipses can be drawn to give a clue about the shape of the bivariate data distribution.

If interactiveis TRUE, clicking in the bottom margin redraws the plot with information about missing/imputed values in the first variable and clicking in the left margin redraws the plot with information about missing/imputed values in the second variable. Clicking anywhere else in the plot quits the interactive session.

Note

The argument zeros has been introduced in version 1.4. As a result, some of the argument positions have changed.

Author(s)

Andreas Alfons, modifications by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

marginplot()

Other plotting functions: aggr(), barMiss(), histMiss(), marginmatrix(), marginplot(), matrixplot(), mosaicMiss(), pairsVIM(), parcoordMiss(), pbox(), scattJitt(), scattmatrixMiss(), spineMiss()

Examples

data(tao, package = "VIM")
## for missing values
scattMiss(tao[,c("Air.Temp", "Humidity")])

## for imputed values
scattMiss(kNN(tao[,c("Air.Temp", "Humidity")]), delimiter = "_imp")

Scatterplot matrix with information about missing/imputed values

Description

Scatterplot matrix in which observations with missing/imputed values in certain variables are highlighted.

Usage

scattmatrixMiss(
  x,
  delimiter = NULL,
  highlight = NULL,
  selection = c("any", "all"),
  plotvars = NULL,
  col = c("skyblue", "red", "orange"),
  alpha = NULL,
  pch = c(1, 3),
  lty = par("lty"),
  diagonal = c("density", "none"),
  interactive = TRUE,
  ...
)

Arguments

Details

scattmatrixMiss uses pairsVIM() with a panel function that allows highlighting of missing/imputed values.

If interactive=TRUE, the variables to be used for highlighting can be selected interactively. Observations with missing/imputed values in any or in all of the selected variables are highlighted (as determined by selection). A variable can be added to the selection by clicking in a diagonal panel. If a variable is already selected, clicking on the corresponding diagonal panel removes it from the selection. Clicking anywhere else quits the interactive session.

The graphical parameter oma will be set unless supplied as an argument.

TKRscattmatrixMiss behaves like scattmatrixMiss, but uses tkrplot to embed the plot in a Tcl/Tk window. This is useful if the number of variables is large, because scrollbars allow to move from one part of the plot to another.

Note

Some of the argument names and positions have changed with version 1.3 due to a re-implementation and for more consistency with other plot functions in VIM. For back compatibility, the argument colcomb can still be supplied to ...{} and is handled correctly. Nevertheless, it is deprecated and no longer documented. Use highlight instead. The arguments smooth, reg.line and legend.plot are no longer used and ignored if supplied.

Author(s)

Andreas Alfons, Matthias Templ, modifications by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

pairsVIM(), marginmatrix()

Other plotting functions: aggr(), barMiss(), histMiss(), marginmatrix(), marginplot(), matrixplot(), mosaicMiss(), pairsVIM(), parcoordMiss(), pbox(), scattJitt(), scattMiss(), spineMiss()

Examples

data(sleep, package = "VIM")
## for missing values
x <- sleep[, 1:5]
x[,c(1,2,4)] <- log10(x[,c(1,2,4)])
scattmatrixMiss(x, highlight = "Dream")

## for imputed values
x_imp <- kNN(sleep[, 1:5])
x_imp[,c(1,2,4)] <- log10(x_imp[,c(1,2,4)])
scattmatrixMiss(x_imp, delimiter = "_imp", highlight = "Dream")

Mammal sleep data

Description

Sleep data with missing values.

Format

A data frame with 62 observations on the following 10 variables.

BodyWgt

a numeric vector

BrainWgt

a numeric vector

NonD

a numeric vector

Dream

a numeric vector

Sleep

a numeric vector

Span

a numeric vector

Gest

a numeric vector

Pred

a numeric vector

Exp

a numeric vector

Danger

a numeric vector

Source

Allison, T. and Chichetti, D. (1976) Sleep in mammals: ecological and constitutional correlates. Science 194 (4266), 732–734.

The data set was imported from GGobi.

Examples

data(sleep, package = "VIM")
summary(sleep)
aggr(sleep)

Spineplot with information about missing/imputed values

Description

Spineplot or spinogram with highlighting of missing/imputed values in other variables by splitting each cell into two parts. Additionally, information about missing/imputed values in the variable of interest is shown on the right hand side.

Usage

spineMiss(
  x,
  delimiter = NULL,
  pos = 1,
  selection = c("any", "all"),
  breaks = "Sturges",
  right = TRUE,
  col = c("skyblue", "red", "skyblue4", "red4", "orange", "orange4"),
  border = NULL,
  main = NULL,
  sub = NULL,
  xlab = NULL,
  ylab = NULL,
  axes = TRUE,
  labels = axes,
  only.miss = TRUE,
  miss.labels = axes,
  interactive = TRUE,
  ...
)

Arguments

Details

A spineplot is created if the variable of interest is categorial and a spinogram if it is numerical. The horizontal axis is scaled according to relative frequencies of the categories/classes. If more than one variable is supplied, the cells are split according to missingness/number of imputed values in the additional variables. Thus the proportion of highlighted observations in each category/class is displayed on the vertical axis. Since the height of each cell corresponds to the proportion of highlighted observations, it is now possible to compare the proportions of missing/imputed values among the different categories/classes.

If only.miss=TRUE, the missing/imputed values in the variable of interest are also visualized by a cell in the spine plot or spinogram. If additional variables are supplied, this cell is again split into two parts according to missingness/number if imputed values in the additional variables.

Otherwise, a small spineplot that visualizes missing/imputed values in the variable of interest is drawn on the right hand side. The first cell corresponds to observed values and the second cell to missing/imputed values. Each of the two cells is again split into two parts according to missingness/number of imputed values in the additional variables. Note that this display does not make sense if only one variable is supplied, therefore only.miss is ignored in that case.

If interactive=TRUE, clicking in the left margin of the plot results in switching to the previous variable and clicking in the right margin results in switching to the next variable. Clicking anywhere else on the graphics device quits the interactive session.

Value

a table containing the frequencies corresponding to the cells.

Note

Some of the argument names and positions have changed with version 1.3 due to extended functionality and for more consistency with other plot functions in VIM. For back compatibility, the arguments xaxlabels and missaxlabels can still be supplied to ...{} and are handled correctly. Nevertheless, they are deprecated and no longer documented. Use labels and miss.labels instead.

The code is based on the function graphics::spineplot() by Achim Zeileis.

Author(s)

Andreas Alfons, Matthias Templ, modifications by Bernd Prantner

References

M. Templ, A. Alfons, P. Filzmoser (2012) Exploring incomplete data using visualization tools. Advances in Data Analysis and Classification, Online first. DOI: 10.1007/s11634-011-0102-y.

See Also

histMiss(), barMiss(), mosaicMiss()

Other plotting functions: aggr(), barMiss(), histMiss(), marginmatrix(), marginplot(), matrixplot(), mosaicMiss(), pairsVIM(), parcoordMiss(), pbox(), scattJitt(), scattMiss(), scattmatrixMiss()

Examples

data(tao, package = "VIM")
data(sleep, package = "VIM")
## for missing values
spineMiss(tao[, c("Air.Temp", "Humidity")])
spineMiss(sleep[, c("Exp", "Sleep")])

## for imputed values
spineMiss(kNN(tao[, c("Air.Temp", "Humidity")]), delimiter = "_imp")
spineMiss(kNN(sleep[, c("Exp", "Sleep")]), delimiter = "_imp")

create table with highlighted missings/imputations

Description

Create a reactable table that highlights missing values and imputed values with the same colors as histMiss()

Usage

tableMiss(x, delimiter = "_imp")

Arguments

Examples

data(tao)
x_IMPUTED <- kNN(tao[, c("Air.Temp", "Humidity")])
tableMiss(x_IMPUTED[105:114, ])
x_IMPUTED[106, 2] <- NA
x_IMPUTED[105, 1] <- NA
x_IMPUTED[107, "Humidity_imp"] <- TRUE
tableMiss(x_IMPUTED[105:114, ])

Tropical Atmosphere Ocean (TAO) project data

Description

A small subsample of the Tropical Atmosphere Ocean (TAO) project data, derived from the GGOBI project.

Format

A data frame with 736 observations on the following 8 variables.

Year

a numeric vector

Latitude

a numeric vector

Longitude

a numeric vector

Sea.Surface.Temp

a numeric vector

Air.Temp

a numeric vector

Humidity

a numeric vector

UWind

zonal wind, i.e. latitude-parallel wind

VWind

meridional wind, i.e. longitude-parallel wind

Details

All cases recorded for five locations and two time periods.

Source

http://www.pmel.noaa.gov/tao/

Examples

data(tao, package = "VIM")
summary(tao)
aggr(tao)

Simulated data set for testing purpose

Description

2 numeric, 2 binary, 2 nominal and 2 mixed (semi-continous) variables

Format

The format is: List of 4

• ⁠$wna⁠ : a data.frame with 500 obs. of 8 variables:

  • x1: numeric 10.87 9.53 7.83 8.53 8.67 ...

  • x2: numeric 10.9 9.32 7.68 8.2 8.41 ... ..

  • c1: Factor w/ 4 levels "a","b","c","d": 3 2 2 1 2 2 1 3 3 2 ...

  • c2: Factor w/ 4 levels "a","b","c","d": 2 3 2 2 2 2 2 4 2 2 ...

  • b1: Factor w/ 2 levels "0","1": 2 2 1 2 1 2 1 2 1 1 ...

  • b2: Factor w/ 2 levels "0","1": 2 2 1 1 1 1 1 2 2 2 ...

  • m1: numeric 0 8.29 9.08 0 0 ...

  • m2: numeric 10.66 9.39 7.8 8.11 7.33 ...

• ⁠$wona⁠ : a 'data.frame“ with 500 obs. of 8 variables:

  • x1: numeric 10.87 9.53 7.83 8.53 8.67 ...

  • x2: numeric 10.9 9.32 7.68 8.2 8.41 ...

  • c1: Factor w/ 4 levels "a","b","c","d": 3 2 2 1 2 2 1 3 3 2 ...

  • c2: Factor w/ 4 levels "a","b","c","d": 2 3 2 2 2 2 2 4 2 2 ...

  • b1: Factor w/ 2 levels "0","1": 2 2 1 2 1 2 1 2 1 1 ...

  • b2: Factor w/ 2 levels "0","1": 2 2 1 1 1 1 1 2 2 2 ...

  • m1: numeric 0 8.29 9.08 0 0 ...

  • m2: numeric 10.66 9.39 7.8 8.11 7.33 ...

• ⁠$mixed⁠: c("m1", "m2")

• ⁠$outlierInd⁠: 'NULL“