---
title: "Multiple Imputation and Bootstrapping - Method boot_MI"
author: "Martijn W Heymans"
date: "`r Sys.Date()`"
output:
  rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Multiple Imputation and Bootstrapping - Method boot_MI}
  %\VignetteEngine{knitr::rmarkdown}
  \usepackage[utf8]{inputenc}
---

# Introduction

This page contains information of the `boot_MI` method that is implemented 
in the `psfmi_validate` function of the [`psfmi`](https://github.com/mwheymans/psfmi) package. 

# Method boot_MI

The method follows the internal validation procedure of the `validate` function in the `rms` package for
complete data but now within the context of multiply imputed data. With the method boot_MI, 
first bootstrap samples are drawn from the original incomplete dataset and than multiple imputation 
is applied in each of these incomplete bootstrap samples. The pooled model is analyzed in each bootstrap 
sample (training data) and subsequently tested in the original multiply imputed data to determine the amount
of optimism. The method can be performed in combination with backward or forward selection.

How these steps work is visualized in the Figure below. 

```{r , echo = FALSE, fig.cap="Schematic overview of the boot_MI method", out.width='100%', fig.align='center'}
knitr::include_graphics("boot_MI.png")

```

# Examples

* [Method boot_MI]
* [Method boot_MI including BW selection]

## Method boot_MI 

internal validation is done of the last model that is selected by the function `psfmi_lr`. 
In the example below, `psfmi_lr` is used with `p.crit` set at 1. This setting is also
used in the `psfmi_validate` function. This means that first the full model is pooled and
subsequently interval validation is also done of this full model.  
 
```{r}

library(psfmi)
pool_lr <- psfmi_lr(data=lbpmilr, formula = Chronic ~ Pain + JobDemands + rcs(Tampascale, 3) +
                   factor(Satisfaction) + Smoking, p.crit = 1, direction="FW",
                 nimp=5, impvar="Impnr", method="D1")

set.seed(100)
res_MI_boot <- psfmi_validate(pool_lr, val_method = "boot_MI", data_orig = lbp_orig, nboot = 5,
                     p.crit=1, nimp_mice = 3, direction = "BW", miceImp = miceImp,
                     printFlag = FALSE)

res_MI_boot
  
```

Back to [Examples]

## Method boot_MI including BW selection

Internal validation is done of the last model that is selected by the function `psfmi_lr`. 
In the example below, `psfmi_lr` is used with `p.crit` set at 1, and pooling is than done 
of the full model. Then interval validation is done with the `psfmi_validate` function 
including BW selection by setting p.crit=0.05. BW selection is than applied in 
each bootstrap sample from the full model of pool_lr. In this way, shrinkage of models can 
be performed including backward selection of variables. In this way a fair shrinkage factor
can be determined because variable selection is responsible for a large amount of overfitting in
coefficients.
 
```{r}

library(psfmi)
pool_lr <- psfmi_lr(data=lbpmilr, Outcome="Chronic", predictors = c("Pain", "JobDemands", "Smoking"), 
                   cat.predictors = "Satisfaction", spline.predictors = "Tampascale", nknots=3,
                   p.crit = 1, direction="FW", nimp=5, impvar="Impnr", method="D1")

set.seed(100)
res_MI_boot <- psfmi_validate(pool_lr, val_method = "boot_MI", data_orig = lbp_orig, nboot = 5,
                     p.crit=0.05, nimp_mice = 3, direction = "BW", miceImp = miceImp,
                     printFlag = FALSE)

res_MI_boot

```

Back to [Examples]

