---
title: "Practical Exercises Variable Selection"
output: learnr::tutorial
runtime: shiny_prerendered
---

```{r setup, include=FALSE}
library(learnr)
knitr::opts_chunk$set(echo = FALSE)
library(psfmi)
```

## Exercise 1a

Use the lbpmilr dataset that is included in the psfmi package and the psfmi_lr function to pool a prediction model without using backward selection. Use as outcome the “Chronic” variable and as predictors: "Gender", "Smoking", "Function", "JobControl", "JobDemands", "SocialSupport", as categorical predictors "Carrying" and "Satisfaction and use method “D1”.  Showed the pooled model and the pooled p-values as a result of the D1 method. 

```{r exercise1a, exercise=TRUE, exercise.eval=TRUE}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
                    predictors=c("Gender"), p.crit = 1, cat.predictors = c("Carrying"), method="D1")
pool_lr$RR_model

```


```{r exercise1a-hint}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
                    predictors=c("Gender", "Smoking", "Function", "JobControl", "JobDemands",
                     "SocialSupport"), p.crit = 1, cat.predictors = c("Carrying",  "Satisfaction"), 
                    method="D1", direction = NULL)
pool_lr$RR_model
pool_lr$multiparm

```

## Exercise 1b

Repeat question 1a but now do also backward selection from the pooled model. Use as p-value selection criterium p < 0.05. Show the pooled models at each selection step.

```{r exercise1b, exercise=TRUE, exercise.eval=TRUE}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
            predictors=c("Gender", "Smoking", "Function", "JobControl", "JobDemands",
           "SocialSupport"), p.crit = 1, cat.predictors = c("Carrying",  "Satisfaction"), method="D1", direction="BW")


```

```{r exercise1b-hint}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
            predictors=c("Gender", "Smoking", "Function", "JobControl", "JobDemands",
           "SocialSupport"), p.crit = 0.05, cat.predictors = c("Carrying",  "Satisfaction"), method="D1", direction=NULL)

pool_lr$RR_model
pool_lr$multiparm

```

## Exercise 1c

Repeat question 1b, but now use forward selection. Is the same model chosen as under 1b?

```{r exercise1c, exercise=TRUE, exercise.eval=TRUE}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
            predictors=c("Gender", "Smoking", "Function", "JobControl", "JobDemands",
           "SocialSupport"), p.crit = 1, cat.predictors = c("Carrying", "Satisfaction"), method="D1", 
           direction=NULL)

```


```{r exercise1c-hint}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
                    predictors=c("Gender", "Smoking", "Function", "JobControl", "JobDemands",
                                 "SocialSupport"), p.crit = 0.05, cat.predictors = c("Carrying",  "Satisfaction"),
                    method="D1", direction="FW")
pool_lr$RR_model
pool_lr$multiparm

```

## Exercise 1d

Repeat question b but now do backward selection from the pooled model by using the Median P Rule as selection method. Use as p-value selection criterium p < 0.05. Is the result different from question 1b.

```{r exercise1d, exercise=TRUE, exercise.eval=TRUE}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
            predictors=c("Gender", "Smoking", "Function", "JobControl", "JobDemands",
           "SocialSupport"), p.crit = 1, cat.predictors = c("Carrying", "Satisfaction"), method="D1", 
           direction=NULL)

```


```{r exercise1d-hint}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
          predictors=c("Gender", "Smoking", "Function", "JobControl", "JobDemands",
         "SocialSupport"), p.crit = 0.05, cat.predictors = c("Carrying",  "Satisfaction"),
         method="MPR", direction = "BW")
pool_lr$RR_model
pool_lr$multiparm

```

## Exercise 1e

It is also possible to force variables in the model by using the parameter setting keep.predictors of the psfmi_lr function. This may be of interest when you now from other research that specific predictors are important. Re-run the analyses of question 1b and force the variables Gender and Jobcontrol in the model. Use a p-value of 0.05. Use method = D1. Which model is finally selected?

```{r exercise1e, exercise=TRUE, exercise.eval=TRUE}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
           predictors=c("Gender", "Smoking", "Function", "JobControl", 	"JobDemands", "SocialSupport"),
           p.crit = 0.05, cat.predictors = 	c("Carrying",  "Satisfaction"), method="D1",
           keep.predictors = 	NULL, direction = "BW")


```


```{r exercise1e-hint}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
           predictors=c("Gender", "Smoking", "Function", "JobControl", 	"JobDemands", "SocialSupport"),
           p.crit = 0.05, cat.predictors = 	c("Carrying",  "Satisfaction"), method="D1",
           keep.predictors = 	c("Gender", "JobControl"), direction = "BW")

pool_lr$RR_model
pool_lr$multiparm

```

## Exercise 1f

It is also possible to apply variable selection when there are also interaction terms included in the model by using the parameter setting int.predictors of the psfmi_lr function. Re-run the analyses of question 1b and include the interaction between Smoking and Gender in the model. Do not force variables in the model, in other words, do not define variables as keep.predictors and use a p-value of 0.05. Use method = D1. Explore the selection steps. When is the interaction term excluded from the model and which model is finally selected?

```{r exercise1f, exercise=TRUE, exercise.eval=TRUE}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
                    predictors=c("Gender", "Smoking", "Function", "JobControl", 	"JobDemands", "SocialSupport"),
                    p.crit = 0.05, cat.predictors = 	c("Carrying",  "Satisfaction"), method="D1",
                    int.predictors = 	NULL, direction = "BW")


```


```{r exercise1f-hint}

pool_lr <- psfmi_lr(data=lbpmilr, nimp=5, impvar="Impnr", Outcome="Chronic",
                    predictors=c("Gender", "Smoking", "Function", "JobControl", 	"JobDemands", "SocialSupport"),
                    p.crit = 0.05, cat.predictors = 	c("Carrying",  "Satisfaction"), method="D1",
                    int.predictors = 	c("Smoking:Gender"), direction = "BW")
pool_lr$RR_model
pool_lr$multiparm

```
