---
title: "Overview"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Overview}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r, include = FALSE}
knitr::opts_chunk$set(
  collapse = TRUE,
  comment = "#>",
  fig.align = "center",
  fig.width = 6
)
```

The `adaptr` package simulates adaptive (multi-arm, multi-stage) clinical trials
using adaptive stopping, adaptive arm dropping and/or response-adaptive
randomisation.

The package has been developed as part of the
[INCEPT (Intensive Care Platform Trial) project](https://incept.dk/), funded
primarily by a grant from
[Sygeforsikringen "danmark"](https://www.sygeforsikring.dk/).

Additional guidance on the key methodological considerations when planning and
comparing adaptive clinical trials can be found in the open access article
*"[An overview of methodological considerations regarding adaptive stopping, arm dropping and randomisation in clinical trials](https://doi.org/10.1016/j.jclinepi.2022.11.002)"*
available in Journal of Clinical Epidemiology.

## Basic example
First, load the package:

```{r}
library(adaptr)
```

Parallelisation is supported in many `adaptr` functions, and a cluster of
parallel workers may be setup for the entire session using `setup_cluster()`,
which is ideally called early in a script. Alternatively, parallelisation
can be controlled by the global `"mc.cores"` option (set by calling
`options(mc.cores = <number>)`) or the `cores` argument of many functions.

### Set up trial
Then, setup a trial with the desired specifications. `adaptr` offers the
general purpose function `setup_trial()`, but here we use the built-in
`setup_trial_binom()` for a trial with a binary, binomially distributed,
undesirable outcome such as mortality (`adaptr` also includes
`setup_trial_norm()` for continuous, normally distributed outcomes).

The example trial specification has the following characteristics:

- The allocation probability to each arm cannot be lower than 15% (`min_probs`).
- Default thresholds for `inferiority` (< 1% probability of being the best arm)
and `superiority` (> 99% probability of being the best arm) are used and hence
not specified.
- Equivalence stopping rule: if the simulation yields a 90% probability
(`equivalence_prob`) of treatment differences being < 5 %-points
(`equivalence_diff`), the trial is stopped.
- We soften allocation ratios (`soften_power`) by a constant factor.

```{r}
binom_trial <- setup_trial_binom(
  arms = c("Arm A", "Arm B", "Arm C"),
  true_ys = c(0.25, 0.20, 0.30),
  min_probs = rep(0.15, 3), 
  data_looks = seq(from = 300, to = 2000, by = 100),
  equivalence_prob = 0.9,
  equivalence_diff = 0.05,
  soften_power = 0.5 
) 
```

See `?setup_trial()` for more details on the arguments or
`vignette("Basic-examples", "adaptr")` for **basic** example trial
specifications and a thorough review of the general trial specification
settings, and `vignette("Advanced-example", "adaptr")` for an **advanced**
example including details on how to specify user-written functions for
generating outcomes and posterior draws.

We can print an overview of the trial specification by simply running:

```{r}
binom_trial
```

By default, (most) probabilities are shown with 3 decimals. This can be changed
by explicitly `print()`ing the specification with the `prob_digits` arguments,
for example:

```{r}
print(binom_trial, prob_digits = 2)
```

Finally, a trial specification may be calibrated to obtain a specific value for
a certain performance metric (e.g., the Bayesian type 1 error rate for trial
specifications with no between-arm differences; not done in this overview) by
using the `calibrate_trial()` function. 

### Simulate a single trial
Remember to define the `seed` to ensure reproducible results:

```{r}
trial_res <- run_trial(binom_trial, seed = 12345)

trial_res
```

Again, we can choose the number of decimals with `print()`:

```{r}
print(trial_res, prob_digits = 2)
```

### Simulate multiple trials
Generally, we want to run many simulations using the same trial specification to
assess and compare performance metrics of different trial designs. This is the
job of `run_trials()` (note the final ***s***); again, we specify a `base_seed`
for reproducible results. Here we run 25 simulations, but in practice you will
generally want to run more simulations. The low number of simulations in this
example has been chosen to make run-time tolerable when producing the example,
but leads to uncertainty and instability in the results, as seen below.

```{r}
trial_res_mult <- run_trials(binom_trial, n_rep = 25, base_seed = 67890)
```

`run_trials()` can run simulations on several CPU cores concurrently by using
the `setup_cluster()` function (recommended), the `"mc.cores"` global option, or
the `cores` argument of `run_trials()`, as mentioned above (further details in
the `setup_cluster()` and `run_trials()` function definitions). Most functions
used to post-process, extract, and plot results can similarly be run in
parallel.

###  Calculate performance metrics and summmarise results
The results of multiple simulations may be summarised by printing the resulting
object:

```{r}
trial_res_mult
```

This calls the `summary()` method (as known from, e.g., regression models in
`R`), which summarises the results, and prints the output from that function in
a human-friendly manner using the `print()` method for summarised simulations.

The `summary()` function can also be called directly, which allows more control
of how results are summarised (including which arms are selected in inconclusive
trials), and allows subsequent extraction of individual key results. In
addition, the number of digits can be controlled when printed:

```{r}
res_sum <- summary(trial_res_mult)

print(res_sum, digits = 1)
```

The `summary()` method, however, may not necessarily be what you want. `adaptr`
has additional functions that may be used on multiple simulation results of the
same trial specification.

The `extract_results()` function extract key trial results and yields a tidy
`data.frame` with one simulation per row:

```{r}
extr_res <- extract_results(trial_res_mult)

nrow(extr_res)

head(extr_res)
```

The `check_performance()` function calculates key performance metrics and
returns them in a tidy `data.frame` with one metric per row, and may also be
used to assess the uncertainty in these estimates using bootstrapping:

```{r}
perf_res <- check_performance(trial_res_mult, uncertainty = TRUE, n_boot = 1000,
                              boot_seed = "base")

print(perf_res, digits = 3)
```

The `plot_metrics_ecdf()` function plots empirical cumulative distribution
functions of numerical performance metrics:
```{r}
plot_metrics_ecdf(trial_res_mult)
```

Note that all `adaptr` plotting functions require the `ggplot2` package.

The stability of performance metrics according to the number of simulations may
be assessed visually using the `plot_convergence()` function.

```{r}
# Convergence plots for four performance metrics
plot_convergence(trial_res_mult, metrics = c("size mean", "prob superior",
                                             "rmse", "idp"))
```

It is seen that the low number of simulations used in this example leads to
substantial uncertainty and instability in performance metrics (while the ideal
design percentage is always at 100% here, it would likely drop somewhat if more
simulations were conducted).

Finally, all combinations of remaining arms after trial completion may be
summarised using `check_remaining_arms()`:

```{r}
check_remaining_arms(trial_res_mult)
```


### Visualise trial results
In addition to the convergence plots, results may be visualised using the
`plot_status()` and `plot_history()` functions.
We need non-sparse results for `plot_history()` (but *not* for `plot_status()`
or `plot_convergence()` presented above), so we re-run `run_trials()` with the
`sparse` argument set to `FALSE`:

```{r}
trial_res_mult <- run_trials(binom_trial, n_rep = 25, base_seed = 67890,
                             sparse = FALSE)
```

First, we plot the overall trial statuses according to the total number of
patients randomised (this function does *not* require `sparse = FALSE`):

```{r}
plot_status(trial_res_mult, x_value = "total n")
```

We can also plot the statuses for specific arms, or for all arms if supplying
`NA` to the `arm`-argument:

```{r}
plot_status(trial_res_mult, x_value = "total n", arm = NA, ncol = 1)
```

Next, we plot the history of allocation probabilities at each adaptive
analysis look. Intervals cover the inter-quartile range by default
(`interval_width = 0.5`):

```{r}
plot_history(trial_res_mult)
```

Plotting other summary metrics is possible; see the `plot_history()`
documentation.

## Citation
If using the package, please consider citing:

```{r}
citation(package = "adaptr")
```
