Description
Tools for Type S (Sign) and Type M (Magnitude) Errors.
Description
Provides tools for working with Type S (Sign) and Type M (Magnitude) errors, as proposed in Gelman and Tuerlinckx (2000) <doi:10.1007/s001800000040> and Gelman & Carlin (2014) <doi:10.1177/1745691614551642>. In addition to simply calculating the probability of Type S/M error, the package includes functions for calculating these errors across a variety of effect sizes for comparison, and recommended sample size given "tolerances" for Type S/M errors. To improve the speed of these calculations, closed forms solutions for the probability of a Type S/M error from Lu, Qiu, and Deng (2018) <doi:10.1111/bmsp.12132> are implemented. As of 1.0.0, this includes support only for simple research designs. See the package vignette for a fuller exposition on how Type S/M errors arise in research, and how to analyze them using the type of design analysis proposed in the above papers.
README.md
retrodesign
Tools for calculating and working with Gelman et al's Type S (sign), and M (magnitude or exaggeration ratio) errors for analyzing hypothesis tests in R.
Features:
Functions for calculating Power and Type S/M errors across variety of effect sizes, building on code provided in Gelman and Carlin (2014, see below).
Graphics function for visualizing these errors with or without ggplot
Implementation of Lu et al's (2018, see below) closed form solution for Type M error, providing a speed-up.
Usage
You can install retrodesign with:
install.packages("retrodesign")
You can find an online version of retrodesign's vignette on my website, which provides an introduction to both retrodesign and Type S/M errors.
More Reading on Type S/M Errors
- Gelman and Tuerlinckx's Type S error rates for classical and Bayesian single and multiple comparisons procedures (2000): A comparison of the properties of Type S errors of frequentist and Bayesian confidence statements. Useful for how this all plays out in a Bayesian context. Bayesian confidence statements have the desirable property of being more conservative than frequentist ones.
- Gelman and Carlin's Assessing Type S and Type M Errors (2014): Gelman and Carlin compare their suggested design analysis, as we've written about above, to more traditional design analysis, through several examples, and discuss the desirable properties it has in more depth than I do here. It is also the source of the original retrodesign() function, which I re-use in the package with permission.
- Lu et al's A note on Type S/M errors in hypothesis testing (2018): Lu and coauthors go further into the mathematical properties of Type S/M errors, and prove the closed form solutions implemented in
retrodesign. - McShane et al's Abandon Statistical Significance (2017): If you want a starting point on the challenges with NHST that have led many statisticians to argue for abandoning NHST all together, and starting points for alternative ways of doing science.