On making multiple comparisons in clinical and experimental pharmacology and physiology

Clin Exp Pharmacol Physiol. 1991 Jun;18(6):379-92. doi: 10.1111/j.1440-1681.1991.tb01468.x.

Abstract

1. It is a central thesis of this review that in clinical and experimental pharmacology and physiology the goal of statistical analysis should be to minimize the risk of making any false-positive inferences from the results of an experiment (experimentwise Type I error). 2. It is common in clinical and experimental pharmacology and physiology for the effects of several treatments to be tested within a single experiment. Specific intercomparisons of these several effects, made in a pairwise or more complex fashion, inflates the risk of making false-positive inferences unless special statistical procedures are used. 3. A number of multiple comparison procedures is described and their ability to control experimentwise Type I error is evaluated critically. 4. When only a few (less than 5) of all possible pairwise or more complex comparisons are made between treatment groups, the Dunn-Sidák procedure provides maximum protection against excessive experimentwise Type I error and is very convenient to use. 5. When a control group is compared with all other treatment groups in a pairwise fashion, especially when the number of groups is large, the Dunnett procedure is more powerful than the Dunn-Sidák. 6. If investigators insist on making all possible pairwise comparisons among treatment groups, the Tukey-Kramer procedure provides maximum protection against false-positive inferences but inflates the Type II error rate. If it is especially important to avoid Type II error then the more complicated, stepwise procedures of the Ryan-Peritz-Welsch variety should be considered.

Publication types

  • Comparative Study
  • Editorial
  • Review

MeSH terms

  • Analysis of Variance
  • False Positive Reactions
  • Software
  • Statistics as Topic / methods*