This paper is from Session 8B: Inference in Times of Crisis, Part 2
which comes under Topic 8: New approaches to research in statistics education
Paper 8B3 (Tuesday 10th, 14:00-15:30)
A simulation study of the strength of evidence in the recommendation of medications based on two trials with statistically significant results
Presenter
- Don van Ravenzwaaij (University of Groningen, The Netherlands)
Abstract
For the endorsement of new medications, the US Food and Drug Administration typically require two trials, each with p.05, to demonstrate effectiveness. In this paper, we calculated with simulations what it means to have exactly two trials with p.05 in terms of the actual strength of evidence quantified by Bayes factors. Our results show that different cases where two trials have a p-value below .05 have wildly differing Bayes factors. In a non-trivial number of cases, evidence actually points to the null hypothesis. We recommend use of Bayes factors as a routine tool to assess endorsement of new medications, because Bayes factors consistently quantify strength of evidence.