This paper is from Session 7D: Interactive software targeting specific statistical concepts
which comes under Topic 7: Technology in Statistics education Full topic list
(Thursday 6th, 10:30-12:30)
Understanding replication: confidence intervals, p-values, and what’s likely to happen next time
Presenter
Abstract
Science loves replication: We conclude an effect is real if we believe replications would also show the effect. It is therefore crucial to understand replication. However, there is strong evidence of severe, widespread misconception about p values and confidence intervals, two of the main statistical tools that guide us in deciding whether an observed effect is real. I propose we teach about replication directly. I describe three approaches: Via confidence intervals (What is the chance the original confidence interval will capture the mean of a repeat of the experiment?); Via
p values (Given an initial
p value, what is the distribution of
p values for replications of the experiment?): and via Peter Killeen’s ‘
prep’, which is the average probability that a replication will give a result in the same direction. In each case I will demonstrate an interactive graphical simulation designed to make the tricky ideas of replication vividly accessible.
 | 
