This observational study examines inferential reasoning development in students taking a randomization-based introductory college course (n = 38). The proportions of students answering each item correctly are compared from Pretest to Posttest using a simple descriptive scatterplot. This “canoe plot” includes a superimposed 95% confidence band which differentiates items with no statistical difference between the proportion answering correctly on the Pretest and Posttest from those that do differ. A brief discussion follows highlighting the content for those items where inferential reasoning differs from Pretest to Posttest. The Reasoning about P-values and Statistical Significance (RPASS) scale is used to measure students’ inferential reasoning outcomes and gains. Directions for future research are discussed.