Large-scale assessment results for schools, school boards/districts, and entire provinces or states are commonly reported as the percentage of students achieving a standard – that is, the percentage of students scoring above the cut score that defines the standard on the assessment scale. Recent research has shown that this method of reporting is sensitive to small changes in the cut score, especially when comparing results across years or between groups. This study extends that work by investigating the effects of reporting group size on the stability of results. For each of ten group sizes, 1000 samples with replacement were drawn from the May 2009 Ontario Grade 6 Assessment of Reading, Writing and Mathematics. The results showed that for small group sizes – analogous to small schools – there is little confidence and that extreme caution must be taken when interpreting differences observed between years or groups.

Large-scale assessment results for schools, school boards/districts, and entire provinces or states are commonly reported as the percentage of students achieving a standard – that is, the percentage of students scoring above the cut score that defines the standard on the assessment scale. Recent research has shown that this method of reporting is sensitive to small changes in the cut score, especially when comparing results across years or between groups. This study extends that work by investigating the effects of reporting group size on the stability of results. For each of ten group sizes, 1000 samples with replacement were drawn from the May 2009 Ontario Grade 6 Assessment of Reading, Writing and Mathematics. The results showed that for small group sizes – analogous to small schools – there is little confidence and that extreme caution must be taken when interpreting differences observed between years or groups.