The purpose of this study was to rate the oral retellings of fifth-grade students to determine to what degree passages, raters, and rating occasions affect those ratings, and to identify what combination of those elements will produce reliable retelling ratings. Thirty-six fourth-grade students read and orally retold three contemporary realistic fiction passages. Two raters rated these retellings on two separate occasions using the Reader Retelling Rating Scale. These ratings were analyzed quantitatively using generalizability software. Two research questions were answered by the generalizability (G) and decision (D) studies. The G study answers the first question regarding the percentages of the total variation that can be attributed to the students, the raters, the rating occasions, the passages, and interactions among these factors. The G study found that the largest sources variation were the students, the passages, and the student-by-passage interaction. The D study answered the second question about how many raters, rating occasions, and passages would be needed to obtain a reliability coefficient for similar students in another setting. To obtain high reliability coefficients, retellings of a minimum of four (preferably six) passages should be rated by at least two raters on one occasion.

The purpose of this study was to refine a rating procedure used to assess intermediate elementary school students' ability to orally retell what they had read from two expository passages. Oral retellings from 28 fourth grade students were tape-recorded and rated on two different occasions by each of 4 raters. A four-facet (passage, day of test administration, rater, and rating occasion) generalizability study was conducted using a partially nested design. The six largest sources of variability identified in the G-study included (a) students, (b) the student-by-day interaction, (c) the interaction of passage with rater (nested within student and day), (d) the student-by-day-by-occasion interaction, (e) the passage-by-raters (nested within students and day)-by-occasion interaction, and (f) the residual. A D-study was conducted to predict the values of the error variances and generalizability indices for both relative and absolute decisions. The results show how the error variance and the generalizability coefficients vary as a function of the number of passages, days of test administration, raters, and rating occasions. The results of the D study indicate that adding an extra reading day would produce a greater increase in reliability than asking the students to read more passages, or using more raters or more rating occasions. To achieve the greatest gain in generalizability, teachers should have students read at least two passages on at least two separate days and have their retelling rated by at least two raters and then compute a mean rating for each student averaged across the various passages, testing days, and raters.

No description available.