In response to concerns about using only standardized multiple-choice assessments, some school districts have moved to using alternative ratings of student achievement with authentic assessments. However, such assessments are often limited in terms of the psychometric validity data supporting their use. The present study mixed quantitative and qualitative methods to examine the validity, development, and refinement of an authentic rating approach currently in use in middle school mathematics classes from a large suburban school district in the Southwest United States. A sample of teachers (n = 14), rated students (n = 110) using a pilot rubric of 187 items. Analyses resulted in a 32-item rubric with 20 themes and 9 factors. Results from a G-study revealed the facet that best explained variance in student scores was the interaction between raters and assessment units, as well as students and assessment units. As part of the development of the assessment, a content validity exercise revealed 18% of the rubric items as below average quality. Findings highlight the need to enhance contextualization of rubrics, use a strategy of assessment that includes contextualized and decontextualized assessment, and to investigate the role of utilization deficiency in explaining low student scores.
| Identifer | oai:union.ndltd.org:unt.edu/info:ark/67531/metadc1707388 |
| Date | 08 1900 |
| Creators | Raadt, Jay Schyler |
| Contributors | Hull, Darrell, Combes, Bertina, Tashakkori, Abbas, Middlemiss, Wendy |
| Publisher | University of North Texas |
| Source Sets | University of North Texas |
| Language | English |
| Detected Language | English |
| Type | Thesis or Dissertation |
| Format | iv, 33 pages, Text |
| Rights | Public, Raadt, Jay Schyler, Copyright, Copyright is held by the author, unless otherwise noted. All rights Reserved. |
Page generated in 0.0023 seconds