Thesis (Ph.D.)--Lehigh University, 2009. / Adviser: George J. DuPaul.

Thesis (Ph.D.)--Lehigh University, 2009. / Adviser: Christine L. Cole.

The purpose of this study was to investigate the construct validity of the performance levels and their associated descriptors for a state standards-based fifth grade mathematics assessment. Using item responses from 65,291 students, who took the test during the Spring 2000 semester, the study seeks to evaluate the degree to which the descriptions of the performance levels constructed for a state standards-based assessment accurately represent what students at each level know. This was done through four separate, but related, analyses: (1) mapping items to performance level descriptions, (2) assessing the fit of the items to an IRT model, (3) assessing correspondence between items and performance level descriptors, and (4) assessing fit of student responses to the expected response pattern. Results of the analyses show that the performance level descriptions do not provide completely valid portrayals of what students know and are able to do.

High stakes tests have become a prominent tool in the systemic reform movement documenting the need for change and serving as the instrument of educational change. The primary purpose of this investigation was to determine whether the positive consequences associated with high stakes test use and interpretation in Arizona were shared among all grade levels, not just the tested grades. Additionally, a curriculum alignment variable was examined to observe its association with curricular and instructional change. The AIMS Questionnaire was developed using principal components with varimax rotation and the Multitrait-Multimethod Matrix (Campbell & Fiske, 1959). The questionnaire was administered to elementary teachers using a Posttest-Only with Nonequivalent Groups quasi-experimental research design (Cook & Campbell, 1979) where teachers in the nontested grades (1, 2, and 4) served as the comparison group. A two-factor analysis of variance was performed to examine the primary hypothesis, and the Pearson Product Moment correlation was computed to observe the strength of the relationship between the curriculum alignment variable and the curricular/instructional change variable. Results of the analysis suggested that positive consequences were not equally shared among all grade levels in the elementary school. Additionally, the curriculum alignment variable accounted for less than 2% of the variance in the change variable. It is recommended that policymakers use a randomized testing model and select a new grade level and a new form of the test each year. Further, educational leaders should use curriculum alignment strategies with caution as they may be viewed as top-down change strategies that constrain a teacher's creativity. Future researchers should consider the use of predicted pattern testing (Levin & Neumann, 1999) to statistically examine the system-wide effects of a high-stakes assessment designed to impact student learning.

This dissertation offers a comprehensive rhetorical analysis and theoretical critique of Advanced Placement English. AP English is identified and explored throughout as a site that instantiates most of the controversies that have arisen over the past several generations regarding rhetoric, composition, and the teaching of English. Chapter 1 focuses on the AP English Language and AP English Literature exams to probe the tensions between expanded notions of writing and reading processes against the demands of large-scale testing. It is argued that the exams valorize a problematic formalist approach to literary texts and undercut and attenuate the writing process. Some of the exams' flaws are considered as a means of demonstrating ways their validity can be challenged. In Chapter 2 the College Board's two AP English course descriptions are examined in detail. While explicitly acknowledging contemporary theoretical reconsiderations for the teaching of English, each course's "explicit curriculum" is subverted by an "implicit curriculum" rooted in New Criticism and current-traditional rhetoric that are fostered by the demands of the AP exams. In Chapter 3 the development of the AP Program is situated within its earliest historical, political, and ideological contexts by linking it to the values and methodologies of the first College Board entrance exams of the early 1900s, key aspects of which were later resurrected in the AP Program. The historical consideration continues in Chapter 4 through an analysis of the specific impetuses that created the AP program in the 1950s. Particular attention is paid to the often confused and contrary relationship between high school and college English as well as the often unstable and undefined position of the first-year college English course. Chapter 5 concludes with a personal explanation and interpretation of what it means to teach AP and its version of college English.

This study used a standardized teacher observation rubric and procedures to evaluate the performance of 63 new teachers with various preparation backgrounds. Observers rated teachers from seven different school districts on 29 separate criteria of teaching effectiveness. New teachers were assessed on their lesson planning, assessment practices, classroom management, and implementing instruction during interviews and observations. Results show significant differences in the performance of new teachers depending on the model of the teacher education program they attended. Traditional undergraduate program graduates were rated higher than teachers who received their preparation from post-baccalaureate or master's degree certification programs (particularly in the area of classroom management and at the middle school level). Follow-up interviews were conducted with a stratified random sample of 15 of 63 participants. New teachers reported difficulty setting up classroom management procedures at the beginning of the semester especially small group instruction and would have preferred more classroom experiences during their teacher education program. Teachers strongly affirmed the importance of teacher education for their ability own to teach.

The conceptualization of validity has evolved over time, from the reign of criterion validity as a prominent type of validity through the phase of the traditional validity trinity concept that considered construct, content, and criterion validity as different kinds or types of validity. The current view among the measurement community is that there are no distinct forms of validity; instead validity is the extent to which the appropriateness of proposed uses and interpretations can be supported by various kinds of validity evidence. National examinations such as the Junior Certificate Examination in Botswana typically assess content and skills defined by national curricula. The extent to which items in examination papers are relevant to important content and cognitive skills espoused by national curricula is critical to the accuracy, appropriateness, and fairness of examinations results. This study investigated content, substantive, reliability, and internal structure validity evidence of the Junior Certificate Mathematics Examination over a period of three years, 2000, 2001, and 2002. Three alignment models were used to investigate content and cognitive skill validity evidence. A correlational analysis and exploratory factor analysis were used to detect the internal structure of the 2000, 2001, and 2002 Junior Certificate Mathematics examination papers and reliability of the objective tests was assessed through Coefficient alpha. The results showed that the sampling of mathematics content fluctuates from year to year, and does not always reflect content emphases in the Mathematics syllabus. Content of items in all three years' examination papers was judged as sufficiently aligned to content expressed in syllabus objectives the items were intended to measure using a liberal alignment criterion. The results of the study also indicated that the 2000, 2001, and 2002 Paper 1 component of the Mathematics examinations were sufficiently reliable albeit minimally so. Results of the exploratory factor analysis indicated that the Paper 1 component of the Mathematics examination assesses a possibly multidimensional construct. The findings of this study highlight the need for more comprehensive and systemic validity studies that would continue to generate information concerning the validity of examinations in Botswana.

The purpose of the present study was to investigate WISC-III Verbal short forms for school-age children identified as visually impaired. The rationale for use of WISC-III Verbal short forms is to provide a time saving and technically sound measure for the screening and/or periodical re-evaluation of children with visual impairments who may require special education services. The total sample consisted of the WISC-III test scores of 78 children identified with the handicapping condition of either partial sightedness or legal blindness, who had been given the regularly administered Verbal subtests. The mean age level of the total sample was 11 years, 4 months. Three additional groups were obtained from the test records of the original data which included, children identified as either partially sighted or legally blind (n=62) who were given the regularly administered Verbal subtests and supplementary Digit Span subtest (referred to as the Digit Span group), and children identified as partially sighted (n=47), and legally blind (n=31), who were given only the regularly administered Verbal subtests. Ninety-seven percent of the data were obtained from existing test records, and the remaining data from the actual administration of the WISC-III Verbal subtests for the purpose of collecting data for the study. Children identified with the handicapping conditions of multiple disabilities or mental retardation were not included in the sample. For the total sample and additional groups, the mean Verbal IQ score and mean scaled score of the individual subtests fell in the average range. The WISC-III Verbal short form combinations were identified by utilizing the formulas of Tellegen and Briggs (1967) for determining the reliability and validity coefficients of short form combinations. High reliability and validity coefficients were obtained for all two-, three-, and four-subtest combinations of the total sample and additional groups. The study results suggest WISC-III Verbal short forms offer a time efficient and technically sound measure to be utilized in assessment of the verbal intellectual development of school-age children with visual impairments.

The performance of children with Attention-Deficit Hyperactivity Disorder (ADHD) or Learning Disability (LD) were compared to "typical" children using the Attention-Deficit Hyperactivity Disorder Experimental Assessment Battery (AEAB). The AEAB is a battery of assessment instruments that have been used individually and in various combinations in research to either (1) measure attention or other ADHD characteristics in research participants or (2) discriminate LD characteristics from the characteristics of typical participants. The present study examined whether these assessment instruments differentiated ADHD and LD from typical children. The two hypotheses that were addressed were (1) the combined scores on the Child Behavior Checklist - Attention Problems Scale, the Stroop Color - Word Test, the Hand Movements Test, and the Audiovisual Checking Task will discriminate ADHD from typical participants and (2) the combined scores on the Minnesota Percepto-Diagnostic Test - Revised and the Trail Making Test will discriminate LD from typical participants. The instruments used to measure attention characteristics were the Child Behavior Checklist - Attention Problems Scale (Achenbach & Edelbrock, 1983), the Stroop Color - Word Test (Golden, 1978), The Hand Movements Test from the Kaufman Assessment Battery for Children (Kaufman & Kaufman, 1983) and the Audiovisual Checking Task (Margolis, 1973). Two instruments were used to measure and discriminate LD characteristics. They were the Minnesota Percepto-Diagnostic Test - Revised (Fuller, 1983) and the Trail Making Test from the Halstead - Reitan Neuropsychological Test Battery (Reitan & Wolfson, 1985). A Kruskal - Wallis Analysis of Variance by Ranks (Hays, 1973) was calculated for the groups. The results showed that the groups were significantly different for the analyzed combined scores. These findings were discussed in regard to the research literature in this area.

The primary purpose of this study was to assess certain aspects of the internal structure of the DISCOVER assessment checklist to determine its construct validity. A secondary purpose was to assess gender differences in identifying giftedness using the assessment. The sample of this study consisted of 368 participants from kindergarten, fourth, fifth, and sixth grade levels divided into three subsamples. Participants were from two culturally diverse populations: Navajo Indians and Mexican-Americans. The methodology consisted of correlational analyses and chi-square tests. Separate, but identical analyses were conducted on each subsample. Three questions guided this study: (a) What is the relationship between observers' ratings of participants' problem-solving ability in one activity and their rating of participants in the other activities? (b) What items characterize each of the four rating categories? and (c) What are the gender differences occurring when using the assessment? The results of this study showed low and non significant inter-rating correlations, indicating high discriminant validity of the checklist. Significant, but low to moderately high inter-rating correlations were found between the Storytelling and Storywriting activities across subsamples. R-squared analyses revealed low percentages of variance accounted for, indicating low convergent validity of the checklist. Moreover, a pattern of higher percentages of item checks was found for higher ratings, indicating that observers checked items at a higher frequency rate for participants given higher ratings. Only items characterizing the "Definitely" category were possible to identify because all items represent superior problem-solving skills, thus were mostly checked for participants who demonstrated superior problem-solving processes or products. Items with zero frequencies were identified as well. Chi-square tests for gender by activity revealed significant gender differences in two activities. In Pablo°ler, a significantly higher number of sixth grade boys were rated "Definitely" and in Storytelling, a significantly higher number of fourth and fifth grade girls were rated "Definitely". No significant gender differences were found for any other activity across subsamples. Similarly, no significant overall gender differences were found across subsamples for gifted participants, indicating that equal proportions of males and females are identified through the use of the DISCOVER assessment.