The purpose of the present study was to investigate the viability of a computer-based assessment system, called the System for the Assessment of Reading Competencies, to aid in the description and diagnosis of reading difficulties. The practical need for such a system, the theoretical bases that serve as its foundation, and the evidence for the system's validity are presented. Currently, there is little relation among diagnosis, assessment, remediation, and instruction in reading. The lack of integration of these areas and the need for assessment instruments that provide sufficient information for diagnosis and remediation led to this system's development. The present study involved the implementation and initial validation of the system. The system is a computer-based, componential, reading assessment instrument that is based on an information processing model of reading. One-hundred-and-twelve second, third, fourth, and fifth grade students were administered Sentence Verification Technique (SVT) Tests of listening and reading comprehension in addition to several computer tasks. These tasks included a response time measure, letter matching, word naming, pseudoword naming, category matching, syntactic analysis, and sentence comprehension. Both response times and response accuracies were collected on each task. The results were analyzed to examine the reliability and validity of the assessment system. The results showed that the assessment battery was reliable. Further, the results showed that the system successfully discriminated between students in different grade levels and between students of differing abilities within the same grade. The results showed a developmental trend such that the largest differences between ability groups were on the lower level tasks for second graders and on the higher level tasks for third and fourth graders. It was concluded that the evidence supports the reliability and validity of the system as a measure of reading ability, and further that the system has potential as a diagnostic instrument. It was also concluded that measures of response time may discriminate between ability groups even when measures of response accuracy do not. The development of a remedial component to the assessment system is discussed as a direction for future research.

Leadership for learning has arisen as a robust framework for measuring principal leadership practices that impact student achievement by synthesizing fundamental theories of principal leadership, such as instructional leadership, transformational leadership, and shared instructional leadership. Research affirms the need for principals to prioritize leadership for learning practices based on the context. However, we know little about how context impacts which leadership for learning practices are prioritized by teachers and administrators in these different contexts of school level (elementary or secondary) and region (rural, small town, suburban, urban). The purpose of this dissertation study is to examine how differences in context affect specific leadership for learning practices. These differences will provide evidence for the need to revamp the principal evaluation systems that are currently in use and allow us to identify opportunities for preparation, support, and professional development for principals. I use the Comprehensive Assessment of Leadership for Learning (CALL) survey for this dissertation study as it combines recent research on principal leadership to measure specific leadership skills. Collected by the researchers of the CALL group at the University of Wisconsin-Madison, I use the 2016-2019 CALL dataset in the present study, which includes 281 schools with 536 administrators and 8,027 teachers. This dissertation study examines a) differences in teacher and administrator ratings within the CALL assessment, b) differences in secondary and elementary school level ratings within the CALL assessment, c) differences in administrator’s ratings by region within the CALL assessment. The results of this dissertation study show that a) administrator perceptions of the frequency of leadership for learning practices are always higher than teacher scores for any leadership for learning practice, b) elementary schools had fewer disagreements across the leadership for learning practices in comparison to secondary schools when comparing the perceptions of the frequency of practices in leadership for learning scores between teachers and administrators, and c) substantial differences occur in the perception of the frequency of leadership for learning practices between urban school administrators and administrators in other regional contexts (rural, small town, suburban), especially around issues of school climate, summative evaluation of teachers and students, and resourcing strategically.

Constructing School Organization Through Metaphor: Making Sense of School Reform is an in-depth case study of a K-12 laboratory school affiliated with a major state university. The school was undergoing a series of significant changes, mainly due to mandates brought about through educational reform policy and the upcoming loss and replacement of their school facilities. The study was of a particular period in time, and spanned approximately one year. The directed, open-ended questions asked during interviews with adults in the school were related to changes the school was undergoing, how these changes were being implemented, and how these were affecting teachers in relation to their teaching. Insights about the organization were in part gained by the use of metaphor as a tool for looking at organizational structure, and for viewing and describing the meanings that educators created around their roles, professions, and organization. Qualitative research was chosen as the best method for studying these research questions: 1) What understandings about the organization and the changes exist? A) What organizational metaphors are played out in this school undergoing rapid change? B) Are multiple metaphors conflicting or complementary? 2) What are the consequences of competing metaphors on educators and the organization? During times of change, an organization's metaphors are more readily apparent as the actors respond to the pressures of change: this was true at the lab school. Metaphors helped expose how individuals constructed shared meanings about their school, the changes impacting the school, and themselves as members of the organization. The study provides insights into how metaphor and rhetoric were used by educators and others to help construct the social reality of their school, a reality played out through the school's culture. Great concerns surfaced during the interviews about the role of the teacher, the needs of students, the purposes of education, and issues about reform. At the same time, holding the culture together were certain underlying values, characteristics, and expectations—mainly a commitment to student learning and the best interests of the students. The shared metaphor of "teacher" allowed the culture of the school to survive with some strength, even while co-existing with the dissonance caused by other, competing metaphors. However, the shared metaphor of "teacher" did not reduce the school's struggle with change, and the socializing aspects of the culture did not appear to be contributing to an overall understanding or acceptance of the proposed new school and new metaphors necessary to implement the changes and mandates. The research is descriptive in nature, and data (observations, interviews, and study of artifacts inside and outside of the setting) were inductively analyzed. The narratives of the people interviewed are the primary data. Aggregated data reported in this study are excerpts from the interviews with forty-seven adults within the school setting, compiled in such a way as to represent the repeated issues and mix of "voices" of those interviewed. The thick data collected provides information on how educators within the school were making sense and meaning of themselves and their organization as the school underwent great change. The events that took place were observed, recorded, and analyzed through open coding into themes that described the changes, metaphors, negotiations, and processes taking place: these constructed the realities within the school. Metaphors were seen to effect and be affected by a series of changes within the school and by the rhetoric of school members. The researcher's intention was accomplished--i.e., to examine and consider how the institutional arguments, as reported by stakeholders in the organization, were negotiated through a complex, interactive process. The examination was fundamentally based on the metaphors used by the participants, which both revealed and helped to create their views of the reality of the organization. Metaphors were also used in the production of the study as a means of helping the reader understand what was taking place in this school. As a way of looking at people's behavior, organizations, and life constructs, metaphors serve as tools for understanding, identifying, and describing how educators within the school perceive and construct their organization and manage their work lives. Educators use metaphor to construct their realities of themselves as professionals and of their school as an organization, to share beliefs and realities with others, and to influence decisions. Some of the metaphors found and used in the lab school had to do with organizational issues, others with individual issues, including identity. Understanding the dynamics between the existence and use of metaphors, organization structure, and the people who work in the organization is important to educators and policy makers as schools re-create themselves to meet new mandates. The narratives describe and give insights into how people in the organization used metaphors to organize their structures and work, and to negotiate, manage, construct, and deal with their realities and relationships with each other. The narratives and descriptions of the research also use metaphors to facilitate readers' understanding of this study and to link the narratives of those interviewed back to the literature review. The narrative data reveals that mandated changes were affecting the identity of teachers as professionals. Educators' typifications of themselves as teachers, with the best interests of students at heart, allowed the educators to function as a school that enjoyed some measure of success, even when their was no consensus around changes in the organization. Paradoxes existed in the form of opposing beliefs and realities of what was happening in the school, and educators talked about the school and its changes in ways that were contradictory to how they behaved as members of the school. Some educators talked about the organization as a dictatorship or other type of organization, while almost all of them behaved in ways consistent with a learning organization. On occasion, educators talked about competing realities within the school. Consternation about changes in the school and individual realities caused a variety of reactions, including fight-flight, avoidance, and engagement. The use of rhetoric to inform, build, or eliminate metaphors was in evidence, and members of the school tended to group with "like minded" people who reinforced their existing beliefs. People in the school interacted with others based on each person's own stock of knowledge, which was informed, enlarged, reinforced, and changed through metaphor: realities were a constant work in progress. These sets of old assumptions and beliefs helped create paradoxes: the teachers who were interviewed focused on their stressors and distress about the changes in the school, versus the way they interacted as a learning organization with the students and each other in positive and supportive ways. This focus on the negative aspects of changes in the school appeared to be in large part due to the probe questions with which the researcher began the interviews. While the school's Director focused on organizational metaphors, the teachers were focused on person-centered metaphors. There was no metaphor being promoted by the leaders in the school that was more attractive than the metaphors and identities members of the school were losing, particularly those of "lab" school, "professor," and "families." Organizational literature mainly deals with organization-centered metaphors, whereas the person-centered ones that teachers related to most point to a gap in the organizational literature. The metaphor of democracy, which is important to site-based management such as the school had, was jeopardized by a lack of participation across stakeholders such as parents, teachers, and students. There were many valid reasons for this lack of participation, as reflected in the study and which were in agreement with findings of other researchers. School members struggled with issues unique to their school, but also those faced by other lab and site-managed schools and the U.S. workforce in general. Communications appeared to be one of the biggest barriers to effecting change, and the reasons for communication breakdowns were varied. In addition, the school was undergoing a transformation from lab school to professional development school (PDS), yet forty-five of the forty-seven people interviewed seemed unaware of this. The transformation to a PDS reflected the trend of other lab schools in the U.S. that had managed to survive by changing their organizations and identities. Change brought on other challenges as well: those who taught core or state-tested disciplines were challenged to cover all of the requirements and still keep their teaching engaging, up-to-date, and meaningful. Most of the excitement and innovation in the middle and high schools seemed to come from "elective" courses, with the exception of an integrative arts-based program that included core courses, such as math and science. Although educators focused mainly on the stressors caused by change when questioned about change during the interviews, the majority of them behaved in the sharing and interactive ways of a learning organization. At the same time that the school had most of the attributes of a learning organization, most of the educators did not appear to understand key points of this type organization, particularly that chaos is real and embraceable, that change is constant, quick, and part of an everyday process. / A Dissertation submitted to the Department of Educational Leadership and Policy Studies in partial fulfillment of the requirements for the degree of Doctor of Philosophy. / Summer Semester, 2004. / April 22, 2004. / Educational Reform, Mimesis, Eduactional Research, School Administration, Change Management / Includes bibliographical references. / Carolyn D. Herrington, Professor Directing Dissertation; Karen L. Laughlin, Outside Committee Member; Sande D. Milton, Committee Member; Terrence R. Russell, Committee Member; Jerome S. Osteryoung, Committee Member; Patrice M. Iatarola, Committee Member.

Boys and men tend to do better in math class and to have higher math confidence than their female classmates. It has been hypothesized that low confidence is a precursor to poor performance. Because of this, a great deal of effort has been expended on raising our students' confidence. As a result of this, students in the United States are more confident of their math ability than students anywhere else in the world. Despite this, math performance remains low, for both male and female students. American students seem to interpret this low performance as an indication of limited ability. To change this interpretation, I told some students in introductory calculus classes that their prior math failures were due to low effort and that increased effort should lead to success in college. To make this information more believable, it was embedded in a personality profile that had been generated specifically for the student, and given a scientific sounding name: Talent/Motive Disjunction (TMD). Students who were told that they had TMD did significantly better in calculus than students who were not told they had TMD. This increase in performance was not the result of increased confidence. Students in the TMD condition were no more confident at the end of the semester than students in the No TMD condition. This suggests that changing attributions about success and failure may be effective in improving our students' performance even if confidence raising is not.

The goal underlying educational testing is to measure psychological constructs in a particular domain and to produce valid inferences about examinees’ ability. To achieve this goal of getting a precise ability evaluation, test developers construct questions with different formats, such as multiple-choice (MC) items, and open-ended questions or constructed response (CR) test items, for example, essay items. In recent years, large-scale assessments have implemented CR items in addition to MC items as an essential component of the educational assessment landscape. However, utilizing CR items in testing involves two main challenges, including rater effects and rater correlations. One challenge is the error added by human raters’ subjective judgments, such as rater severity and rater central tendency. Rater severity effect refers to the effect that raters may tend to give consistently low or high ratings that cause biased ability evaluation (Leckie & Baird, 2011). Central tendency describes when raters tend to use middle categories in the scoring rubric and avoid using extreme criteria (Saal et al., 1980). The second challenge is that multiple raters usually grade an examinee’s essay for quality control purposes; however, ratings based on the same item are correlated and need to be handled carefully by appropriate statistical procedures (Eckes, 2011; Kim, 2009). To solve these problems, DeCarlo (2010) proposed an HRM-SDT model that extended the traditional signal detection theory (SDT) model used in the first level of HRM. The HRM-SDT model not only considers the hierarchical structure of rating data but also deals with various rater effects beyond rater severity. This research examined to what extent the HRM-SDT separates rater effects (i.e., rater severity and rater central tendency) from item effects (i.e., item difficulty). Accordingly, one goal of this study was to simulate various rater effects and item effects to investigate the performance of the HRM-SDT model with respect to separating these effects. The other goal was to compare the fit of the HRM-SDT model with one commonly used model in language assessments, the Rasch model, in different simulation conditions and to examine the difference between these two models in terms of segregating rater and item effects. To answer these questions, Simulation A and Simulation B were conducted. In Simulation A, seven sets of parameters were varied in the first set of simulations. Simulation B addressed some questions of particular interest using another four sets of parameters, where both the rater and item parameters were simultaneously varied. This study found the HRM-SDT accurately recovered parameters, and clearly detected and separated changes in rater severity, rater central tendency, and item difficulty in most conditions.

The purpose of the current study was to test the theory of action hypothesized for the Mathematics Formative Assessment System (MFAS) based on results from a large-scale randomized field trial. Using a multilevel structural equation modeling analytic approach with multiple latent response variables decomposed across student, teacher, and school levels of clustering, the current study found evidence of effects of MFAS that were consistent with the MFAS theory of action. First, assignment to the treatment condition was associated with higher mean student mathematics performance and a higher prevalence of small group instruction compared to schools assigned to the control condition—both of which are outcomes hypothesized to result from MFAS use. Also, a positive association between teacher-level mathematics knowledge for teaching and student mathematics performance was found in the current study, which is consistent with the interrelation of constructs specified in the MFAS theory of action. However, evidence of the particular linkages of MFAS use→teacher knowledge→classroom practice→student mathematics performance and the putative cascade of effects that would substantiate the mechanisms of change posited in the MFAS theory of action were not detected in the current study. Thus, positive effects of MFAS on teacher and student outcomes were substantiated; however, as to how the effects of MFAS on teachers transfer to improved outcomes for students remains to be empirically demonstrated. Based on my review of the results from the current study and consideration of the literature on formative assessment as it relates to the design of MFAS tasks and rubrics, I discuss a proposed modification to the theory of action that specifies the addition of a direct path from MFAS use to student mathematics performance, in addition to the indirect path currently specified. / A Dissertation submitted to the Department of Educational Leadership and Policy Studies in partial fulfillment of the requirements for the degree of Doctor of Philosophy. / Summer Semester 2016. / June 29, 2016. / Dynamic Assessment, Formative Assessment, Mediation Analysis, Multilevel Structural Equation Modeling, Multisite Cluster Randomized Field Trial / Includes bibliographical references. / Laura B. Lang, Professor Directing Dissertation; Barbara R. Foorman, University Representative; Carolyn D. Herrington, Committee Member; Jeffrey A. Milligan, Committee Member.

Collaborative problem solving (CPS) is a critical competency, because much of the work people do occurs in a social context involving direct collaboration. Meanwhile, schools are being pressured to reduce the amount of time devoted to large-scale assessments, and to adopt more natural or authentic assessments. It may be possible to address both these issues at once, if collaboration experiences are viewed as opportunities to assess student achievement.However, several issues arise in evaluating individuals’ problem solving skills in a collaborative context: (1) collaborative learning outcomes may obscure individuals’ contributions, making it difficult to isolate individuals’ performance; (2) outcome-based measures may ignore the processes of individuals’ or groups’ problem solving, thus leading to inaccurate estimations of individuals’ or groups’ knowledge, abilities and skills; and (3) prior evaluative focus of CPS research has usually been on social aspects of collaboration rather than domain-relevant cognitive skills. Therefore, the present study aimed to develop a process-based scoring rubric to evaluate individual student achievement in problem solving using collaboration process data. Furthermore, this study explored how group ability composition affects group performance and individual learning gains. The content domain was solving mathematical combinatorics and probability word problems. Participants included 306 Chinese high school students, who performed the following three tasks in order: (1) an individual pretest with seven problems; (2) a collaborative task with three problems; and (3) an individual posttest with seven problems. The results were as follows: First, a four-indicator scoring rubric was developed to evaluate student achievement in solving combinatorics and probability problems using collaboration process data. Evidence suggested that the scoring rubric can be considered reliable and valid in terms of being used as an individual assessment and a teaching tool. Thus, this scoring rubric may provide insights useful for developing relevant performance assessments on more complex and authentic performance tasks. Second, significant differences were found in group performance among dyads as a function of the group’s minimum student ability. Third, students working in dyads with higher maximum or average ability tended to gain more after collaboration. Finally, certain collaborative problem solving behaviors could be linked to individual learning gains.

Problem. Most of the diagnostic tools available for the assessment of language and cognition in early childhood education have been designed for fluent English-speakers. Few instruments include items appropriate to the needs of Spanish-speaking children with limited English Proficiency (LEP). Available Spanish language test are translations of English tools and have not been standardized among the Hispanic population. Classroom teachers and education specialists need an appropriately validated Spanish test for educational planning and programing of LEP Spanish-speaking children. Purpose. The purpose of this study was three dimensional: (1) to develop a functional adaptation of a criterion-referenced diagnostic test for the assessment of language and cognitive skills of preschool LEP Spanish-speaking children for educational planning, (2) to establish validity and reliability of the adapted Spanish criterion-referenced test and two norm-referenced tests to be used by educational diagnosticians for the assessment of language and learning abilities of young LEP Spanish-speaking pupils. Procedures. The test Evaluacion de Languaje y Habilidades Cognoscitivas en Edad Pre-Escolar (ELCEP) was developed as the Spanish adaptation of the Preschool Language-Cognitive Skills Assessment for Curriculum Entry (PLACE), by John Logfren (1978). The first draft of ELCEP was field tested and revised in preparation for validation of six Spanish-English bilingual professionals from different fields of education. A sample of 35 LEP Spanish-speaking children was used to establish the reliability of the ELCEP. A sample of 100 LEP Spanish-speaking children was used to establish concurrent validity of the ELCEP, and the two norm-referenced test, Prueba Illinois de Habilidades Psicolinguisticas (PIHP) and Bateria Woodcock de Proficiencia del Idioma or Woodcock Language Proficiency Battery Spanish (WLPB-Span.) Findings. The level of significance for rejecting the null hypotheses was set at .05. No significant differences were found between male and female scores, or between age group scores from the reliability sample of the ELCEP. Significant correlations and traits of communality were found from comparison of 15 variables from the ELCEP, PHIP, and WLPB-Span, through factor analysis. Recommendation. Additional extension of the ELCEP for ensuring quality of test constructs to permit criterion-referenced and norm-referenced application of the test. Further studies to ensure appropriate development of equivalent English and Spanish forms of the ELCEP, and local norms for the PHIP AND WLPB-Span.

No description available.

Recent efforts to help teachers improve have centered on teacher evaluation. This qualitative dissertation explored how eight teachers from one middle school described and understood improving their practice Additionally, this study used purposeful developmental sampling to explore how, if at all, participants’ way of knowing (meaning their internal cognitive, emotional, interpersonal, and intrapersonal capacities), as assessed by an expert developmental psychologist who employed the Subject-Object Interview (a reliable developmental assessment tool), might help with understanding how they teachers themselves describe and make sense of improving their practice. The participant’s way of knowing influenced their conceptions of improvement, as well as the supports and challenges they encountered. This study has implications for teachers themselves, as well as school districts and district leaders, as they work to improve teacher practice. This study is unique in that it focuses on the experiences and understandings of teachers, all of whom work within one middle school, as they strive to improve their practice. The research was set in a district with a unique teacher evaluation system through which teachers participate in setting evaluation goals and evaluations were not at all tied to standardized test scores. I recruited an expert developmental psychologist to conduct Subject-Object Interviews in order to develop a purposeful sample of eight participants with a developmental range from socializing to self-authoring way of knowing. Eight Subject-Object interviews and 24 in-depth, qualitative interviews (approximately 36 hours, transcribed verbatim) were the primary data source. Data analysis involved several iterative steps, including writing analytic notes and memos; reviewing, coding, categorizing data to identify key themes within and across cases; and crafting narrative summaries. For each of the connected dimensions of understanding improvement conceiving, recognizing, and supporting improvement participants’ way of knowing was intimately tied to how they described how they improved their practice. The key difference between the dominant socializing knowers and the dominant self-authoring knowers emerged that the socializing knowers were subject to external authorities and factors, while the dominant self-authoring knowers relied on their internal values and judgment. In describing their understanding of what it meant to improve, all of the participants described how they sought to improve their practice by deepening their PCK and improving rapport with students (8 of 8), with the dominant socializing knowers relying on external authorities and providing their descriptions from within their own experiences. The dominant self-authoring knowers had strong internal systems from which they evaluated external information to evaluate its relevance to their improvement. Participants also discussed their uncertainty recognizing improvement (8 of 8). For the fully socializing knowers, they were uncertain about their own improvement because of changeable external forces. Those participants who were dominant socializing knowers with full capacity for self-authoring ways of knowing, they expressed uncertainty in themselves, so the source was internal. The dominant self-authoring knowers had their own theories of the inherent uncertainties of measuring improving their practice, yet also described ways that they could gauge improvement. Almost all participants named both observing others (7 of 8) and time to meet with colleagues (7 of 8) as practices that supported improvement. The dominant socializing knowers valued time to observe others and to meet with colleagues as opportunities to take in ideas from external sources to help them improve their practice. In contrast, the dominant self-authoring knowers appreciated time to meet with and to observe colleagues so that they could problem-solve, evaluate ideas, and build community. In sharing their understanding of district initiatives and teacher evaluation plan, how participants described supports and obstacles for their improvement were qualitatively different based on their way of knowing. Some participants described district initiatives as helpful (3 of 8) for their improvement, but that all participants (8 of 8) said that the high volume and short life span of the initiatives created obstacles to their improvement. For the dominant socializing knowers, they described feeling judged and that they had to “keep up” with new initiatives. The dominant self-authoring knowers discussed initiatives as distractions from their self-determined improvement path. Importantly, the dominant socializing knowers in leadership roles expressed increased anxiety in having to represent new initiatives to colleagues. All participants (8 of 8) identified features of the teacher evaluation plan that were helpful for their improvement. For the dominant socializing knowers, they valued the external authority of guiding documents, whereas the dominant self-authoring knowers valued the time to discuss and evaluate their work with their evaluator. Most participants (7 of 8) also described ways in which the teacher evaluation plan created obstacles in their efforts to improve. For the dominant socializing knowers, they were concerned about feeling inadequate in their improvement, while most of the dominant self-authoring knowers expressed that the evaluation plan took away time and focus from how they thought they could best improve their practice. According to my research, teachers benefit from time to meet with and to observe their colleagues as well as transparency as to how to reconcile past and present initiatives. Additionally, to support teachers who are dominantly socializing in their way of knowing, my research shows that they profit from clearly delineated written guidance and affirmative discussion with evaluators.