Despite support for the notion that evaluation utilization is facilitated by stakeholder engagement, evaluation practice in the Government of Canada may not be conducive to such engagement. This mixed-methods study explores the extent to which process use is manifest within government; the conditions, influences and factors that enhance process use; and the relationship between process use and findings use. The study supports the notion that process use and its consequences are important predictors of findings use in government. The level of stakeholder engagement present in this context appears to support the development of knowledge/skills, which may not be sufficient in enhancing findings use; changes in attitude/affect and in actions/behaviours may be of greater importance. This study highlights the importance of timely, higher-level engagement over frequent, surface-level engagement, as well as the importance of organizational learning capacity and conditions mediating evaluation use in setting the stage for process use to occur.

The essays in this thesis attempt to answer the most difficult questions that I have faced as a teacher and consultant for school-based data teams. When we report statistics to our fellow educators, what do we say and what do we leave unsaid? What do averages mean when no student is average? Why do we treat our population of students as infinite when we test for statistical significance? I treat these as important philosophical questions. In the first essay, I use Paul Grice’s philosophical analysis of conversational logic to understand how data teams can accidentally mislead with true statistics, and I use Bernard Williams’s philosophical analysis of truthfulness to understand the value, for data teams, of not misleading with statistics. In short, statistical reports can be misleading when they violate the Gricean maxims of conversation (e.g., “be relevant,” “be orderly”). I argue that, for data teams, adhering to the Gricean maxims is an intrinsic value, alongside Williams’s intrinsic values of Sincerity and Accuracy. I conclude with some recommendations for school-based data teams. In the second essay, I build on Nelson Goodman and Catherine Z. Elgin’s analyses of exemplification to argue that averages (i.e., medians and means) are attenuated, moderate, and sometimes fictive exemplars. As such, medians and means lend themselves to scientific objectivity. In the third essay, I use Goodman’s theory of counterfactuals and Carl Hempel’s theory of explanation to articulate why data teams should make statistical inferences to infinite populations that include possible but not actual students. Data teams are generally concerned that their results are explainable by random chance. Random chance, as an explanation, implies lawlike generalizations, which in turn imply counterfactual claims about possible but not actual subjects. By statistically inferring to an infinite population of students, data teams can evaluate those counterfactual claims in order to assess the plausibility of random chance as an explanation for their findings.

Research has found that, on average, deaf and hard of hearing (DHH) students graduate from high school reading at the fourth grade level (Allen, 1986). Additionally, DHH children of deaf parents (Charrow & Fletcher, 1974) and those with strong American Sign Language (ASL) proficiency (Strong & Prinz, 1997) tend to outperform DHH students without parents who are proficient in ASL. The Simple View of Reading (SVR; Gough & Tunmer, 1986) suggests that reading comprehension is a product of decoding and language proficiency. Many DHH students have limited auditory access, and may struggle to acquire English, especially the more demanding academic English characteristics of school texts (Mayer & Wells, 1996). Academic English has been identified as a strong predictor of reading comprehension among hearing children (Uccelli et al., 2015). Guided by a modified SVR model, in this study I investigate DHH secondary school students’ reading comprehension as predicted by receptive ASL proficiency, word reading fluency/decoding, and academic English proficiency. Guided by prior research on DHH and hearing students, I investigate the hypothesis that for secondary school DHH students enrolled in ASL/English bilingual/bimodal schools for the deaf, academic English proficiency is a significant predictor of reading comprehension alongside ASL proficiency. In this study, a sample of secondary school DHH students were tested in ASL proficiency, academic English proficiency, word reading fluency (a proxy for decoding), and reading comprehension. Using linear regression, an interaction between academic English proficiency and word reading fluency was detected, such that the lower the level of academic English proficiency, the higher the impact of word reading fluency on reading comprehension. ASL skills predicted reading comprehension across all models. Findings support a model in which socio-demographic factors, ASL proficiency, and word reading fluency are predictors of reading comprehension for secondary DHH students. This study is innovative in assessing three sets of language and reading skills essential for DHH students’ reading comprehension. The continued adaptation of instruments that target these constructs, as well as studies with larger samples, are critical to further explore the innovative theoretical model of reading comprehension for DHH students proposed in this study. / Human Development and Education

European nations, built on democratic foundations, rely on participation that is inclusive of all groups. Among efforts to support social cohesion in Europe, investigating the development of intercultural attitudes—attitudes toward others on the basis of their intersecting group memberships—is a crucial area of research. Further, examining attitudes among adolescents is useful because of their growing capacity to understand complex systems, while still being engaged in formal education in which interventions aimed at developing positive attitudes are often applied. In this dissertation, I used data from the 2009 IEA ICCS (International Civic and Citizenship Education Study) to examine determinants of intercultural attitudes among adolescents (n=16,847) in seven countries across Europe—the United Kingdom (England only), Sweden, Switzerland, Spain, Bulgaria, Poland, and Greece—focusing on cultural contexts and school climates. First, I established the measurement invariance of a variety of measures of intercultural attitudes to support the validity of cross-cultural comparison, using a novel approach in the Bayesian framework. Using this method, I was able to validly compare intercultural attitudes across contexts in Europe, and these findings are interpreted and discussed. Then, I examined the ways in which intergroup contact, gender, and school climates were associated with intergroup attitudes across these seven countries. I found limited evidence of an association between native-born and immigrant contact and positive intercultural attitudes. However, I found that positive intercultural attitudes were consistently associated with positive and democratic school climates, as well as with gender and attitudes toward gender equality. In this dissertation, I present these findings, as well as provide interpretation, discussion, and future directions with regard to educational interventions. / Culture, Communities, and Education

Leaders of education organizations must determine how to positively impact students, staff, and families while ensuring organizational strength. Innovative leaders and entrepreneurs seek to develop and roll out new ideas to dramatically advance students’ achievement. However, the need to innovate does not supersede the need to execute on a core business while maintaining high levels of quality; this paradox makes it challenging to know how to allocate resources and people toward innovation. In this capstone, I describe and analyze my experiences leading a strategic project for supporting innovation and experimentation at Panorama Education, an education technology company focused on school improvement through data analytics. I developed a framework for promoting innovation, workshopping, and experimentation in order to support the organization to make evidence-driven product decisions. I describe the theoretical foundations for the framework and literature documenting the leadership challenge of implementing it at a fast-paced, rapidly growing organization. I argue that by deliberately allocating resources and effort toward ideating and designing, aligning to company strategy and vision, and prototyping and testing, a growing organization can prepare for ongoing innovation and adaptability. I also find that change management on the part of the implementer is a necessary part of innovative change, so I reflect on my successes and challenges as a leader developing and implementing the new system at Panorama Education.

Recent innovations in teacher evaluation include the development of standards-based frameworks for observational evaluations of teacher practice and development of test-based measures of teacher effectiveness. These tools raise questions about how best to identify good teaching, about the roles of both evaluations and principals in improving teaching, and about the relationship between school context and teacher evaluation. This qualitative study examines principals’ understanding and use of a test-based ranking of teacher quality—the Academic Growth over Time (AGT) ranking—and a standards-based observation framework—the Teaching and Learning Framework (TLF)—in the Los Angeles Unified School District. Findings suggest principals believed outcomes matter, but harbored concerns about what AGT rankings actually measured. In addition, principals felt AGT rankings provided little information about how to improve teaching. Some were concerned that in low-scoring schools, the AGT rankings might reinforce test-focused instruction in ways that disadvantaged students. Principals preferred the TLF because it was more comprehensive than previous observation tools, helped teachers evaluate their practice against clearly defined expectations, and helped principals improve instruction. Though principals used the TLF to assign teachers ratings, they primarily described it as a tool to improve the quality of teaching in their buildings. Principals noted that the TLF process placed heavy demands on the time of teachers and principals, and expressed concern about how they could sustain high-quality implementation of the TLF when they had to use it at scale. Most principals noted cases where the AGT rankings indicated a different reality about a teacher’s effectiveness than what the principals expected, based on classroom observations. How principals reconciled discrepancies varied. Principals in higher-scoring schools were more likely to discount test-based measures and more likely to emphasize the professional capability of their staff as an asset. The findings suggest principals’ use of evaluation tools is mediated by their confidence in the tools and their own leadership, by how useful they feel the tools are, and by their own school contexts. In addition, this study suggests that the unique burdens of persistent socioeconomic and racial segregation may hamper local efforts to implement promising practices at some sites.

Abstract not available.

No description available.

Master of Science / Curriculum and Instruction Programs / Kay A. Taylor / This historical and qualitative inquiry investigates recent educational reform in Ukraine. On Tuesday, April 22, 2008 more than half a million Ukrainian high-school graduates were ordered to take an external assessment of student achievements. The new assessment model was named the External Independent Knowledge Testing and replaced the traditional forms of high school exit-entrance to higher educational institution exams in Ukraine. These changes in assessment strategies in the Ukrainian educational system illustrate the global trend of replacing diverse forms of national examinations with standardize multiple-choice assessments that can be scored by machines and could be viewed as a new page in the history of educational assessment. Educational assessment has become one of the most significant areas of research in the United States. It is not only a prominent issue in education today but also one of the most controversial issues in contemporary educational science. The debate is concentrated around the question of what purpose educational assessment serves. A growing body of international research suggests that educational assessment is a new mechanism of social and political control aimed to legitimize social inequalities. Thus, the primary goal of this study was to investigate what kind of changes the External Independent Knowledge Testing in Ukraine was expected to bring to Ukrainian society and whether the initial results of the reform were consistent with what was expected. Data was collected by means of interview, survey, and examining Ukrainian publications. The data revealed that the recent changes in traditional practices of assessment in Ukraine were aimed at serving much broader social purposes than those identified by Ukrainian policymakers. Under the guise of improving the quality of Ukrainian education and fighting corruption in higher education, the educational reforms in Ukraine are implementing an undetected new form of social and political preferences through education and testing. Thus, the reform helps new Ukrainian nobilities legitimize their status through the new system of exercising control over Ukrainian education.

The purpose of this study was to evaluate current practice of using profile analysis of the WISC-III in the diagnosis of processing deficit in earning disabled students. The types of processing deficit prescribed by school psychologists were examined. The validity of the profile analysis approach in processing deficit diagnosis was investigated. Statistical profile analysis, Cattel's r(p(k)) coefficient, and cluster analysis were used to determine whether the mean WISC-III subtest profiles differ among different processing deficit subgroups, and to determine whether individuals diagnosed with the same processing deficit have similar WISC-III profiles. This study was conducted on a sample of 134 non-Hispanic White students identified as learning disabled in a school district in the West Coast area. The results found numerous types of processing deficit, which were not limited to the eight types defined by the school district. About 50% of the individuals were identified with just one processing deficit, while the other half had two or more processing deficits in combination. In addition, the most frequently diagnosed processing deficits were deficits in auditory processing, visual processing and sensory-motor integration. The mean group profiles of the two largest groups with Auditory Processing Deficit (APD) and Visual Processing Deficit (VPD) were compared using the statistical profile analysis technique. The results showed that the mean profiles of the two groups are not parallel. Based on Cattel's r(p(k)) coefficient, both APD and VPD individuals resemble their own mean group profile more than the other group. However, only 52.5% of individuals matched the mean profile of the group in which they were diagnosed. A similar finding was found in cluster analysis. Only 50% of individuals could be correctly classified into their processing deficit groups based on profile similarities. These findings suggested that only half of the individuals diagnosed with the same processing deficit have similar WISC-III subtest profiles. Caution should be exercised in using the diagnostic information and the use of profile analysis technique in making processing deficit diagnosis. Future studies should be conducted to determine the validity of the various processing deficit diagnoses, and how they are related to the WISC-III subtests.