School leaders in the United States live in an educational era characterized by a desire for and expectation that all students attain high levels of academic proficiency. There is an increased reliance on all types of educational assessment as a key component to help school leaders attain that goal. The purpose of this study is to understand how school administrators can foster a balanced system of assessment at the local level to genuinely harness the power of assessment to enhance student learning. The significance of the study rests in the fact that there is a general failure of states and school districts to conceive of educational assessment as a system that operates at all levels of the educational system from the classroom up to the district and state level. The findings of this study support the efforts of a group of administrators to develop a balanced system of math assessments in their school district.

abstract: Over the past 20 years in the United States (U.S.), teachers have seen a marked shift in how teacher evaluation policies govern the evaluation of their performance. Spurred by federal mandates, teachers have been increasingly held accountable for their students’ academic achievement, most notably through the use of value-added models (VAMs)—a statistically complex tool that aims to isolate and then quantify the effect of teachers on their students’ achievement. This increased focus on accountability ultimately resulted in numerous lawsuits across the U.S. where teachers protested what they felt were unfair evaluations informed by invalid, unreliable, and biased measures—most notably VAMs. While New Mexico’s teacher evaluation system was labeled as a “gold standard” due to its purported ability to objectively and accurately differentiate between effective and ineffective teachers, in 2015, teachers filed suit contesting the fairness and accuracy of their evaluations. Amrein-Beardsley and Geiger’s (revise and resubmit) initial analyses of the state’s teacher evaluation data revealed that the four individual measures comprising teachers’ overall evaluation scores showed evidence of bias, and specifically, teachers who taught in schools with different student body compositions (e.g., special education students, poorer students, gifted students) had significantly different scores than their peers. The purpose of this study was to expand upon these prior analyses by investigating whether those conclusions still held true when controlling for a variety of confounding factors at the school, class, and teacher levels, as such covariates were not included in prior analyses. Results from multiple linear regression analyses indicated that, overall, the measures used to inform New Mexico teachers’ overall evaluation scores still showed evidence of bias by school-level student demographic factors, with VAMs potentially being the most susceptible and classroom observations being the least. This study is especially unique given the juxtaposition of such a highly touted evaluation system also being one where teachers contested its constitutionality. Study findings are important for all education stakeholders to consider, especially as teacher evaluation systems and related policies continue to be transformed. / Dissertation/Thesis / Doctoral Dissertation Educational Policy and Evaluation 2020

In the past decade, many American high school graduates have been deprived of the cultural vocabulary that was once the fundamental and common possession of educated persons in past generations. This dearth of knowledge--also described as a lack of cultural literacy is a glaring problem, reflected quite noticeably in declining scores on standardized tests. This national paucity in cultural literacy has not left the Roosevelt Junior-Senior High School (in Roosevelt, New York) unscathed; therefore, this study assessed the needs of the students and devised strategies to meet those needs. In 1991, five workshops were conducted for teachers in the English/language arts department. Because the Roosevelt Junior-Senior High School, which comprises a predominantly Black, middle-class student population, has not escaped this national malady, this study utilized action research to improve teaching techniques and enhance cultural literacy. Using African-American culture as a vehicle, an effort was made to forge a link between Black culture (emphasizing the literary aspect) and other required materials. The main objective was, and still is to motivate students to learn, thereby improving their academic status and raising their test scores. In every field of endeavor--be it educational, or otherwise, it would benefit the leaders or staff members to make attempts to meet the students, workers, or members, where they are on their various intellectual and inspirational levels, tapping into their latent talents. With emphasis on collaboration and an action research approach, an effort was made to implement measures for solving the stated problem. Academically, this study offered students the opportunity to use their cultural heritage, their interests, and their talents as a catalyst to general "world knowledge." This rise in levels of cultural literacy may lead to greater feelings of self-worth, and eventual improvement in test scores on teacher-made tests as well as standardized tests.

In this study, the associations of socioeconomic variables with reading test scores in grade 4 (PIRLS) and with math and science test scores in grades 4 and 8 (TIMSS) were examined across 28 OECD countries. This study adds to the current knowledge base by integrating measures of income inequality, which have been used in a few studies involving test scores, with other socioeconomic variables of interest. Bivariate correlations show that certain socioeconomic measures have stronger relationships with test score inequality than with average test scores: income inequality, gender inequality, and adolescent fertility rates all have significant relationships with test score inequality in reading, math, and science. There are also strong intercorrelations among these three socioeconomic variables. Income inequality is significantly associated with average science test scores in grades 4 and 8, while adolescent fertility rates hold significant relationships with average math and science test scores in both grades. Intercorrelations among the variables show that people who live in a country with high income inequality, and are at the lower end of that country’s income distribution, struggle in ways that people in countries with low income inequality do not. Health insurance and access to health care, paid maternity leave, and preschool education are easier to obtain in countries with lower income inequality. Examination of individual countries gives additional insight into the important role of income inequality. Slovenia, for example, has a relatively low GDP per capita but enjoys good test scores, perhaps because of its low income inequality. Finland, another country with low income inequality, attains PIRLS and TIMSS test scores that are among the highest in the world, in part because of social services that have the effect of further reducing income inequality. The U.S. is something of a puzzle because it has relatively high scores despite substantial inequality. Based on the results of this study and other current research, it seems likely that the U.S. could reduce test score inequality by providing targeted supports to low-income families, effectively reducing income inequality. Such supports may include: 1) installing wraparound services within school settings for low-income families; and 2) substantially expanding preschool access, especially among 3-year-olds.

The 2001 reauthorization of the Elementary and Secondary Education Act (ESEA), No Child Left Behind, brought the use of student assessment data to the fore. This mandate-based and sanction-laden legislation ushered in a new era of accountability rooted in the collection, analysis, and use of student assessment data for educational improvements. As a result, a new boon industry has emerged around formative assessment products. This study begins by exploring the policies that have ushered in an assessment-driven accountability era. Then, the empirical and conceptual literature around the implementation and use of formative assessment systems is summarized. While there is promising literature on the effectiveness of formative assessment products, understanding what fosters or inhibits the use of assessment data for school teachers‘ needs further exploration. The purpose of this study is to understand if and how teachers are using formative assessment products or systems to impact their pedagogical decision-making in classrooms. This study will also focus on how school and district leadership teams choose or support the formative assessment system. In essence, this study seeks to investigate the fidelity of formative assessment products the actual impact used by schoolteachers; and more importantly, how data is transformed into information and ultimately knowledge. Research Questions. The research questions from this study are anchored in elements of fit, readiness, coherence, and use. Specifically, the intent of this research is to understand: (1) The district's intended purpose and the formative assessment system purpose of validity ( Fit); (2) The capacity of teachers' to use formative assessment data to inform their pedagogical decision-making (Readiness); (3) The school district (and individual schools') ability to create, support, and resources the implementation and use of formative assessment systems ( Coherence); And, (4) The actual use of formative assessment data by school teachers' in regard to pedagogical decision-making. Consequently, the study's research questions are as follows: (1) Are teachers and district/school leaders using the formative assessment data to inform educative decision-making and pedagogy? (a). How are teachers using these data to modify instruction/practice? (b) How are school leaders using these data for programmatic and in-service decisions? (2) Are the formative assessment products valid for their intended purposes? (a) Are the assessments used by the district valid for evaluating and improving instruction? (b) Are the assessments used by the district valid for measuring student progress? (c) Are the assessment used by the district valid for modifying teaching practices? (3) Are teachers ready and able to use formative assessment data in meaningful and effective ways? (a) How has the school district supported and resource the use of the formative assessment system? (b) How are teachers using these data to modify instruction/practice? Research Design. To answer the research questions, the research implements a three-phase design. To begin, phase one involves the identification of school districts in the state of Florida that have implemented a formative assessment system. These systems are examined to ascertain which are truly student-level diagnostic instruments (the Black and Wiliam (1998), definition of formative assessment was used to guide this analysis). Phase two documents why and how the formative assessment system was implemented into the district. Finally, phase three examines the actual use of the formative assessment system by teachers in the district.

There were two goals of this study. The first was to continue reliability and validity investigations of the Body-Karlson Adolescent Development Inventory (BKADI). The second was to compare college students with a learning disability to a control group of college students focussing on adolescent development issues. The study used a matched sample procedure of 40 students diagnosed with a learning disability and 40 students without a learning disability diagnosis at the University of Massachusetts-Amherst. The students were administered the BKADI, The College Adjustment Scales (CAS), and the Seligman Learned Optimism Inventory (SLOT). The BKADI assesses seven developmental concerns: Body Image, Autonomy Within the Family, Life in School, Interpersonal Relationships, Career and Lifestyle, Gender Identity, and Comfort with Changing Cultural Values while the CAS contains nine clinical scales including Anxiety, Depression, Suicidal Ideation, Substance Abuse, Self-esteem Problems, Interpersonal Problems, Family Problems, Academic Problems, and Career Problems. The SLOT assesses overall perceptions the person has about good and bad events happening in their life, as well as, a hopelessness rating, and a self-esteem rating. This study reveals a strong reliability for the BKADI for both LD and non-LD groups. Also concurrent validity was found. Comparative results of the BKADI indicated that LD students scored significantly less on the Life in School and Career Lifestyle scales. Also, results of the comparative study from the CAS showed LD students scored significantly more problematic on the scales of Academic Problems, Career Problems, Self-esteem, Family Problems, Anxiety, and Depression. Total percentages of LD students falling within the problematic ranges are significant in the previous scales, as well as, Suicidal Ideation and Substance Abuse. On the SLOT, results indicated that LD and non-LD students explained good and bad events in their life similarly. LD students, however, were significantly more optimistic on the Personalization Good scale. Conclusions related to previous research and implications for policy and practice are included.

The purpose of this study was to investigate the degree to which preschool children are able to provide reliable and valid sociometric data pertaining to their classroom peers. The applicability of a sociometric picture rating scale was assessed by examining its concurrent validity using the Kohn Social Competence Scale, a teacher rating instrument designed to provide an indication of the classroom social behavior of preschool children based on teacher observation. Reliability of the sociometric rating scale was investigated through a test-retest procedure over a six-week interval. Subject were thirty-two 3 and 4 year olds (16 male; 16 female) attending public school preschool programs in a rural community in Western Massachusetts. The data obtained included two sets of sociometric scores for each subject (test, retest) as well as scores from the rating scales teachers completed for each subject. Analysis of the data involved investigation of the strength of association between teacher rating scale scores and sociometric scores, as well as stability of the sociometric scores over the six-week test-retest interval. Additionally, analyses were undertaken to determine if there were statistically significant differences in the sociometric scores due to a subject's age, gender, prior socialization experiences, or Special Education status. Results indicated fairly good reliability of the sociometric picture rating scale (correlation between the test and retest sets of scores =.71, p $<$.01). Significant relationships were also found between the sociometric scores and those obtained from the teacher rating scale. Therefore, concurrent validity of the sociometric measure was demonstrated. The only differences of statistical significance found in the sociometric scores were those related to a subject's gender. In this study, females were more likely to receive high sociometric ratings from peers and males were more likely to receive low ratings. Based on the findings, it was concluded that the picture rating scale technique represents a valid and reliable sociometric measure for preschoolers. It was suggested that sociometrics always be used in conjunction with other measures of social functioning, namely direct observations of behavior or teacher rating scales.

To meet the affective and cognitive needs of adolescents and to choose appropriate teaching strategies it is necessary for teachers to be able to diagnose these needs. This feasibility study investigates the Developmental Assessment Paradigm (DA) as a tool to enable teachers to diagnose their students' cognitive and affective levels. The paradigm is based on the model of Developmental Assessment created by Ivey (1986) and researched by Rigazio-DiGilio (1989), Bradford Ivey (1990) and Gonclaves (1988). Ivey's research demonstrated that counselors could learn to use DA effectively. A Pilot Study (Brodhead, 1988) found that it was possible, using DA, to identify different levels of cognitive and affective development within the dialogue of a sample of high school classrooms. This feasibility study evolved from these previous studies. The dissertation discusses the need for the diagnosis of developmental levels, it introduces developmental assessment as a diagnostic tool and describes the results of the pilot study and the design and results of the feasibility study. The review of the literature in Chapter II supports both the need for teachers to be able to diagnose their students' developmental levels and the potential of this diagnostic ability to enable teachers to choose effective and appropriate teaching techniques. To this end, the review develops a working definition of "effective schools" and "effective teachers", and examines paradigms that identify and define interpersonal skills. A small group of teachers participated in two training sessions, one week apart. Data from pre- and post-tests showed significant improvement in the participants' ability to recognize DA levels in sample statements of high school students' dialogue. On their evaluation forms, the participants expressed their belief in the value of DA as a diagnostic technique and suggested strategies for implementation in their classrooms. Discussion during the sessions indicated they were able to identify DA levels using transcribed dialogue from their own classrooms. The results of this study indicated that these teachers were able to significantly improve their ability to recognize levels of DA in samples of students' dialogue and in dialogue from their own classrooms. Further study with a larger sample was recommended.

Three studies were conducted using Q-Sort methodology to investigate: (1) the relationship between counselor's level of experience and the conceptual systems they utilize in understanding supervision discourse, (2) the construct validity of the Cognitive Developmental Model, and (3) the feasibility of using open Q-Sort methodology to study qualitative differences in the complexity of conceptual systems. Results suggest there is a link between counselor's conceptual systems and their level of experience, the CDM has construct validity and Q-Sort methodology is an effective device when used to evaluate qualitative differences in the complexity of conceptual systems.

Item response theory has improved the area of educational and psychological measurement significantly. However, the effectiveness of the applications of item response theory is dependent on the adequacy of techniques of parameter estimation. When item parameters are precalibrated and treated as known, the ability estimation is relatively straightforward. Currently, several competing estimators of the ability parameters in item response models are available. These are: Maximum Likelihood estimator (ML), the Bayesian Modal estimator (BM), the Expected A Posterior estimator (EAP), and the Mean of the likelihood function (abbreviated as "MM" to differentiate from ML, above). The primary purpose of the study was to examine and compare the properties of the above ability estimators when item parameters were precalibrated and treated as known. In particular, the properties of the ability estimators, such as distribution, bias, and accuracy were investigated. The secondary purpose of this study was to investigate the asymptotic properties of the ML ability estimator with respect to accuracy, bias, and the asymptotic normal distribution. In addition, the effects of test lengths and ability levels were studied in the three-parameter item response model. Simulated data were generated under various test lengths and ability levels in the three-parameter models. In order to accomplish the purpose of this study analyses such as: (1) accuracy of the ability estimators; (2) bias of the ability estimators; (3) distributional property of the ability estimators; and (4) the asymptotic properties of the ML ability estimators were carried out. The results of this study indicate that the ML ability estimator tends to be better than the MM, BM, and EAP ability estimators in the three-parameter item response model. This is particularly true in the proficiency test data set based on the three-parameter item response model. In general, the ML, BM, MM, and EAP ability estimators are normally distributed except when the true ability levels are at both of the extremes and tests are short (n $\le$ 40). The ML ability estimator is asymptotically normally distributed with tests longer than 20 items and when true ability is in the range ($-$1, 1).