A fundamental issue in educational measurement is what frame of reference to use when interpreting students’ performance on an assessment. One frame of reference that is often used to enhance interpretations of test scores is normative, which adds meaning to test score interpretations by indicating the rank of an individual’s score within a distribution of test scores of a well-defined reference group. One of the most commonly used frames of reference on student achievement provided by test publishers of large-scale assessments is national norms, whereby students’ test scores are referenced to a distribution of scores of a nationally representative sample. A national probability sample can fail to fully represent the population because of student and school nonparticipation. In practice, this is remedied by weighting the sample so that it better represents the intended reference population. The focus of this study was on weighting and determining the extent to which weighting grade 4 and grade 8 student records that are not fully representative of the nation can recover distributions of reading and math scores in a national probability sample. Data from a statewide testing program were used to create six grade 4 and grade 8 datasets, each varying in its degree of representativeness of the nation, as well as in the proximity of its reading and math distributions to those of a national sample. The six datasets created for each grade were separately weighted to different population totals in two different weighting conditions using four different bivariate stratification designs. The weighted distributions were then smoothed and compared to smoothed distributions of the national sample in terms of descriptive statistics, maximum absolute differences between the relative cumulative frequency distributions, and chi-square effect sizes. The impact of using percentile ranks developed from the state data was also investigated. By and large, the smoothed distributions of the weighted datasets were able to recover the national distribution in each content area, grade, and weighting condition. Weighting the datasets to the nation was effective in making the state test score distributions more similar to the national distributions. Moreover, the stratification design that defined weighting cells by the joint distribution of median household income and ethnic composition of the school consistently produced desirable results for the six datasets used in each grade. Log-linear smoothing using a polynomial of degree 4 was effective in making the weighted distributions even more similar to those in the national sample. Investigation of the impact of using the percentile ranks derived from the state datasets revealed that the percentile ranks of the distributions that were most similar to the national distributions resulted in a high percentage of agreement when classifying student performance based on raw scores associated with the same percentile rank in each dataset. The utility of having a national frame of reference on student achievement, and the efficacy of estimating such a frame of reference from existing data are also discussed.

Many accountability systems use data from large-scale assessments to make judgements about school performance. In Ontario, school performance is often assessed using the percentage of proficient students (PPS). The purpose of this study was to shed light on the degree and frequency of changes from year to year in the percentage of proficient students, at a school, in the areas of reading, writing and mathematics for both grades 3 and 6 in Ontario from 2006 to 2010. A second purpose was to assess the influence of cohort size on the variability in scores from year to year. Once schools not having data for 5 consecutive years and outliers were omitted secondary data analysis was used to examine nearly 3000 schools in each subject and grade. For the first part of the study, descriptive statistics and frequencies were the main method of examination. In the second part of the study, variance scores and correlations were used in order to understand the relationship between changes in PPS and cohort size. Findings revealed that changes in school scores from year to year are very large for many schools. Approximately 50 percent of schools experienced changes in PPS greater than 10 percent in any given year. When examining how often, from 2006 to 2010, a school experienced a similar amount of change – generally, both the smallest and largest change categories had a larger percentage of schools experiencing a similar amount of change for two and three years. Very seldom did schools experience the same degree of change in PPS across all 5 years. Results from correlations revealed a significant and inverse relationship between average cohort size and variability in PPS. Considering over 80 percent of schools have 60 or fewer students in a cohort the unpredictability in PPS may prove to be quite frustrating to schools and confusing to stakeholders. Annual PPS scores appear to be a poor indicator of real school performance, and their use to rank or rate schools should be avoided. Recommendations are made about using PPS to report school level results for EQAO, schools and the public.

Many accountability systems use data from large-scale assessments to make judgements about school performance. In Ontario, school performance is often assessed using the percentage of proficient students (PPS). The purpose of this study was to shed light on the degree and frequency of changes from year to year in the percentage of proficient students, at a school, in the areas of reading, writing and mathematics for both grades 3 and 6 in Ontario from 2006 to 2010. A second purpose was to assess the influence of cohort size on the variability in scores from year to year. Once schools not having data for 5 consecutive years and outliers were omitted secondary data analysis was used to examine nearly 3000 schools in each subject and grade. For the first part of the study, descriptive statistics and frequencies were the main method of examination. In the second part of the study, variance scores and correlations were used in order to understand the relationship between changes in PPS and cohort size. Findings revealed that changes in school scores from year to year are very large for many schools. Approximately 50 percent of schools experienced changes in PPS greater than 10 percent in any given year. When examining how often, from 2006 to 2010, a school experienced a similar amount of change – generally, both the smallest and largest change categories had a larger percentage of schools experiencing a similar amount of change for two and three years. Very seldom did schools experience the same degree of change in PPS across all 5 years. Results from correlations revealed a significant and inverse relationship between average cohort size and variability in PPS. Considering over 80 percent of schools have 60 or fewer students in a cohort the unpredictability in PPS may prove to be quite frustrating to schools and confusing to stakeholders. Annual PPS scores appear to be a poor indicator of real school performance, and their use to rank or rate schools should be avoided. Recommendations are made about using PPS to report school level results for EQAO, schools and the public.

This qualitative study examines the connections between teachers’ views of large-scale assessment and their classroom practice, based on a case study of Ontario teachers and the Ontario Education Quality and Accountability Office [EQAO] Grade 9 Applied Assessment of Mathematics. Large-scale assessments are a prominent aspect of the Ontario education system; given that they are also mandatory, it is imperative that their impact on teaching be documented. This study enriches the existing literature on the topic of the Grade 9 EQAO Assessment of Mathematics, and provides a more-recent portrayal of the teachers’ views of this assessment and the potential impact of these views on classroom teaching by highlighting teacher attitudes, concerns, and classroom practices.

Esta dissertação tem como problema de pesquisa: quais são as representações de professores e de alunos sobre a Provinha Brasil? Objetiva: (1) identificar e problematizar as representações – significados, modos de operacionalização e usos – sobre a Provinha Brasil a partir das falas de professores de três turmas de 2º ano do Ensino Fundamental e (2) inventariar e analisar o que representa ser avaliado pela Provinha Brasil na perspectiva dos alunos. A Provinha Brasil é uma avaliação em larga escala com função diagnóstica, aplicada no início e no final do 2º ano do Ensino Fundamental, visando aferir os progressos no rendimento dos alunos (em leitura e matemática) e dar subsídio ao planejamento dos professores. A pesquisa foi realizada em três escolas públicas de âmbito federal, estadual e municipal. Se enquadra em uma abordagem de pesquisa qualitativa, utilizando as seguintes ferramentas metodológicas: entrevista semiestruturada, observação e “aula-conversa”, esta última ferramenta criada para produção dos dados junto aos alunos. Os conceitos-ferramentas utilizados na investigação foram: Representação cultural (HALL, 1997) e Governamento (FOUCAULT, 1982). As análises mostraram que: os significados que as professoras atribuem à Provinha Brasil envolvem enxergá-la como um parâmetro, a partir da legitimação dos conhecimentos e habilidades abordados pelo exame, por serem similares aos que trabalham em suas salas de aula, mas que não têm fortes impactos nas suas práticas, sendo um reforço ao que elas já vinham observando sobre seus alunos. Os modos de operacionalização mostram que as professoras tentam tornar o momento de aplicação tranquilo, para que os alunos consigam fazer a prova e para que esse jeito de conduzir o processo, motivando os alunos, tenha impacto nos desempenhos. As professoras se põem contrárias a focar suas aulas na preparação dos alunos para “irem bem” na avaliação, mas os preparam, fazendo questões de múltipla escolha esporadicamente, para dar conta dos rituais de exame. Quanto aos usos, as professoras tentam “fazer desse limão uma limonada”, usando a Provinha Brasil dentro de sua potência, que é dar uma visão geral do desempenho da turma. Elas também apontam que interpretam os resultados, às vezes categorizando os alunos nos níveis que a avaliação propõe, e pensam ações para dar conta das aprendizagens não consolidadas pelos alunos. Os alunos representaram a avaliação como uma novidade bem aceita e trouxeram à tona significações partilhadas culturalmente sobre “prova”. Mesmo em menor número, alguns alunos também representaram a avaliação como um lugar de desconforto, a partir da insegurança, do nervosismo e das dúvidas, fruto de uma relação recém iniciada com o instrumento “prova”. Também relacionaram a avaliação à autorresponsabilização, trazendo outro significado partilhado culturalmente, uma vez que produzem uma ligação entre desempenho e futuro. / This dissertation has as research problem: what are the representations from teachers and students about the Provinha Brasil? It aims to: (1) identify and problematize the representation – meanings, ways of operationalization and the uses – about the Provinha Brasil based on the talks of three teachers from 2nd grade of elementary school and (2) listing and analyze what represents being evaluated by the Provinha Brasil on the students’ perspective. The Provinha Brasil is a large-scale assessment with diagnostic function that is applied at the beginning and at the end of 2nd grade of elementary school, aiming to assess the progress in student achievement (in reading and mathematics) and to contribute to teachers’ lesson plan. The research was realized in three public schools from different spheres, federal, state and municipal. It is a qualitative research that uses as methodological resources: the semi structured interview, the observations and “class-talk”, this last one created to produce data with the students. The main concepts that were used in this investigation were Cultural Representation (HALL, 1997) and Government (FOUCALT, 1982). The analyses revealed that: the meanings that teachers attribute to the Provinha Brasil involve consider it as a parameter, based on the legitimation of knowledge and abilities that are approached by the exam, because they are similar to the ones worked in their classrooms, but they do not have a strong impact in their practices, being considered a reinforcement to the things they have already observed about their students. The ways of operationalizing shows that the teachers try to make it a calm moment, in order to the students can answer the text and so that, this way of conducting the process, motivating the students, may cause an impact on their performance. The teachers do not put the focus of their classes on the preparation of their students to “do it well” in the evaluation, but they prepare them, proposing questions of multiple choice, occasionally, so the ritual of the exam is practiced. Concerning the uses, the teachers try to “make a lemonade out of lemons”, using the Provinha Brasil in its force, that is to give a general view from the group performance. They also point out that they interpret the results, sometimes categorizing the students in the levels proposed by the evaluation and planning actions to work on these nonconsolidated learning aspects of the students. The students represent the evaluation as a novelty well accepted and bring out meanings culturally shared concerning the meaning of “test”. Even though being in a small number, some students also represent the evaluation as a uncomfortable place, due to the insecurity, the jitters and the doubts, that come from a relation that began recently with the“test” as an instrument of assessment. They also associate evaluation and self-responsability, bringing out another meaning culturally shared, since they produce a connection between performance and future.

Esta dissertação tem como problema de pesquisa: quais são as representações de professores e de alunos sobre a Provinha Brasil? Objetiva: (1) identificar e problematizar as representações – significados, modos de operacionalização e usos – sobre a Provinha Brasil a partir das falas de professores de três turmas de 2º ano do Ensino Fundamental e (2) inventariar e analisar o que representa ser avaliado pela Provinha Brasil na perspectiva dos alunos. A Provinha Brasil é uma avaliação em larga escala com função diagnóstica, aplicada no início e no final do 2º ano do Ensino Fundamental, visando aferir os progressos no rendimento dos alunos (em leitura e matemática) e dar subsídio ao planejamento dos professores. A pesquisa foi realizada em três escolas públicas de âmbito federal, estadual e municipal. Se enquadra em uma abordagem de pesquisa qualitativa, utilizando as seguintes ferramentas metodológicas: entrevista semiestruturada, observação e “aula-conversa”, esta última ferramenta criada para produção dos dados junto aos alunos. Os conceitos-ferramentas utilizados na investigação foram: Representação cultural (HALL, 1997) e Governamento (FOUCAULT, 1982). As análises mostraram que: os significados que as professoras atribuem à Provinha Brasil envolvem enxergá-la como um parâmetro, a partir da legitimação dos conhecimentos e habilidades abordados pelo exame, por serem similares aos que trabalham em suas salas de aula, mas que não têm fortes impactos nas suas práticas, sendo um reforço ao que elas já vinham observando sobre seus alunos. Os modos de operacionalização mostram que as professoras tentam tornar o momento de aplicação tranquilo, para que os alunos consigam fazer a prova e para que esse jeito de conduzir o processo, motivando os alunos, tenha impacto nos desempenhos. As professoras se põem contrárias a focar suas aulas na preparação dos alunos para “irem bem” na avaliação, mas os preparam, fazendo questões de múltipla escolha esporadicamente, para dar conta dos rituais de exame. Quanto aos usos, as professoras tentam “fazer desse limão uma limonada”, usando a Provinha Brasil dentro de sua potência, que é dar uma visão geral do desempenho da turma. Elas também apontam que interpretam os resultados, às vezes categorizando os alunos nos níveis que a avaliação propõe, e pensam ações para dar conta das aprendizagens não consolidadas pelos alunos. Os alunos representaram a avaliação como uma novidade bem aceita e trouxeram à tona significações partilhadas culturalmente sobre “prova”. Mesmo em menor número, alguns alunos também representaram a avaliação como um lugar de desconforto, a partir da insegurança, do nervosismo e das dúvidas, fruto de uma relação recém iniciada com o instrumento “prova”. Também relacionaram a avaliação à autorresponsabilização, trazendo outro significado partilhado culturalmente, uma vez que produzem uma ligação entre desempenho e futuro. / This dissertation has as research problem: what are the representations from teachers and students about the Provinha Brasil? It aims to: (1) identify and problematize the representation – meanings, ways of operationalization and the uses – about the Provinha Brasil based on the talks of three teachers from 2nd grade of elementary school and (2) listing and analyze what represents being evaluated by the Provinha Brasil on the students’ perspective. The Provinha Brasil is a large-scale assessment with diagnostic function that is applied at the beginning and at the end of 2nd grade of elementary school, aiming to assess the progress in student achievement (in reading and mathematics) and to contribute to teachers’ lesson plan. The research was realized in three public schools from different spheres, federal, state and municipal. It is a qualitative research that uses as methodological resources: the semi structured interview, the observations and “class-talk”, this last one created to produce data with the students. The main concepts that were used in this investigation were Cultural Representation (HALL, 1997) and Government (FOUCALT, 1982). The analyses revealed that: the meanings that teachers attribute to the Provinha Brasil involve consider it as a parameter, based on the legitimation of knowledge and abilities that are approached by the exam, because they are similar to the ones worked in their classrooms, but they do not have a strong impact in their practices, being considered a reinforcement to the things they have already observed about their students. The ways of operationalizing shows that the teachers try to make it a calm moment, in order to the students can answer the text and so that, this way of conducting the process, motivating the students, may cause an impact on their performance. The teachers do not put the focus of their classes on the preparation of their students to “do it well” in the evaluation, but they prepare them, proposing questions of multiple choice, occasionally, so the ritual of the exam is practiced. Concerning the uses, the teachers try to “make a lemonade out of lemons”, using the Provinha Brasil in its force, that is to give a general view from the group performance. They also point out that they interpret the results, sometimes categorizing the students in the levels proposed by the evaluation and planning actions to work on these nonconsolidated learning aspects of the students. The students represent the evaluation as a novelty well accepted and bring out meanings culturally shared concerning the meaning of “test”. Even though being in a small number, some students also represent the evaluation as a uncomfortable place, due to the insecurity, the jitters and the doubts, that come from a relation that began recently with the“test” as an instrument of assessment. They also associate evaluation and self-responsability, bringing out another meaning culturally shared, since they produce a connection between performance and future.

CoordenaÃÃo de AperfeiÃoamento de Pessoal de NÃvel Superior / A presente pesquisa busca verificar se à possÃvel avaliar em larga escala a escrita de textos de alunos do 1 ano do ensino fundamental. Para tanto, foram analisadas diferentes produÃÃes textuais de crianÃas, participantes de prÃ-testes realizados pela equipe de avaliaÃÃo do PAIC-Alfa (Programa de AlfabetizaÃÃo na Idade Certa). Durante a pesquisa foram sistematizados possÃveis critÃrios de anÃlise visando a uma melhor investigaÃÃo de escritas no 1 ano, tomando uma proporÃÃo maior, num quantitativo maior de alunos. Tais critÃrios tentam contemplar aspectos essenciais da produÃÃo textual, tÃo subjetivos na avaliaÃÃo da escrita. Trata-se de uma pesquisa qualitativa e descritiva, na medida em que sÃo analisadas produÃÃes textuais de crianÃas em fase de alfabetizaÃÃo. Os dados foram colhidos em uma pesquisa de campo durante os prÃ-testes, ocasiÃes em que tambÃm foram aplicados questionÃrios de investigaÃÃo para algumas das professoras das turmas testadas, no intuito de aprofundar os estudos acerca da alfabetizaÃÃo e da produÃÃo textual das crianÃas que estavam sendo avaliadas. Para aprofundar a anÃlise qualitativa, tambÃm foi realizado um laboratÃrio com especialistas em avaliaÃÃo e alfabetizaÃÃo, em que foram observadas categorias de anÃlise da produÃÃo escrita de crianÃas no 1 ano do Ensino Fundamental. Este estudo foi importante para entender peculiaridades das escritas infantis, achados preciosos, que revelaram a capacidade de crianÃas que, com menos de sete anos, estudantes de escola pÃblica, conseguiram escrever textos, com sentido, significado e obedecendo a padrÃes convencionais da escrita; e aquelas que nÃo o conseguiram jà tinham, no mÃnimo, se aproximado de escritas prÃ-silÃbicas e silÃbicas, nÃveis tambÃm interessantes em crianÃas com essa idade e avaliadas no meio do 1 semestre do ano letivo. Os resultados desse trabalho mostraram que à possÃvel a avaliaÃÃo da escrita em larga Escala de crianÃas em processo de alfabetizaÃÃo, especificamente, no 1 ano do Ensino Fundamental, considerando a avaliaÃÃo como diagnÃstica e com o objetivo da intervenÃÃo na aprendizagem. / This research aims to verify whether it is possible to evaluate large-scale writing texts for students of 1st year of elementary school. Thus, we analyzed different textual productions of children participating in pre-tests conducted by the evaluation team of the Alfa-PAIC (Age Literacy Program in One). During the study, we systematized possible criteria for analysis in order to better investigate written in the 1st year, taking a greater proportion, a greater quantity of students. These criteria attempt to cover essential aspects of textual production, as the subjective assessment of writing. This is a qualitative and descriptive in that it is analyzed textual productions of children in the literacy phase. In the research field during the pre-tests, data were collected and questionnaires were applied to research some of the teachers of the classes tested in order to deepen the studies of literacy and the textual production of the children who were being evaluated. For further qualitative analysis, a laboratory evaluation specialists and literacy was held in which categories were observed analyzing the written production of children at 1 year of elementary school. This study was important to understand the peculiarities of writing for children, precious findings, which revealed the ability of children with less than seven years, public school students, were able to write texts with meaning, significance and obeying the conventional standards of writing, and those who had not succeeded, at least, approached written pre-syllabic and syllabic levels also interesting for children at this age and evaluated in the middle of the first semester of the school year. The findings showed that it is possible to evaluate large-scale writing of children in the literacy process, specifically, in the 1st grade of elementary school, considering the evaluation as a diagnostic and intervention with the goal of learning.

Esta pesquisa teórica investiga a controvérsia acadêmica sobre avaliações em larga escala no Brasil, sistematizando as posições/argumentos desse debate na forma de um continuum argumentativo, além de delinear tensões discursivas e contextuais que circundam e perpassam a polêmica sobre a avaliação da escola básica. A revisão de literatura pertinente e a apreciação dos discursos acadêmicos gravitantes da temática, do ponto de vista teórico-metodológico, orientaram-se pela noção de polifonia (BAKHTIN, 2005; MAINGUENEAU, 2002; CHARAUDEAU; MAINGUENEAU, 2016). Ou seja, pelo esforço de abordar, de forma simultânea e equipolente, as várias vozes que se pronunciam diversamente sobre o tema das avaliações em larga escala no Brasil. Desse modo, a incursão teórica segue por três eixos de sistematização/interpretação: 1) a reconstituição da trama histórica do desenvolvimento do aparelho de avaliação externa no Brasil, por meio de uma periodização interpretativa que põe em relevo as tensões e disputas centrais que incidiram nas discussões entre acadêmicos e especialistas da avaliação; 2) o mapeamento dos conceitos abrangentes (categorias-chave da problemática educacional estreitamente vinculadas à esfera da avaliação), evidenciado tensões potencialmente constitutivas da controvérsia da avaliação; 3) o desvelamento do embate ideológico-discursivo no campo da avaliação e seu desdobramento políticopedagógico em diferentes zonas argumentativas - gradações de um continumm com diferentes graus de apoio e recusa às avaliações externas e em larga escala. Por fim, ao relacionar as tensões discursivas geradoras e as tensões contextuais constitutivas da controvérsia em questão, propõe-se uma formulação teórica exploratória sobre a gênese do continuum argumentativo, com potencial de orientar possíveis desdobramentos teóricos e empíricos da investigação aqui empreendida. / This theoretical research investigates the academic controversy about large scale assessments in Brazil, systematizing the views/arguments of this debate in a form of a continuum argumentation, outlining the discursive and contextual tensions that surround and permeate the polemics about large scale assessment in basic school. The relevant review of the literature and the appraisal of academic discourses about the theme, from the theoretical-methodological point of view, were supported by the notion of polyphony (BAKHTIN, 2005; MAINGUENEAU, 2002; CHARAUDEAU; MAINGUENEAU, 2016). By the effort to approach, simultaneously and in an equipollent way, the different voices that are expressed diversely about the theme of large scale assessment in Brazil. Therefore, the theoretical incursion is in accordance with three pillars of systematization/interpretation: 1) The reconstruction of a historical plot about the development of external assessment apparatus in Brazil, by means of interpreting the periods that highlight the tensions and central disputes in the discussions between scholars and experts in assessment; 2) The mapping of broad concepts ( key-categories of the educational problematic closely linked to the sphere of evaluation), highlight tensions that potentially constitute the controversies of assessment.; 3) the unveiling of the ideologically-discursive conflict in the field of assessment and its political-pedagogical unfolding in different argumentative areas gradations of a continuum with different degrees of support and the refusal of external assessment in large scale. Finally, in relating the discursive tensions generators and the constitutive contextual tensions of the controversy at issue, an exploratory theoretical formulation is proposed about the genesis of the continuum argumentative, with the purpose to guide possible theoretical and empirical unfolding of investigation here pursued.

The aim of this study is to validate the Tshivenda learner performance in the Progress in International Reading Literacy Study (PIRLS) 2006 in which there is an anomaly in the Tshivenda language group. By comparing the performance of the Tshivenda learners to that of learners who wrote the PIRLS 2006 test in the other official languages, the notion of performance is related to equivalence in translation in that, if the learners wrote equivalent instruments across all official South African languages, then it is possible that the difference in performance was related to different translation equivalence. Therefore, the validation of the learner performance in this study is directly linked to the validation of the translation. The South African national results of PIRLS 2006 revealed that the Tshivenda language speakers, who had written the PIRLS tests in a secondary language, achieved higher scores than those Tshivenda speakers who had written the tests in their mother tongue (Tshivenda). This result was considered an anomaly. This research investigated the role of translation as an influencing factor in learner comprehension, which may have contributed to this anomaly. Some of the procedures and standards set in place for PIRLS 2006 related to translation and verification were examined. Issues of language and culture, with specific reference to the availability of media in Tshivenda are discussed in the literature. Further investigation was conducted into what translation entails including translation and back-translation, equivalence and non-equivalence as well as the comprehension processes required by each of the four released PIRLS 2006 texts. This study is a secondary analysis of data gathered for PIRLS 2006. Permission to use the data was given in 2011 by the Centre for Evaluation and Assessment at the University of Pretoria, the PIRLS National Centre. Details of the original sampling, collection and analysis methods are provided as part of the discussion on the quality assurance, validity and reliability of the original study. The secondary analysis of the data utilised a mixed methods approach which involved Classical Test Theory and Content Analysis in order to accurately explore this data. The results of this study indicated that, despite the fact that the back-translation revealed many errors, the translation did not affect the learners’ level of comprehension. / Dissertation (MEd)--University of Pretoria, 2015. / tm2015 / Science, Mathematics and Technology Education / MEd / Unrestricted

Thesis advisor: Henry Braun / The State NAEP program only reports the mean achievement estimate of a subgroup within a given state if it samples at least 62 students who identify with the subgroup. Since some subgroups of students constitute small proportions of certain states’ general student populations, these low-incidence groups of students are seldom sufficiently sampled to meet this rule-of-62 requirement. As a result, education researchers and policymakers are frequently left without a full understanding of how states are supporting the learning and achievement of different subgroups of students.Using grade 8 mathematics results in 2015, this dissertation addresses the problem by comparing the performance of three different techniques in predicting mean subgroup achievement on NAEP. The methodology involves simulating scenarios in which subgroup samples greater or equal to 62 are treated as not available for calculating mean achievement estimates. These techniques comprise an adaptation of Multivariate Imputation by Chained Equations (MICE), a common form of Small Area Estimation known as the Fay-Herriot model (FH), and a Cross-Survey analysis approach that emphasizes flexibility in model specification, referred to as Flexible Cross-Survey Analysis (FLEX CS) in this study. Data used for the prediction study include public-use state-level estimates of mean subgroup achievement on NAEP, restricted-use student-level achievement data on NAEP, public-use state-level administrative data from Education Week, the Common Core of Data, the U.S. Census Bureau, and public-use district-level achievement data in NAEP-referenced units from the Stanford Education Data Archive. To evaluate the accuracy of the techniques, a weighted measure of Mean Absolute Error and a coverage indicator quantify differences between predicted and target values. To evaluate whether a technique could be recommended for use in practice, accuracy measures for each technique are compared to benchmark values established as markers of successful prediction based on results from a simulation analysis with example NAEP data. Results indicate that both the FH and FLEX CS techniques may be suitable for use in practice and that the FH technique is particularly appealing. However, before definitive recommendations are made, the analyses from this dissertation should be conducted employing math achievement data from other years, as well as data from NAEP Reading. / Thesis (PhD) — Boston College, 2021. / Submitted to: Boston College. Lynch School of Education. / Discipline: Educational Research, Measurement and Evaluation.