The inclusion of all students in large-scale assessment mandated by the No Child Left Behind (2003) requires that these large-scale assessments be developed to allow all students to show what they know, and that the results are comparable and equitable across diverse cultural and linguistic populations. This study examined the validity of the 5th grade 2009 Science Texas Assessment of Knowledge and Skills (TAKS) for diverse cultural and linguistic groups. The student groups considered for this study were selected based on all the possible combinations of three variables: ethnicity--White and Hispanic, test language--English and Spanish, and Limited English Proficiency (LEP) classification. Validity was assessed at the item and construct levels, and was analyzed from a psychometric, cultural and linguistic stance. At the item level, Differential Item Function (DIF) was conducted using the Mantel-Haenszel procedure. The presence of biased items was revealed for all pairwise group comparisons; with a high number of DIF items between groups which differed in English proficiency (approximately 50% of the test items), and a low number of DIF items between groups which only differ in ethnicity (approximately 15% of the test items). However, an analysis of the Item Characteristic Curves (ICCs), revealed that items classified by the Mantel-Haenszel procedure as advantaging the LEP groups, did so for students at low proficiency levels; while the advantage at high proficiency levels was for non-LEP groups. At the construct level, the structure of the English version of the TAKS was compared across three student groups using Confirmatory Factor Analysis with Multiple Groups. The hypothesized structure based on the TAKS blueprint, was rejected for the Group conformed by White, non-LEP students (MLM[subscript x]²[subscript(734)] = 1042.110; CFI= 0.845; RMSEA= 0.020); but, it was a good fit for Hispanic, non-LEP (MLM[subscript x]²[subscript(734)] = 819.356; CFI= 0.980; RMSEA= 0.011) and LEP (MLM[subscript x]²[subscript(734)] = 805.124; CFI= 0.985; RMSEA= 0.010) Groups. The results obtained from this study call to reinterpret the achievement gap observed in TAKS scores between the populations considered, and highlight the need for further development of guidelines that can better help to develop fair large-scale tests for all students. / text

No description available.

Results from large-scale assessments of academic achievement are key sources of evidence in the development of education policy and reform. The increasing influence of these assessments underscores the need for the results to be valid and reliable. This study investigates possible threats to the validity of reading proficiency assessments by examining the influence of two motivational variables: the interest attributed to the texts students read, and the amount of effort that students invest in undertaking the reading assessment. Using data from Australian pilot assessments and the Programme for International Student Achievement (PISA) this study explores the influence of interest and effort on reading proficiency outcomes and on the conclusions that can be drawn from these assessments.

Current research suggests that a high level of confidence and a low level of anxiety are predictive of higher math achievement. Compared to students from other provinces, previous research has found that Saskatchewan students have a higher level of confidence and a lower level of anxiety for learning math, but still tend to achieve lower math scores compared to students in other provinces. The data suggest that there may be unique factors effecting math learning for students in Saskatchewan. The purpose of the study is to determine the factors that may affect Saskatchewan students math achievement. Exploratory factor analyses and regression methods were employed to investigate possible traits that aid students in achieving higher math scores. Results from a 2007 math assessment administered to grade 5 students in Saskatchewan were used for the current study. The goal of the study was to provide a better understanding of the factors and trends unique to students for mathematic achievements in Saskatchewan.<p> Using results from a province-wide math assessment and an accompanying questionnaire administered to students in grade five across public school in Saskatchewan (n=11,279), the present study found statistical significance in three factors that have been supported by previous studies to influence math achievement differences, specifically in (1) confidence in math, (2) parental involvement in math and (3) extracurricular participation in math. The three aforementioned factors were found to be related to math achievement as predicted by the Assessment for Learning (AFL) program in Saskatchewan, although there were reservations to the findings due to a weak amount of variances accounted for in the regression model (r2 =.084). Furthermore, a multivariate analysis of variance indicated gender and locations of schools to have effects on students math achievement scores. Although a high amount of measurement errors in the questionnaire (and subsequently a low variance accounted for by the regression model) limited the scope and implications of the model, future implications and improvements are discussed

Current research suggests that a high level of confidence and a low level of anxiety are predictive of higher math achievement. Compared to students from other provinces, previous research has found that Saskatchewan students have a higher level of confidence and a lower level of anxiety for learning math, but still tend to achieve lower math scores compared to students in other provinces. The data suggest that there may be unique factors effecting math learning for students in Saskatchewan. The purpose of the study is to determine the factors that may affect Saskatchewan students math achievement. Exploratory factor analyses and regression methods were employed to investigate possible traits that aid students in achieving higher math scores. Results from a 2007 math assessment administered to grade 5 students in Saskatchewan were used for the current study. The goal of the study was to provide a better understanding of the factors and trends unique to students for mathematic achievements in Saskatchewan.<p> Using results from a province-wide math assessment and an accompanying questionnaire administered to students in grade five across public school in Saskatchewan (n=11,279), the present study found statistical significance in three factors that have been supported by previous studies to influence math achievement differences, specifically in (1) confidence in math, (2) parental involvement in math and (3) extracurricular participation in math. The three aforementioned factors were found to be related to math achievement as predicted by the Assessment for Learning (AFL) program in Saskatchewan, although there were reservations to the findings due to a weak amount of variances accounted for in the regression model (r2 =.084). Furthermore, a multivariate analysis of variance indicated gender and locations of schools to have effects on students math achievement scores. Although a high amount of measurement errors in the questionnaire (and subsequently a low variance accounted for by the regression model) limited the scope and implications of the model, future implications and improvements are discussed

Despite the lack of substantial evidence for improvement in the quality of teaching and learning with information and communication technology (ICT), governmental organizations, including those of Canada and Australia, have made large investments into ICT. This investment has been largely predicated on the hypothesized relationship between ICT and science achievement, and the need for ICT as a means of providing broad-scale training to meet the demand for a skilled workforce. To better understand this possible relationship, this study used data from the 2006 administration of the Programme for International Student Assessment (PISA 2006) to determine the extent to which scientific literacy is predicted by student- and school-level variables related to ICT, after adjusting for student demographic characteristics and school characteristics. The findings suggest that, once student demographic characteristics and school characteristics have been accounted for, students with prior experience with ICT, who browse the Internet more frequently, and who are confident with basic ICT tasks earned higher scientific literacy scores. Gender differences existed with respect to types of productivity and entertainment software used; this difference may be attributed to personal choice and initiative to learn ICT. Finally, differences in ICT use between Canada and Australia, particularly with school use, may be due to initiatives in Australia (e.g., National Goals of Schooling for the Twenty-first Century) that promote the increased use of ICT in classrooms. / Thesis (Master, Education) -- Queen's University, 2009-08-31 11:14:15.59

Thesis advisor: Ina V.S. Mullis / This dissertation describes the foundation for maintaining TIMSS’ 20 year trend measurements with the introduction of a new computer- and tablet-based mode of assessment delivery—eTIMSS. Because of the potential for mode effects on the psychometric behavior of the trend items that TIMSS relies on to maintain comparable scores between subsequent assessment cycles, development efforts for TIMSS 2019 began over three years in advance. This dissertation documents the development of eTIMSS over this period and features the methodology and results of the eTIMSS Pilot / Item Equivalence Study. The study was conducted in 25 countries and employed a within-subjects, counterbalanced design to determine the effect of the mode of administration on the trend items. Further analysis examined score-level mode effects in relation to students’ socioeconomic status, gender, and self-efficacy for using digital devices. Strategies are discussed for mitigating threats of construct irrelevant variance on students’ eTIMSS performance. The analysis by student subgroups, similar item discriminations, high cross-mode correlations, and equivalent rankings of country means provide support for the equivalence of the mathematics and science constructs between paperTIMSS and eTIMSS. However, the results revealed an overall mode effect on the TIMSS trend items, where items were more difficult for students in digital formats compared to paper. The effect was larger in mathematics than science. An approach is needed to account for the mode effects in maintaining trend measurements from previous cycles to TIMSS 2019. Each eTIMSS 2019 trend country will administer the paper trend booklets to an additional nationally representative bridge sample of students, and a common population equating approach will ensure the link between paperTIMSS and eTIMSS scores. / Thesis (PhD) — Boston College, 2018. / Submitted to: Boston College. Lynch School of Education. / Discipline: Educational Research, Measurement and Evaluation.

Esta pesquisa teórica investiga a controvérsia acadêmica sobre avaliações em larga escala no Brasil, sistematizando as posições/argumentos desse debate na forma de um continuum argumentativo, além de delinear tensões discursivas e contextuais que circundam e perpassam a polêmica sobre a avaliação da escola básica. A revisão de literatura pertinente e a apreciação dos discursos acadêmicos gravitantes da temática, do ponto de vista teórico-metodológico, orientaram-se pela noção de polifonia (BAKHTIN, 2005; MAINGUENEAU, 2002; CHARAUDEAU; MAINGUENEAU, 2016). Ou seja, pelo esforço de abordar, de forma simultânea e equipolente, as várias vozes que se pronunciam diversamente sobre o tema das avaliações em larga escala no Brasil. Desse modo, a incursão teórica segue por três eixos de sistematização/interpretação: 1) a reconstituição da trama histórica do desenvolvimento do aparelho de avaliação externa no Brasil, por meio de uma periodização interpretativa que põe em relevo as tensões e disputas centrais que incidiram nas discussões entre acadêmicos e especialistas da avaliação; 2) o mapeamento dos conceitos abrangentes (categorias-chave da problemática educacional estreitamente vinculadas à esfera da avaliação), evidenciado tensões potencialmente constitutivas da controvérsia da avaliação; 3) o desvelamento do embate ideológico-discursivo no campo da avaliação e seu desdobramento políticopedagógico em diferentes zonas argumentativas - gradações de um continumm com diferentes graus de apoio e recusa às avaliações externas e em larga escala. Por fim, ao relacionar as tensões discursivas geradoras e as tensões contextuais constitutivas da controvérsia em questão, propõe-se uma formulação teórica exploratória sobre a gênese do continuum argumentativo, com potencial de orientar possíveis desdobramentos teóricos e empíricos da investigação aqui empreendida. / This theoretical research investigates the academic controversy about large scale assessments in Brazil, systematizing the views/arguments of this debate in a form of a continuum argumentation, outlining the discursive and contextual tensions that surround and permeate the polemics about large scale assessment in basic school. The relevant review of the literature and the appraisal of academic discourses about the theme, from the theoretical-methodological point of view, were supported by the notion of polyphony (BAKHTIN, 2005; MAINGUENEAU, 2002; CHARAUDEAU; MAINGUENEAU, 2016). By the effort to approach, simultaneously and in an equipollent way, the different voices that are expressed diversely about the theme of large scale assessment in Brazil. Therefore, the theoretical incursion is in accordance with three pillars of systematization/interpretation: 1) The reconstruction of a historical plot about the development of external assessment apparatus in Brazil, by means of interpreting the periods that highlight the tensions and central disputes in the discussions between scholars and experts in assessment; 2) The mapping of broad concepts ( key-categories of the educational problematic closely linked to the sphere of evaluation), highlight tensions that potentially constitute the controversies of assessment.; 3) the unveiling of the ideologically-discursive conflict in the field of assessment and its political-pedagogical unfolding in different argumentative areas gradations of a continuum with different degrees of support and the refusal of external assessment in large scale. Finally, in relating the discursive tensions generators and the constitutive contextual tensions of the controversy at issue, an exploratory theoretical formulation is proposed about the genesis of the continuum argumentative, with the purpose to guide possible theoretical and empirical unfolding of investigation here pursued.

Esta dissertação tem como problema de pesquisa: quais são as representações de professores e de alunos sobre a Provinha Brasil? Objetiva: (1) identificar e problematizar as representações – significados, modos de operacionalização e usos – sobre a Provinha Brasil a partir das falas de professores de três turmas de 2º ano do Ensino Fundamental e (2) inventariar e analisar o que representa ser avaliado pela Provinha Brasil na perspectiva dos alunos. A Provinha Brasil é uma avaliação em larga escala com função diagnóstica, aplicada no início e no final do 2º ano do Ensino Fundamental, visando aferir os progressos no rendimento dos alunos (em leitura e matemática) e dar subsídio ao planejamento dos professores. A pesquisa foi realizada em três escolas públicas de âmbito federal, estadual e municipal. Se enquadra em uma abordagem de pesquisa qualitativa, utilizando as seguintes ferramentas metodológicas: entrevista semiestruturada, observação e “aula-conversa”, esta última ferramenta criada para produção dos dados junto aos alunos. Os conceitos-ferramentas utilizados na investigação foram: Representação cultural (HALL, 1997) e Governamento (FOUCAULT, 1982). As análises mostraram que: os significados que as professoras atribuem à Provinha Brasil envolvem enxergá-la como um parâmetro, a partir da legitimação dos conhecimentos e habilidades abordados pelo exame, por serem similares aos que trabalham em suas salas de aula, mas que não têm fortes impactos nas suas práticas, sendo um reforço ao que elas já vinham observando sobre seus alunos. Os modos de operacionalização mostram que as professoras tentam tornar o momento de aplicação tranquilo, para que os alunos consigam fazer a prova e para que esse jeito de conduzir o processo, motivando os alunos, tenha impacto nos desempenhos. As professoras se põem contrárias a focar suas aulas na preparação dos alunos para “irem bem” na avaliação, mas os preparam, fazendo questões de múltipla escolha esporadicamente, para dar conta dos rituais de exame. Quanto aos usos, as professoras tentam “fazer desse limão uma limonada”, usando a Provinha Brasil dentro de sua potência, que é dar uma visão geral do desempenho da turma. Elas também apontam que interpretam os resultados, às vezes categorizando os alunos nos níveis que a avaliação propõe, e pensam ações para dar conta das aprendizagens não consolidadas pelos alunos. Os alunos representaram a avaliação como uma novidade bem aceita e trouxeram à tona significações partilhadas culturalmente sobre “prova”. Mesmo em menor número, alguns alunos também representaram a avaliação como um lugar de desconforto, a partir da insegurança, do nervosismo e das dúvidas, fruto de uma relação recém iniciada com o instrumento “prova”. Também relacionaram a avaliação à autorresponsabilização, trazendo outro significado partilhado culturalmente, uma vez que produzem uma ligação entre desempenho e futuro. / This dissertation has as research problem: what are the representations from teachers and students about the Provinha Brasil? It aims to: (1) identify and problematize the representation – meanings, ways of operationalization and the uses – about the Provinha Brasil based on the talks of three teachers from 2nd grade of elementary school and (2) listing and analyze what represents being evaluated by the Provinha Brasil on the students’ perspective. The Provinha Brasil is a large-scale assessment with diagnostic function that is applied at the beginning and at the end of 2nd grade of elementary school, aiming to assess the progress in student achievement (in reading and mathematics) and to contribute to teachers’ lesson plan. The research was realized in three public schools from different spheres, federal, state and municipal. It is a qualitative research that uses as methodological resources: the semi structured interview, the observations and “class-talk”, this last one created to produce data with the students. The main concepts that were used in this investigation were Cultural Representation (HALL, 1997) and Government (FOUCALT, 1982). The analyses revealed that: the meanings that teachers attribute to the Provinha Brasil involve consider it as a parameter, based on the legitimation of knowledge and abilities that are approached by the exam, because they are similar to the ones worked in their classrooms, but they do not have a strong impact in their practices, being considered a reinforcement to the things they have already observed about their students. The ways of operationalizing shows that the teachers try to make it a calm moment, in order to the students can answer the text and so that, this way of conducting the process, motivating the students, may cause an impact on their performance. The teachers do not put the focus of their classes on the preparation of their students to “do it well” in the evaluation, but they prepare them, proposing questions of multiple choice, occasionally, so the ritual of the exam is practiced. Concerning the uses, the teachers try to “make a lemonade out of lemons”, using the Provinha Brasil in its force, that is to give a general view from the group performance. They also point out that they interpret the results, sometimes categorizing the students in the levels proposed by the evaluation and planning actions to work on these nonconsolidated learning aspects of the students. The students represent the evaluation as a novelty well accepted and bring out meanings culturally shared concerning the meaning of “test”. Even though being in a small number, some students also represent the evaluation as a uncomfortable place, due to the insecurity, the jitters and the doubts, that come from a relation that began recently with the“test” as an instrument of assessment. They also associate evaluation and self-responsability, bringing out another meaning culturally shared, since they produce a connection between performance and future.

Using eighth-grade mathematics scores from TIMSS 2003, a large-scale international achievement assessment database, this study investigated correlates of math achievement in two developed countries, Canada and the United States and two developing countries, Egypt and South Africa. Variation in math achievement within and between schools for individual countries was accounted for by a series of two-level HLM models. Specifically, there were five sets of HLM models representing student background, home resources, instructional practices, teacher background, and school background related factors. In addition, a final model was built by including all the statistically significant predictors in earlier models to predict math achievement. Findings from this study suggested that whereas the instructional practices model worked the best for the United States and the teacher background model served as the most efficient and parsimonious model for predicting math achievement in Egypt, the final model served as the best model for predicting math achievement in Canada and South Africa. These findings provide empirical evidence that different models are needed to account for factors related to achievement in different countries. This study, therefore, highlights the importance that policy makers and educators from developing countries should not base their educational decisions and educational reform projects solely on research findings of developed countries. Rather, they need to use their country-specific findings to support their educational decisions. This study also provides a methodological framework for applied researchers to evaluate the effects of background and contextual factors on students' math achievement