Thesis advisor: Ina V. S. Mullis / TIMSS and PIRLS assess representative samples of students at regular intervals, measuring trends in student achievement and student contexts for learning. Because individual students are not tracked over time, analysis of international large-scale assessment data is usually conducted cross-sectionally. Gustafsson (2007) proposed examining the data longitudinally by analyzing relationships between country-level trends in background constructs and trends in student achievement. Through longitudinal analysis of international large-scale assessment data, it becomes possible to mitigate some of the confounding factors in the analysis. This dissertation extends this country-level approach to subpopulations within countries. Adapting a pseudo-panel approach from the econometrics literature (Deaton, 1985), the proposed approach creates subpopulations by grouping students based on demographic characteristics, such as gender or parental education. Following grouping, the subpopulations with the same demographic characteristics are linked across cycles and the aggregated subpopulation means are treated as panel data and analyzed through longitudinal data analysis techniques. As demonstrated herein the primary advantages of the subpopulation approach are that it allows for analysis of subgroup differences, and it captures within-country relationships in the data that are not possible to analyze at country level. Illustrative analysis examines the relationship between early literacy activities and PIRLS reading achievement using PIRLS 2001 and PIRLS 2011 data. Results from the subpopulation approach are compared with student-level and country-level cross-sectional results as well as country-level longitudinal results. In addition, within-country analysis examines the subpopulation-level relationship between early literacy activities and PIRLS reading achievement, multiple group analysis compares regression coefficient estimates between boys and girls and across parental education subgroups, and mediation analysis examines the extent that partaking in early literacy activities can explain differences between boys and girls in PIRLS reading achievement. / Thesis (PhD) — Boston College, 2017. / Submitted to: Boston College. Lynch School of Education. / Discipline: Educational Research, Measurement and Evaluation.

The results of the international study TIMSS (The Trends in International Mathematics and Science Study) raise concerns that Czech primary pupils fall behind in some content domains of science (namely the physical sciences) and in some skills (experimental skills). This thesis compares the Czech curriculum with the current international trends in the science education and shows that above mentioned topics/skills are not sufficiently covered in the official curriculum. The second part of the thesis describes a case study of a relatively successful implementation of innovative science lessons focused on these skills in three primary classrooms of a Czech school. The concepts of inquiry-based learning, theory of learning materials and lesson study method are the main components of the theoretical and conceptual background of the study. When comparing the Czech curriculum (official documents and textbooks used in school) and school curricular plans with the TIMSS framework, it emerges that both the national curricular framework and the school curriculum do not cover some of those upper mentioned. Based on these findings, a lesson plan introducing some of the missing areas and task types was developed and tested in a particular school. The qualitative research desigh of case study with embedded...

Thesis advisor: Henry I. Braun / Secondary analyses of international large-scale assessments (ILSA) commonly characterize relationships between variables of interest using correlations. However, the accuracy of correlation estimates is impaired by artefacts such as measurement error and clustering. Despite advancements in methodology, conventional correlation estimates or statistical models not addressing this problem are still commonly used when analyzing ILSA data. This dissertation examines the impact of both the clustered nature of the data and heterogeneous measurement error on the correlations reported between background data and proficiency scales across countries participating in ILSA. In this regard, the operating characteristics of competing modeling techniques are explored by means of applications to data from PISA 2012. Specifically, the estimates of correlations between math self-efficacy and math achievement across countries are the principal focus of this study. Sequentially employing four different statistical techniques, a step-wise model refinement approach is used. After each step, the changes in the within-country correlation estimates are examined in relation to (i) the heterogeneity of distributions, (ii) the amount of measurement error, (iii) the degree of clustering, and (iv) country-level math performance. The results show that correlation estimates gathered from two-dimensional IRT models are more similar across countries in comparison to conventional and multilevel linear modeling estimates. The strength of the relationship between math proficiency and math self-efficacy is moderated by country mean math proficiency and this was found to be consistent across all four models even when measurement error and clustering were taken into account. Multilevel multidimensional mixture IRT modeling results support the hypothesis that low-performing groups within countries have a lower correlation between math self-efficacy and math proficiency. A weaker association between math self-efficacy and math proficiency in lower achieving groups is consistently seen across countries. A multilevel mixture IRT modeling approach sheds light on how this pattern emerges from greater randomness in the responses of lower performing groups. The findings from this study demonstrate that advanced modeling techniques not only are more appropriate given the characteristics of the data, but also provide greater insight about the patterns of relationships across countries. / Thesis (PhD) — Boston College, 2021. / Submitted to: Boston College. Lynch School of Education. / Discipline: Educational Research, Measurement and Evaluation.

This thesis addresses the local reception and use in Sweden of the major international large scale assessments (ILSAs) of student performance: Programme for International Student Assessment (PISA), Trends in International Mathematics and Science Study (TIMSS), Progress in International Reading Literacy Study (PIRLS) and International Civic and Citizenship Education Study (ICCS). The aim is to describe and analyze how Swedish teachers, principals and school directors interpret and possibly use ILSAs in their professional practice. ILSA is treated here as a new idea or a new social technology defining what constitutes good (or bad) education. The theoretical framework combines a top-down perspective provided by Rogers’ (2003) Diffusion of Innovation theory, and a complementary, more critical Policy Enactment approach (Ball et al. 2012), stressing the import-ance of context and local actors’ perspectives. Empirically, the thesis is based on 40 semi-structured interviews carried out in the 2011-2012 school year with teachers, principals and municipal school directors in five municipalities and 12 compulsory schools, selected to cover diverse municipalities, schools, and respondents. The respondents perceive ILSAs as valid evaluations of the Swedish school system. Most think it is important to compare results of different nations, although ILSA is not expected to cover the whole curriculum. Most interviewees are aware that Swedish ILSA results have been declining for years and perceive an urgent need to reverse this trend. However, few of the directors, principals or teachers believe that Swedish schools are in a deep crisis, as described in Swedish media. The participants frequently regard schools as primary determinants of ILSA results, and few blame family, socio-economic, cultural and contextual factors for the Swedish decline in ILSA rankings. There are significant differences between the three occupational groups in their reception and use of ILSA. Municipal school directors who are very well-informed emphasize the influence of ILSA on their local development efforts. Principals and teachers say that ILSAs have had modest direct effects on their work, but they argue that poor Swedish results in international assessments have had indirect effects, for instance by prompting the introduction of a new national curriculum. / <p>Contains an English summary.</p>