The aim of this thesis is to ascertain the degree to which high-stakes assessment for qualifications, such as New Zealand's National Certificate of Educational Achievement (NCEA), act as a barrier to secondary English teachers' use of information and communication technologies (ICTs) with their students. Although the focus is on high-stakes assessment for qualifications, other factors, which might also facilitate or hinder English teachers' use of ICTs, such as professional development and infrastructural, technical and access issues, are also considered. The literature review summarises the factors which tend to constrain or encourage teachers' use of ICTs, with a special focus on the considerable constraint placed on secondary teachers by their role in preparing and assessing students for high-stakes qualifications. The literature review also highlights the lack of convincing research into the impact on learning of ICTs but reveals that, in the subject English, there is some evidence of a positive impact when appropriate ICTs are used by well trained teachers in appropriate contexts. Key NCEA foundation and implementation documents and publicity, achievement standards and assessment activities were analysed to ascertain the degree of official endorsement for the use of ICTs in secondary schools and English programmes in particular. To gauge the perceptions of New Zealand English teachers about the constraints and encouragers of ICTs in their teaching, all NCEA level one English teachers were surveyed. This was followed by face-to-face and online focus groups in which trends revealed in the survey were explored. Document analysis revealed considerable official optimism that the flexibility and internal assessment of the NCEA would enable teachers to make greater use of ICTs. The achievement standards and supporting assessment activities however, tend to situate ICTs at the margins of English programmes as optional extras which, if used at all, tend to support current practice. The focus groups confirmed survey findings that, although English teachers are significant users of ICTs in their personal and professional lives, although they believe in the educational advantages of ICTs and although they work in schools and departments which support the classroom use of ICTs, they face significant constraints which prevent them making as much use of ICTs as they would like in their teaching. Most significant among these constraints is pressure of course coverage and lack of class time (largely attributable to the need to prepare students for high-stakes assessments). Other constraints include lack of adequate access to ICTs and technical support, and lack of appropriate professional development and time to learn about ICTs. Based on the literature review and research findings, recommendations are provided for schools, policy makers and researchers. Key among these is the need to acknowledge the profound influence of high-stakes qualifications on secondary schools and teachers and evolve such qualifications to encourage and enable desired innovations. It is recommended that ICTs could be infused into English and eventually, inter-disciplinary programmes, through the creation of innovative, ICT infused achievement standards which could be combined into flexibly structured courses which better meet the needs of twenty first century students. Also recommended are approaches which enable greater access to ICTs for English teachers and methods of professional development which have proved effective with adult learners. / Subscription resource available via Digital Dissertations only.

Teachers' conceptions are powerful in shaping the quality of their instructional practice. The purpose of this thesis is to defend a four-facet model of teachers' conceptions of assessment, which revolves around emphasising improvement or school accountability, or student accountability purposes or treating assessment as irrelevant. Further, it explores how those conceptions relate to teachers' conceptions of learning, teaching, curriculum, and teacher efficacy. A literature review is used to identify the major conceptions. Multiple studies led to a 50-item Teachers' Conceptions of Assessment (COA-III) questionnaire based on the four main conceptions of assessment. Structural equation modelling showed a close fit of a hierarchical, multi-dimensional model to the data. Teachers moderately agreed with the improvement conceptions and the system accountability conception. Teachers disagreed that assessment was irrelevant. However, teachers had little agreement that assessment was for student accountability. Improvement, school, and student accountability conceptions were positively correlated. The irrelevance conception was inversely related to the improvement conception and not related to the system accountability conception. A four-factor structure of teachers' beliefs about assessment, curriculum, teaching, learning, and teacher efficacy, was found. Teachers agreed that assessment influences and improves their teaching and student learning. They agreed less strongly that assessment, measuring surface learning only, makes schools, teachers, and students accountable and that teachers are able to conduct assessment through a systematic technological approach. They agreed at a similar level with student centred learning that involves deep approaches to learning, divorced from assessment. They disagreed with a telling type of teaching that focuses only on intellectual development of students or on reconstruction or reform of society. Use of the CoA-III makes teachers' conceptions of assessment more explicit and will assist in the development of teacher training programs, the design of assessment policy, and enhance further research into educational assessment practices. Furthermore, explicit attention to teachers' conceptions of assessment is expected to be a precursor to teachers' self-regulation of their assessment beliefs and practices.

No description available.

No description available.

Doctor of Philosophy / Curriculum and Instruction Programs / Charles E. Heerman / This statewide study examined the relationship between building income level and performance level percentage distribution, using 502 schools that earned a 2005-06 Kansas Standard of Excellence (SOE) building-wide award for reading or math. It originated from the premise that excellence is excellence, no matter the setting or income level of a school. A new baseline of data began in 2005-06 due to changes in the Kansas assessments, including more grades being tested than in previous years. The much larger database more accurately reflected the achievement of low-income students in Kansas. Decades of literature were reviewed, addressing influences on the development of Kansas standards, assessments, and the SOE award; the lifelong significance of income levels and achievement; high achievement for low-income students; and the pursuit of excellence through equitable educational reform. For purposes of this study, SOE schools were sorted into six designated types of buildings based on percentages of students eligible for free and reduced lunches, assessed grade levels, and SOE subject award. Results were reported using aggregate building groups as the unit of analysis. A two-way, repeated-measures, mixed design ANOVA general linear model served as an appropriate method to examine means for significant differences. Low-income SOE schools were noticeably fewer than medium- or high-income schools, especially at the senior high level. Three types of buildings showed some significant mean differences, but generally income did not appear to be a major factor. High-income buildings appeared to have a slight advantage; in the Exemplary category, high-income buildings outperformed the others; in the lower performance categories, high-income buildings had significantly lower means. The mean differences for high-income middle school/junior high buildings showed mainly moderate to large differences; other significant differences were rated as small to moderate. SOE schools of a given educational level and of varying income levels generally had similar performance scores in most of the performance level categories. Overall, major differences in performances were not evident among the different income levels of SOE buildings.

Le ministère de l’Éducation, du Loisir et du Sport (MELS) publie tous les ans des indicateurs (MELS, 2007) qui traitent de plusieurs aspects du système scolaire québécois. Quoique le MELS insiste sur l’obligation pour les écoles primaires et secondaires d’obtenir des résultats probants en termes d’« efficacité », les indicateurs utilisés pour apprécier la performance des écoles ne sont pas nécessairement conçus à cette fin. Une étude du MELS rapporte que les élèves de 5e secondaire éprouvent de la difficulté à obtenir de bons résultats à deux critères de correction (syntaxe et ponctuation, orthographe), parmi les six de l’épreuve unique de français écrit (MELS, 2008). Ce fait nous amène à nous intéresser à l’étude de la modélisation des liens entre des facteurs associés à l’environnement scolaire et les résultats des élèves de la région métropolitaine de Montréal, en ce qui a trait à ces deux critères pour les cohortes des années 2006, 2007 et 2008. Nous procédons d’abord à des analyses descriptives des variables pour chacune des trois populations. Nous poursuivons l’analyse en effectuant plusieurs modélisations multiniveaux multivariées des deux critères en fonction de variables indépendantes, caractéristiques de l’élève et de l’école. Les résultats de la présente recherche indiquent une relative stabilité dans la performance des élèves pour les trois années, avec une légère amélioration de la performance pour la cohorte de 2007, et qui se maintient pour la cohorte de 2008. Les élèves du secteur privé obtiennent de meilleurs résultats que ceux du public. Le résultat des filles est supérieur à celui des garçons et les élèves de langue maternelle française obtiennent de meilleurs résultats que ceux de langues maternelles différentes du français. Il importe cependant d’apporter quelques nuances dans l’interprétation de ces résultats. En outre, la part de variance dans la performance des élèves attribuable à l’élève est de l’ordre de 75,0 % et de l’ordre de 25,0 % à l’école. On note un effet différentiel des facteurs d’élève et d’école selon le critère considéré. Les variables caractéristiques de l’élève comptent pour 13,9 % de la variance totale de la performance des élèves en syntaxe et ponctuation et pour 9,8 % en orthographe. Les variables caractéristiques de l’école comptent pour 3,7 % de la variance totale de la performance des élèves en syntaxe et ponctuation et pour 6,4 % en orthographe. Certains facteurs d’école, comme la taille, la mixité ne semblent pas présenter un lien significatif avec la performance des élèves. / The ministère de l’Éducation, du Loisir et du Sport (MELS) of Quebec publishes every year educational indicators which describe many aspects of the school system. Although the MELS insists for elementary and high schools to have good results in terms of the effectiveness of their performance, the indicators that are used presently by the school’s system are not designed to appreciate their performance. A research by the MELS shows that secondary five high school students have some difficulty with two criteria (syntax and punctuation, spelling) out of the six for the French language examination (MELS, 2008). The actual research focuses on the modeling of links between school environment and student performances in the Montreal area over a period of three years. By using data from the MELS for years 2006, 2007 and 2008, this study investigates the relations between variables characteristics of student and school with the student performances. The analysis used descriptive statistics of the variables for each of the three years. We complete the analysis by doing some multilevel multivariate analysis. The results of this study indicated that there is a relative stability in student performances over the three years with a slight improvement in 2007 which continue in 2008. Students from the private sector performed better than those of the public sector. The girls continuously have better results than the boys over the three years period. Students with French as their mother’s tongue do have better results than those with mother’s tongue different than French. Nevertheless, some nuances are necessary for better understanding the interpretation of the results. Moreover, the part of variation in student performances is responsible for about 75,0 % to student level and for about 25,0 % to school level. We note some differential effects of student and school factors on the criteria considered. The student variables are responsible for about 13,9 % of the total variance in syntax and punctuation and for about 9,8 % in spelling. School variables are responsible for 3,7 % for the total variance in student performance for syntax and punctuation and for about 6,4 % for spelling. Some school factors, like the size of the school, the mix sex school do not seem to have any significant link with students’ performances.

This study used multilevel measurement modeling to examine the differential difficulties of math and reading items for Grade 6 students participating in Ontario’s provincial assessment in 2005-2006, in relation to whether they received a setting accommodation, had a learning disability (LD), and spoke a language in addition to English. Both differences in difficulty between groups of students for all items (impact) and for individual items (differential item functioning) were examined. Students’ language backgrounds (whether they spoke a language in addition to English) were not significantly related to item difficulty. Compared to non-accommodated students with LD, math and reading items were relatively difficult for accommodated students with LD. Moreover, the difference in overall impact on math items was larger than on reading items for accommodated and non-accommodated students with LD. Overall, students without LD and who did not receive a setting accommodation outperformed students with LD and/or who received a setting accommodation as well as accommodated students without LD. It is important to note that, because this was an operational test administration, students were assigned to receive accommodations by their schools based on their individual needs. It is, therefore, not possible to separate the effect of the setting accommodation on item difficulty from the effects of other differences between the accommodated and non-accommodated groups. The differences in math and reading item difficulties between accommodated and non-accommodated students with LD may be due in part to factors such as comorbidity of LD and attention deficit hyperactivity disorder (ADHD) or a possible mismatch between the setting accommodation and the areas of disabilities. Moreover, the results of the present study support the underarousal/optimal stimulation hypothesis instead of the premise of the inhibitory control and attention for the use of setting accommodation. After controlling for the impact across all items of setting accommodation and LD, several math and reading items were found to exhibit differential item functioning (DIF). The possible sources of DIF were (1) math items that were not adherent to specific item-writing rules and (2) reading items targeting different types of comprehension. This study also found that the linguistic features of math items (total words, total sentences, average word length, monosyllabic words for math) and reading items (word frequency, average sentence length, and average words per sentence for reading) were associated with math and reading item difficulties for students with different characteristics. The total sentences and average word length in a math item as well as total words in a reading item significantly predicted the achievement gap between groups. Therefore, the linguistic features should be taken into account when assessments are developed and validated for examinees with varied characteristics.

This study used multilevel measurement modeling to examine the differential difficulties of math and reading items for Grade 6 students participating in Ontario’s provincial assessment in 2005-2006, in relation to whether they received a setting accommodation, had a learning disability (LD), and spoke a language in addition to English. Both differences in difficulty between groups of students for all items (impact) and for individual items (differential item functioning) were examined. Students’ language backgrounds (whether they spoke a language in addition to English) were not significantly related to item difficulty. Compared to non-accommodated students with LD, math and reading items were relatively difficult for accommodated students with LD. Moreover, the difference in overall impact on math items was larger than on reading items for accommodated and non-accommodated students with LD. Overall, students without LD and who did not receive a setting accommodation outperformed students with LD and/or who received a setting accommodation as well as accommodated students without LD. It is important to note that, because this was an operational test administration, students were assigned to receive accommodations by their schools based on their individual needs. It is, therefore, not possible to separate the effect of the setting accommodation on item difficulty from the effects of other differences between the accommodated and non-accommodated groups. The differences in math and reading item difficulties between accommodated and non-accommodated students with LD may be due in part to factors such as comorbidity of LD and attention deficit hyperactivity disorder (ADHD) or a possible mismatch between the setting accommodation and the areas of disabilities. Moreover, the results of the present study support the underarousal/optimal stimulation hypothesis instead of the premise of the inhibitory control and attention for the use of setting accommodation. After controlling for the impact across all items of setting accommodation and LD, several math and reading items were found to exhibit differential item functioning (DIF). The possible sources of DIF were (1) math items that were not adherent to specific item-writing rules and (2) reading items targeting different types of comprehension. This study also found that the linguistic features of math items (total words, total sentences, average word length, monosyllabic words for math) and reading items (word frequency, average sentence length, and average words per sentence for reading) were associated with math and reading item difficulties for students with different characteristics. The total sentences and average word length in a math item as well as total words in a reading item significantly predicted the achievement gap between groups. Therefore, the linguistic features should be taken into account when assessments are developed and validated for examinees with varied characteristics.

Le ministère de l’Éducation, du Loisir et du Sport (MELS) publie tous les ans des indicateurs (MELS, 2007) qui traitent de plusieurs aspects du système scolaire québécois. Quoique le MELS insiste sur l’obligation pour les écoles primaires et secondaires d’obtenir des résultats probants en termes d’« efficacité », les indicateurs utilisés pour apprécier la performance des écoles ne sont pas nécessairement conçus à cette fin. Une étude du MELS rapporte que les élèves de 5e secondaire éprouvent de la difficulté à obtenir de bons résultats à deux critères de correction (syntaxe et ponctuation, orthographe), parmi les six de l’épreuve unique de français écrit (MELS, 2008). Ce fait nous amène à nous intéresser à l’étude de la modélisation des liens entre des facteurs associés à l’environnement scolaire et les résultats des élèves de la région métropolitaine de Montréal, en ce qui a trait à ces deux critères pour les cohortes des années 2006, 2007 et 2008. Nous procédons d’abord à des analyses descriptives des variables pour chacune des trois populations. Nous poursuivons l’analyse en effectuant plusieurs modélisations multiniveaux multivariées des deux critères en fonction de variables indépendantes, caractéristiques de l’élève et de l’école. Les résultats de la présente recherche indiquent une relative stabilité dans la performance des élèves pour les trois années, avec une légère amélioration de la performance pour la cohorte de 2007, et qui se maintient pour la cohorte de 2008. Les élèves du secteur privé obtiennent de meilleurs résultats que ceux du public. Le résultat des filles est supérieur à celui des garçons et les élèves de langue maternelle française obtiennent de meilleurs résultats que ceux de langues maternelles différentes du français. Il importe cependant d’apporter quelques nuances dans l’interprétation de ces résultats. En outre, la part de variance dans la performance des élèves attribuable à l’élève est de l’ordre de 75,0 % et de l’ordre de 25,0 % à l’école. On note un effet différentiel des facteurs d’élève et d’école selon le critère considéré. Les variables caractéristiques de l’élève comptent pour 13,9 % de la variance totale de la performance des élèves en syntaxe et ponctuation et pour 9,8 % en orthographe. Les variables caractéristiques de l’école comptent pour 3,7 % de la variance totale de la performance des élèves en syntaxe et ponctuation et pour 6,4 % en orthographe. Certains facteurs d’école, comme la taille, la mixité ne semblent pas présenter un lien significatif avec la performance des élèves. / The ministère de l’Éducation, du Loisir et du Sport (MELS) of Quebec publishes every year educational indicators which describe many aspects of the school system. Although the MELS insists for elementary and high schools to have good results in terms of the effectiveness of their performance, the indicators that are used presently by the school’s system are not designed to appreciate their performance. A research by the MELS shows that secondary five high school students have some difficulty with two criteria (syntax and punctuation, spelling) out of the six for the French language examination (MELS, 2008). The actual research focuses on the modeling of links between school environment and student performances in the Montreal area over a period of three years. By using data from the MELS for years 2006, 2007 and 2008, this study investigates the relations between variables characteristics of student and school with the student performances. The analysis used descriptive statistics of the variables for each of the three years. We complete the analysis by doing some multilevel multivariate analysis. The results of this study indicated that there is a relative stability in student performances over the three years with a slight improvement in 2007 which continue in 2008. Students from the private sector performed better than those of the public sector. The girls continuously have better results than the boys over the three years period. Students with French as their mother’s tongue do have better results than those with mother’s tongue different than French. Nevertheless, some nuances are necessary for better understanding the interpretation of the results. Moreover, the part of variation in student performances is responsible for about 75,0 % to student level and for about 25,0 % to school level. We note some differential effects of student and school factors on the criteria considered. The student variables are responsible for about 13,9 % of the total variance in syntax and punctuation and for about 9,8 % in spelling. School variables are responsible for 3,7 % for the total variance in student performance for syntax and punctuation and for about 6,4 % for spelling. Some school factors, like the size of the school, the mix sex school do not seem to have any significant link with students’ performances.

This study explores the relationship between students’ missing responses on a large-scale assessment and their cognitive skill profiles and characteristics. Data from the 48 multiple-choice items on the 2006 Ontario Secondary School Literacy Test (OSSLT), a high school graduation requirement, were analyzed using the item response theory (IRT) three-parameter logistic model and the Reduced Reparameterized Unified Model, a Cognitive Diagnostic Model. Missing responses were analyzed by item and by student. Item-level analyses examined the relationships among item difficulty, item order, literacy skills targeted by the item, the cognitive skills required by the item, the percent of students not answering the item, and other features of the item. Student-level analyses examined the relationships among students’ missing responses, overall performance, cognitive skill mastery profiles, and characteristics such as gender and home language. Most students answered most items: no item was answered by fewer than 98.8% of the students and 95.5% of students had 0 missing responses, 3.2% had 1 missing response, and only 1.3% had more than 1 missing responses). However, whether students responded to items was related to the student’s characteristics, including gender, whether the student had an individual education plan and language spoken at home, and to the item’s characteristics such as item difficulty and the cognitive skills required to answer the item. Unlike in previous studies of large-scale assessments, the missing response rates were not higher for multiple-choice items appearing later in the timed sections. Instead, the first two items in some sections had higher missing response rates. Examination of the student-level missing response rates, however, showed that when students had high numbers of missing responses, these often represented failures to complete a section of the test. Also, if nonresponse was concentrated in items that required particular skills, the accuracy of the estimates for those skills was lower than for other skills. The results of this study have implications for test designers who seek to improve provincial large-scale assessments, and for teachers who seek to help students improve their cognitive skills and develop test taking strategies.