The purpose of this study is to examine the validity and reliability of summative assessment of EFL student writing in a Swedish context. Three teachers have assessed the same four student essays from the English 6 course in Swedish upper secondary school. In addition to grading each essay, the teachers have indicated the extent of conformity to the grading criteria in terms of flow, structure, cohesion, adaptation to purpose, clarity, and variation. The analyzed data show a variation in assessment criteria interpretation, affecting assessment validity and reliability, and questioning the assessment equivalence of the Swedish criterion-referenced gradingsystem. / Syftet med denna studie är att undersöka validiteten och reliabiliteten hos summativa bedömningar av studentuppsatser ur ett svenskt perspektiv. Tre lärare har bedömt samma fyra studentuppsatser från engelska 6. Förutom att betygsätta varje uppsats har lärarna angett graden av överensstämmelse med betygskriterierna: flöde, struktur, sammanhållning, anpassning till syfte, tydlighet och variation. De analyserade data visar en variation i tolkning av betygskriterier, vilket påverka rbedömningens validitet och reliabilitet och ifrågasätter bedömningslikvärdigheten i det svenska mål-relaterade betygssystemet.

No description available.

The objective of the research was to investigate an improvement in job performance when role-play is used as a training and assessment tool in a sales call centre environment. The research was conducted by means of a competence assessment used at different stages (Quality Assurance Performance Management Questionnaire). The final stage was two weeks after the learners who had qualified to be sales agents had entered a real working environment. At this final stage, calls were retrieved and rated against the Quality Assurance Performance Management Questionnaire (QAPMQ). A sample of 40 novice sales agents (learners) were selected and randomly divided into the control and experimental groups. The research findings indicated that if planned well, role-play can be an effective training and assessment tool. / Industrial & Organisational Psychology / M.Comm. (Industrial & Organisational Psychology)

Au cours des dernières années, les pratiques de l’évaluation comme aide à l’apprentissage (c’est-à-dire l’auto-évaluation, l’évaluation par les pairs, la rétroaction) dans la salle de classe ont été de plus en plus considérées comme des éléments essentiels (Black & Wiliam, 1998; Harlen & Winter, 2004; van de Watering & van der Rijt, 2006). Cependant, dans le domaine de l’apprentissage d’une langue seconde la recherche sur ce sujet est plutôt limitée. En nous fondant sur les études de Colby-Kelly et Turner (2007) et de Lyster et Ranta (1997), nous avons mené une recherche exploratoire visant à combler ce besoin. L’objectif général était de comprendre comment l’évaluation formative se réalise dans deux cours d’anglais intermédiaire à l’Université de Montréal, et de comparer comment ces pratiques sont perçues et vécues par la professeure et ses étudiants. Trois questions de recherche étaient posées: 1. Quelle est la nature de l’évaluation formative dans une salle de classe d’anglais langue seconde? 2. Est-ce que les pratiques de la professeure reflètent ce qu’elle pense de l’évaluation formative? 3. Quels sont les correspondances et différences entre les croyances de la professeure et les perceptions de ses étudiants quant aux bénéfices de l’évaluation formative pour l’apprentissage de l’anglais langue seconde? La collecte de données comprend des observations en classe, des entrevues auprès de la professeure et des questionnaires pour les étudiants. Pour répondre à notre première question de recherche, nous avons analysé la rétroaction donnée par la professeure pendant une production orale réalisée par ses étudiants à partir de la typologie de feedback et de uptake de l’étude de Lyster et Ranta (1997). En ce qui a trait à la deuxième question de recherche, nous avons fait des entrevues avec la professeure en vue d’expliciter ses perceptions relativement à l’évaluation formative. À la fin du trimestre, nous avons comparé ses réponses avec sa pratique à partir des enregistrements vidéo produits en classe. Finalement, pour répondre à notre dernière question de recherche nous avons comparé les réponses données par la professeure aux entrevues avec les réponses des étudiants à un questionnaire adapté à partir de celui de Colby-Kelly et Turner. Finalement, nous présentons et discutons les résultats les plus significatifs obtenus dans cette étude qualitative Nous concluons cette thèse en proposant de avenues pour des recherches futures. / During the last twenty years, assessment practices for improving student learning (i.e., self-assessment, peer assessment, feedback) in the classroom has been considered as essential (Black & Wiliam, 1998; Harlen & Winter, 2004; van de Watering & van der Rijt, 2006). In the field of second language learning, however, research in this area is quite limited. In order to address this gap, an exploratory research , based on Colby-Kelly and Turner’s (2007) and Lyster and Ranta’s (1997)studies has been conducted . The general objective was to understand how formative assessment is practiced in two Intermediate Oral English courses at the Université de Montréal and to compare how these practices are perceived and performed by the teacher and the students. Three research questions were pursued: 1. What is the nature of formative assessment in a second language classroom setting? 2. Do the teacher’s assessment practices reflect what she thinks about formative assessment? 3. What are the coincidences and differences between teacher’s perceptions and her students’ perceptions regarding the benefits of formative assessment for learning English? Data collection instruments consist of teacher interview guidelines, students’ questionnaire and classroom observation grids. In order to answer the first question, the feedback given by the teacher during the students’ oral performance has bee analysed using the types of feedback and uptake in Lyster and Ranta’s (1997) report. For the second research question, I interviewed the teacher at the beginning of each session and I elicited her beliefs about classroom-based formative assessment practice. At the end of the session I the teacher’s answers have beeb compared and contrasted with her actual performance, which was videotaped during the course. Finally, regarding the third question, teacher’s answers in the interviews have been compared with students’ answers on a questionnaire – adapted from Colby-Kelly and Turner’s one. The most significant results of this qualitative research are presented and discussed. In the conclusion, directions for future research are proposed

Genomförda undersökningar visar att formativ bedömning förbättrar elevers inlärning och förståelse. Den svenska läroplanen från 2011 ger tydliga indikationer på att formativ bedömning ska inkluderas i undervisningen. Vår diskussion i denna uppsats är därför aktuell och den visar hur forskare artikulerar fenomenet formativ bedömning. Litteraturstudien inkluderar information om vad formativ bedömning innebär och förslag på hur man kan arbeta utifrån ett sådant arbetssätt. Vår studie visar att de viktigaste komponenterna inom formativ bedömning är att tydliggöra målen, återkoppling samt själv- och kamratbedömning. Beskrivningarna av dessa resulterar i en analys där vi diskuterar det formativa arbetssättet. Både positiva och negativa aspekter tas upp. Vårt syfte med denna uppsats är att införskaffa djupare kunskaper om formativ bedömning likväl för oss själva som för redan aktiva lärare. Vi har en föraning om att många lärare inte har tillräckliga kunskaper kring ämnet och vår förhoppning är att denna uppsats kan fungera som ett verktyg för lärare som arbetar med eller skulle vilja arbeta med formativ bedömning. / Previous research shows that formative assessments improve pupils’ learning and understanding. Also, the Swedish Curriculum 2011 gives clear guidance that formative assessment should be included in school. Our discussion in this essay is therefore needed and it deals with how researchers articulate the phenomenon formative assessment. The study includes information about what formative assessment means and suggestions for how to work with it are given. The main parts within formative assessment are to clarify learning goals, feedback and self- and peer assessment. The information about these main parts ends in an analysis where a discussion about formative assessment and its positive and negative effects takes place.   Furthermore, the purpose of this essay is to inherit deeper knowledge about formative assessment as well for ourselves as for teachers already on duty. We anticipate that many teachers do not have sufficient knowledge about formative assessment and our expectation is that this essay can function as a tool for teachers who work or would like to work with formative assessment.

Au cours des dernières années, les pratiques de l’évaluation comme aide à l’apprentissage (c’est-à-dire l’auto-évaluation, l’évaluation par les pairs, la rétroaction) dans la salle de classe ont été de plus en plus considérées comme des éléments essentiels (Black & Wiliam, 1998; Harlen & Winter, 2004; van de Watering & van der Rijt, 2006). Cependant, dans le domaine de l’apprentissage d’une langue seconde la recherche sur ce sujet est plutôt limitée. En nous fondant sur les études de Colby-Kelly et Turner (2007) et de Lyster et Ranta (1997), nous avons mené une recherche exploratoire visant à combler ce besoin. L’objectif général était de comprendre comment l’évaluation formative se réalise dans deux cours d’anglais intermédiaire à l’Université de Montréal, et de comparer comment ces pratiques sont perçues et vécues par la professeure et ses étudiants. Trois questions de recherche étaient posées: 1. Quelle est la nature de l’évaluation formative dans une salle de classe d’anglais langue seconde? 2. Est-ce que les pratiques de la professeure reflètent ce qu’elle pense de l’évaluation formative? 3. Quels sont les correspondances et différences entre les croyances de la professeure et les perceptions de ses étudiants quant aux bénéfices de l’évaluation formative pour l’apprentissage de l’anglais langue seconde? La collecte de données comprend des observations en classe, des entrevues auprès de la professeure et des questionnaires pour les étudiants. Pour répondre à notre première question de recherche, nous avons analysé la rétroaction donnée par la professeure pendant une production orale réalisée par ses étudiants à partir de la typologie de feedback et de uptake de l’étude de Lyster et Ranta (1997). En ce qui a trait à la deuxième question de recherche, nous avons fait des entrevues avec la professeure en vue d’expliciter ses perceptions relativement à l’évaluation formative. À la fin du trimestre, nous avons comparé ses réponses avec sa pratique à partir des enregistrements vidéo produits en classe. Finalement, pour répondre à notre dernière question de recherche nous avons comparé les réponses données par la professeure aux entrevues avec les réponses des étudiants à un questionnaire adapté à partir de celui de Colby-Kelly et Turner. Finalement, nous présentons et discutons les résultats les plus significatifs obtenus dans cette étude qualitative Nous concluons cette thèse en proposant de avenues pour des recherches futures. / During the last twenty years, assessment practices for improving student learning (i.e., self-assessment, peer assessment, feedback) in the classroom has been considered as essential (Black & Wiliam, 1998; Harlen & Winter, 2004; van de Watering & van der Rijt, 2006). In the field of second language learning, however, research in this area is quite limited. In order to address this gap, an exploratory research , based on Colby-Kelly and Turner’s (2007) and Lyster and Ranta’s (1997)studies has been conducted . The general objective was to understand how formative assessment is practiced in two Intermediate Oral English courses at the Université de Montréal and to compare how these practices are perceived and performed by the teacher and the students. Three research questions were pursued: 1. What is the nature of formative assessment in a second language classroom setting? 2. Do the teacher’s assessment practices reflect what she thinks about formative assessment? 3. What are the coincidences and differences between teacher’s perceptions and her students’ perceptions regarding the benefits of formative assessment for learning English? Data collection instruments consist of teacher interview guidelines, students’ questionnaire and classroom observation grids. In order to answer the first question, the feedback given by the teacher during the students’ oral performance has bee analysed using the types of feedback and uptake in Lyster and Ranta’s (1997) report. For the second research question, I interviewed the teacher at the beginning of each session and I elicited her beliefs about classroom-based formative assessment practice. At the end of the session I the teacher’s answers have beeb compared and contrasted with her actual performance, which was videotaped during the course. Finally, regarding the third question, teacher’s answers in the interviews have been compared with students’ answers on a questionnaire – adapted from Colby-Kelly and Turner’s one. The most significant results of this qualitative research are presented and discussed. In the conclusion, directions for future research are proposed

Online examinations are an integral component of many online learning environments and a high-stake process for students, teachers and educational institutions. They are the target of many security threats, including intrusion by hackers and collusion. Collu-sion happens when a student invites a third party to impersonate him/her in an online test, or to abet with the exam questions. This research proposed a profile-based chal-lenge question approach to create and consolidate a student's profile during the learning process, to be used for authentication in the examination process. The pro-posed method was investigated in six research studies using a usability test method and a risk-based security assessment method, in order to investigate usability attributes and security threats. The findings of the studies revealed that text-based questions are prone to usability issues such as ambiguity, syntactic variation, and spelling mistakes. The results of a usability analysis suggested that image-based questions are more usable than text-based questions (p < 0.01). The findings identified that dynamic profile questions are more efficient and effective than text-based and image-based questions (p < 0.01). Since text-based questions are associated with an individual's personal information, they are prone to being shared with impersonators. An increase in the numbers of chal-lenge questions being shared showed a significant linear trend (p < 0.01) and increased the success of an impersonation attack. An increase in the database size decreased the success of an impersonation attack with a significant linear trend (p < 0.01). The security analysis of dynamic profile questions revealed that an impersonation attack was not successful when a student shared credentials using email asynchronously. However, a similar attack was successful when a student and impersonator shared information in real time using mobile phones. The response time in this attack was significantly different when a genuine student responded to his challenge questions (p < 0.01). The security analysis revealed that the use of dynamic profile questions in a proctored exam can influence impersonation and abetting. This view was supported by online programme tutors in a focus group study.

The objective of the research was to investigate an improvement in job performance when role-play is used as a training and assessment tool in a sales call centre environment. The research was conducted by means of a competence assessment used at different stages (Quality Assurance Performance Management Questionnaire). The final stage was two weeks after the learners who had qualified to be sales agents had entered a real working environment. At this final stage, calls were retrieved and rated against the Quality Assurance Performance Management Questionnaire (QAPMQ). A sample of 40 novice sales agents (learners) were selected and randomly divided into the control and experimental groups. The research findings indicated that if planned well, role-play can be an effective training and assessment tool. / Industrial and Organisational Psychology / M.Comm. (Industrial & Organisational Psychology)

The purpose of this study was to examine the ways by which student learning improvement and the self-regulation of learning are possible through the use of quality formative assessment in the teaching of a university course. In recent years, researchers in educational assessment are showing an increased interest to the improvement of learning resulting from the use of formative assessment. Formative assessment is generally recognised as an improvement oriented assessment. It is believed to result in instructional effectiveness. Quality formative assessment includes formative feedback, self-assessment and peer assessment. Previous studies show the contribution of each of these quality formative assessments to learning improvement. However, less attention has been given to studying the combined effect of quality formative assessments on learning improvement. On the other hand, the predominant use of summative assessment remains a challenge to the improvement in instruction. Thus, the question was to determine the extent to which the use of quality formative assessment improves learning. The literature review in this study show an over reliance upon summative assessment in the context of higher education classrooms. There is also recognition in that formative assessment improves learning and enhances self-regulation. This study followed a mixed-methods research design of the type partially mixed sequential and applied a quasi-experimental intervention, where the educators used quality formative assessment on lessons with the students in the intervention group. The quasiexperimentation was implemented with 378 (214 male and 164 female) first year students of three universities enrolled for “General Psychology” course and six educators who were teaching the course. The students in this study were taken from intact classes, because this is possible in quasi experimental research. Data for the quantitative part of the study were generated using a structured questionnaire and achievement tests. Interviews with the educators, focus group discussions with the students, and classroom observations were used to generate data for the qualitative phase of study. The pretestposttest scores as well as the students’ perceptions on self-regulating learning were compared between the intervention (N =191) and the comparison (N = 187) groups. The quantitative analysis used different inferential statistics, which proved the presence of statistically significant variations between the intervention and comparison groups for the outcome measures (posttest achievement and perception about self-regulating learning). Although the qualitative study showed the presence of positive perceptions towards quality formative assessment, the practice was found to be inconsistent. Perhaps, this may be because of the predominantly summative assessment tradition and the reluctance to use quality formative assessment. Finally, recommendations were made to promote the use of quality formative assessment aiming at the improvement and the self-regulation on learning. / Curriculum and Instructional Studies / D. Ed. (Didactics)

Academics encounter problems with the selection, evaluation, testing and implementation of e-assessment software tools. The researcher experienced these problems while adopting e-assessment at the university where she is employed. Hence she undertook this study, which is situated in schools and departments in Computing-related disciplines, namely Computer Science, Information Systems and Information Technology at South African Higher Education Institutions. The literature suggests that further research is required in this domain. Furthermore, preliminary empirical studies indicated similar disabling factors at other South African tertiary institutions, which were barriers to long-term implementation of e-assessment. Despite this, academics who are adopters of e-assessment indicate satisfaction, particularly when conducting assessments with large classes. Questions of the multiple choice genre can be assessed automatically, leading to increased productivity and more frequent assessments. The purpose of this research is to develop an evaluation framework to assist academics in determining which e-assessment tool to adopt, enabling them to make more informed decisions. Such a framework would also support evaluation of existing e-assessment systems. The underlying research design is action research, which supported an iterative series of studies for developing, evaluating, applying, refining, and validating the SEAT (Selecting and Evaluating an e-Assessment Tool) Evaluation Framework and subsequently an interactive electronic version, e-SEAT. Phase 1 of the action research comprised Studies 1 to 3, which established the nature, context and extent of adoption of e-assessment. This set the foundation for development of SEAT in Phase 2. During Studies 4 to 6 in Phase 2, a rigorous sequence of evaluation and application facilitated the transition from the manual SEAT Framework to the electronic evaluation instrument, e-SEAT, and its further evolution. This research resulted in both a theoretical contribution (SEAT) and a practical contribution (e-SEAT). The findings of the action research contributed, along with the literature, to the categories and criteria in the framework, which in turn, contributed to the bodies of knowledge on MCQs and e-assessment. The final e-SEAT version, the ultimate product of this action research, is presented in Appendix J1. For easier reference, the Appendices are included on a CD, attached to the back cover of this Thesis.. / Computing / PhD. (Information Systems)