Introdução: Feedback é uma ferramenta de ensino poderosa. Para otimizar o feedback em avaliações tipo OSCE (Objective Structured Clinical Examination), uma modificação no papel do avaliador, apelidada de \"avaliador sombra\", foi testada. Os \"avaliadores sombras\" acompanham os estudantes por todas as estações realizando todas as avaliações. Ao término da atividade, realizam feedback do desempenho clínico aos estudantes. Objetivos: Descrever e discutir a experiência na aplicação deste modelo OSCE modificado, avaliar as opiniões dos alunos em relação a este novo método, e apresentar um estudo onde foram realizadas comparações entre o \"avaliador sombra\" e o formato original do examinador OSCE, nomeado de \"avaliador fixo\", com o objetivo de responder se este novo formato de avaliação apresenta vieses em relação ao formato original. Metodologia: Em 2011 foram realizados experimentos com as modificações no papel do avaliador para definir o formato final do \"avaliador sombra\". No período compreendido de fevereiro de 2012 a maio de 2014 foi realizado um protocolo de pesquisa com 415 estudantes do 6º ano de medicina. Destes, 316 estudantes foram randomizados para serem avaliados por avaliadores \"sombras\" e \"fixos\". Para analisar as comparações entre os avaliadores \"sombras\" e \"fixos\", foram utilizados o coeficiente de correlação de Pearson com regressão linear, teste t de Student e gráficos de Bland-Altman. Para aprofundar estas análises, os itens dos checklists foram classificados por domínios para realização de comparações mais específicas. Considerando que esta foi a primeira vez em que esta nova estratégia de avaliador (\"sombra\") foi aplicada, foram realizados questionários de opinião aos 415 estudantes que participaram do OSCE com o \"avaliador sombra\". Resultados: Foram encontradas altas e significativas correlações entre as notas globais por estação dadas aos estudantes pelos avaliadores \"sombras\" e \"fixos\",r = 0,87 (0,85 - 0,89; p < 0,05). Nas análises por domínios (afetivo, psicomotor e cognitivo), os resultados demonstraram maiores correlações entre as notas no domínio cognitivo e menores no afetivo. Não houve diferenças significativas comparando as médias das notas dadas pelos examinadores em relação às notas globais e nos domínios psicomotor e cognitivo. Análises de Teste t e de Bland-Altman demonstraram que as notas dadas aos estudantes no domínio afetivo foram maiores pelos \"avaliadores sombras\", mas a magnitude desta diferença foi muito pequena. De acordo com os questionários aplicados aos 415 estudantes, o feedback realizado pelos \"avaliadores sombras\" contribui para o aprimoramento de habilidades. Mais do que 90% dos estudantes concordaram que o feedback realizado pelo \"avaliador sombra\" é mais efetivo do que outras estratégias de feedback realizadas em outras aplicações de OSCE durante o curso de graduação de medicina até aquele momento. Conclusão: As modificações no papel do avaliador, \"avaliador sombra\", não levaram a vieses significativos nas notas dos estudantes quando comparado ao modelo de avaliador original do OSCE. Esta nova estratégia pode fornecer importantes melhorias na avaliação formativa de competências clínicas / Background: Feedback is a powerful learning tool. To improve the opportunity for feedback provided during an Objective Structured Clinical Examination (OSCE), a modification of the examiner\'s role nicknamed the \"shadow examiner\" was tested. In this new strategy, examiners follow the students across all OSCE stations, and may provide important insights for formative assessments of clinical performance Objective: Describe and discuss the experience in applying this modified OSCE, evaluate students\' opinions regarding this new method, and present an experience to analyze comparisons between the \"shadow examiner\" and original examiner OSCE format, nicknamed \"fixed examiner\". The objective of this study was to answer if this new assessment format presents biases in relation to the original format. Methods: In 2011, some experiments were carried out with modifications in the examiner\'s role to define the format of \"shadow examiner\". From February 2012 to May 2014, the research protocol was performed with 415 6th year medical students. Of these, 316 were randomly assigned to assessments made up of \"shadow\" and \"fixed\" examiners. Pearson correlation coefficients with linear regression, Student\'s t-test analysis, Bland-Altman plots were the statistical methods used to compare the assessment modes. To strengthen the analysis, checklist items were classified by domains in order to make specific comparisons. Considering that this was the first application of this new examiner (\"shadow\") strategy was applied, were conducted opinion surveys of the 415 students who participated in the OSCE with the \"shadow examiner\". Results: High correlations for global scores between \"shadow\" and \"fixed\" examiners were observed, r = 0,87 (0,85 - 0,89; p < 0,05). Analysing specific domains (affective, psychomotor and cognitive), the results demonstrated higher correlations for cognitive scores and lower correlations for affective scores. No statistically significant differences between mean examiners\' global scores, psychomotor and cognitive domains were found. T-test and Bland-Altman reviews showed that affective scores from \"shadow examiners\" were significantly higher than those from \"fixed examiners\", but the magnitude of this difference was small. According to the questionnaires applied to the 415 students, feedback from \"shadow examiners\" contributes to skills training, and more than 90% of these students agree that feedback from \"shadow examiners\" is more effective than other feedback formats achieved in other OSCE applications during the medical graduation course up to that time. Conclusion: The modifications in the role of examiner, \"shadow examiner\", did not lead to any important bias in the students\' scores comparing with the original examiner strategy of OSCE. This new strategy may provide important insights for formative assessments of clinical performance

Despite widespread recognition by science educators, researchers and K-12 frameworks that scientific inquiry should be an essential part of science education, typical classrooms and assessments still emphasize rote vocabulary, facts, and formulas. One of several reasons for this is that the rigorous assessment of complex inquiry skills is still in its infancy. Though progress has been made, there are still many challenges that hinder inquiry from being assessed in a meaningful, scalable, reliable and timely manner. To address some of these challenges and to realize the possibility of formative assessment of inquiry, we describe a novel approach for evaluating, tracking, and scaffolding inquiry process skills. These skills are demonstrated as students experiment with computer-based simulations. In this work, we focus on two skills related to data collection, designing controlled experiments and testing stated hypotheses. Central to this approach is the use and extension of techniques developed in the Intelligent Tutoring Systems and Educational Data Mining communities to handle the variety of ways in which students can demonstrate skills. To evaluate students' skills, we iteratively developed data-mined models (detectors) that can discern when students test their articulated hypotheses and design controlled experiments. To aggregate and track students' developing latent skill across activities, we use and extend the Bayesian Knowledge-Tracing framework (Corbett & Anderson, 1995). As part of this work, we directly address the scalability and reliability of these models' predictions because we tested how well they predict for student data not used to build them. When doing so, we found that these models demonstrate the potential to scale because they can correctly evaluate and track students' inquiry skills. The ability to evaluate students' inquiry also enables the system to provide automated, individualized feedback to students as they experiment. As part of this work, we also describe an approach to provide such scaffolding to students. We also tested the efficacy of these scaffolds by conducting a study to determine how scaffolding impacts acquisition and transfer of skill across science topics. When doing so, we found that students who received scaffolding versus students who did not were better able to acquire skills in the topic in which they practiced, and also transfer skills to a second topic when was scaffolding removed. Our overall findings suggest that computer-based simulations augmented with real-time feedback can be used to reliably measure the inquiry skills of interest and can help students learn how to demonstrate these skills. As such, our assessment approach and system as a whole shows promise as a way to formatively assess students' inquiry.

Despite widespread recognition by science educators, researchers and K-12 frameworks that scientific inquiry should be an essential part of science education, typical classrooms and assessments still emphasize rote vocabulary, facts, and formulas. One of several reasons for this is that the rigorous assessment of complex inquiry skills is still in its infancy. Though progress has been made, there are still many challenges that hinder inquiry from being assessed in a meaningful, scalable, reliable and timely manner. To address some of these challenges and to realize the possibility of formative assessment of inquiry, we describe a novel approach for evaluating, tracking, and scaffolding inquiry process skills. These skills are demonstrated as students experiment with computer-based simulations. In this work, we focus on two skills related to data collection, designing controlled experiments and testing stated hypotheses. Central to this approach is the use and extension of techniques developed in the Intelligent Tutoring Systems and Educational Data Mining communities to handle the variety of ways in which students can demonstrate skills. To evaluate students' skills, we iteratively developed data-mined models (detectors) that can discern when students test their articulated hypotheses and design controlled experiments. To aggregate and track students' developing latent skill across activities, we use and extend the Bayesian Knowledge-Tracing framework (Corbett & Anderson, 1995). As part of this work, we directly address the scalability and reliability of these models' predictions because we tested how well they predict for student data not used to build them. When doing so, we found that these models demonstrate the potential to scale because they can correctly evaluate and track students' inquiry skills. The ability to evaluate students' inquiry also enables the system to provide automated, individualized feedback to students as they experiment. As part of this work, we also describe an approach to provide such scaffolding to students. We also tested the efficacy of these scaffolds by conducting a study to determine how scaffolding impacts acquisition and transfer of skill across science topics. When doing so, we found that students who received scaffolding versus students who did not were better able to acquire skills in the topic in which they practiced, and also transfer skills to a second topic when was scaffolding removed. Our overall findings suggest that computer-based simulations augmented with real-time feedback can be used to reliably measure the inquiry skills of interest and can help students learn how to demonstrate these skills. As such, our assessment approach and system as a whole shows promise as a way to formatively assess students' inquiry.

This study, based on interviews with seven teachers of Swedish, investigates how they define the concept of text interpretation and how they assess pupils’ ability to interpret text. The results show that the teachers define text interpretation as an ability to perceive hidden messages in text and an ability that is closely connected to reading comprehension. This description agrees with the way text interpretation is explained in previous research but not with the description in the curriculum. This is a defect, since teachers ought to know what they are supposed to assess, based on the steering documents they are expected to follow in their professional practice. Here, however, the teachers’ assessment skills are shown to be an asset. When teaching literature the teachers say that they normally use dialogue and written assignments in which pupils reason on the basis of the content of the text and their personal experience. The forms of assessment are summative and formal, although it seems difficult to use fixed answer templates for reasoning in contexts like this where it is not clear what can be said to be right or wrong. Summative assessment, however, seems more reliable and credible than formative assessment. A picture thus emerges of the difficulty of assessing text interpretation which is simultaneously multivalent and reliable. / Denna studie är grundad på intervjuer av 7 svensklärare i syfte att undersöka hur de definierar begreppet texttolkning och hur de bedömer elevers förmåga att tolka text. I resultatet framgår att lärarna definierar texttolkning som en förmåga att uppfatta dolda budskap i text och en förmåga som är nära sammankopplad med läsförståelse. Denna beskrivning stämmer väl överens med hur texttolkning förklaras i tidigare forskning men inte med den beskrivning som framkommer i läroplanen. Detta är en brist då lärare bör veta vad de ska bedöma, utifrån styrdokument de ska utgå från i sin yrkesutövning. Här framstår dock lärares bedömningsfärdighet som en tillgång. Vid litteraturundervisning säger lärarna att de vanligen tillämpar dialog och skriftliga inlämningsuppgifter, med resonemang utifrån både textens innehåll och personliga erfarenheter. Bedömningsformerna är summativa och formella, även om fasta svarsmallar ter sig svåra att använda vid resonemang, då det i dessa sammanhang inte tydligt framgår något som med säkerhet kan sägas vara rätt eller fel. Summativ bedömning ter sig dock mer tillförlitlig och trovärdig vid en bedömningssituation, jämfört med formativ. Här framkommer således en bild av att det är svårt att bedöma texttolkning som på samma gång är flertydig och tillförlitlig.

The democratic era in South Africa has led to the introduction of a wide-ranging series of teaching and learning policies aimed at school reform. The study aimed at establishing how these policies are implemented by educators in selected schools. Qualitative and quantitative methods were used to gather information from educators, members of the School Management Teams and learners through in-depth individual and focus group interviews as the main data collection methods. Critical discourse analysis was used to examine participants‟ spoken words and the content of relevant documents such as lesson plans in detail. Comparisons were drawn and similarities identified through the coding, categorisation and condensation of data. The researcher interpreted the displayed data. A discussion of the main themes was presented and supported by quotations by participants. Findings indicate that there have been only few changes in terms of how educators teach; some educators conceded that they had not changed the way they teach since the introduction of new curriculum and assessment policies. The role played by learning outcomes in teachers‟ lesson planning was uneven. Members of School Management Teams demonstrated diverse views on teaching and learning policies. Learners who are taught in their home language encounter few difficulties in learning; this is not the case with learners who are taught through medium of a second language and thus lack the necessary language proficiency necessary for academic achievement. A shift from teacher-centred to learner-centred approaches to teaching is still required: that is, a shift from teaching to learning to enhance thinking and reflection which accommodates and draws on learners‟ prior knowledge and experiences. Class sizes should be reduced to enable effective educator and learner interaction. Educators‟ interest, tolerance, innovativeness and competency in the subject matter will help sustain learners‟ interest in the learning activities. Policy makers should explore professional development from viewpoint of the participating educators in order to identify the most effective strategies to support and change educators‟ classroom practice where necessary. / Curriculum and Instructional Studies / D. Ed. (Curriculum Studies)

The purpose of the present correlational, ex post facto study was to evaluate the predictive ability and academic achievement criterion outcomes of two district-developed interim mathematics assessment programs for a sample of 5,801 grade 6 students in a large urban school district. Average scores for both interim assessment types were statistically significantly more related to 2013 FCAT 2.0 scores (r = .75 and .72; p < .001) than all other predictors (i.e., student demographics, Florida school grade, and student course GPA) except for 2012 FCAT 2.0 scores (r = .78; p < .001). Further, the newer interim assessment program with an instructional purpose and curriculum-based sequencing had slightly stronger overall predictive power (rs = .88) and a higher criterion mean score (M = 218.08) than the older, state-test mirror interim assessment program (rs = .85; M = 215.47). Regression models by prior year FCAT 2.0 Achievement Level yielded some predictor ranking discrepancies by prior achievement level. Although not statistically significant at the .01 level, groups of students with a more moderate total number of interim assessments outperformed groups with all or nothing. Overall, the two types of interim assessment programs evaluated in the present study were good predictors of the state high-stakes test, 2012 Grade 6 Mathematics FCAT 2.0. However, more research must be done to identify with certainty whether or not the act of taking the interim tests and receiving feedback contributes to improved student learning.

The aims of this Master's thesis are (i) to examine the suitability of using peer-to-peer assessment in teaching writing skills in Czech B1-level courses and (ii) to suggest recommendations for tutors who are considering implementing the method in their Czech classes. The theoretical part is concerned with the description of particular types and forms of assessment and it presents an overview of several studies, focusing on those conducted in the field of ESL teaching. The empirical part focuses on describing the design and execution of the experiment as well as on interpreting its results. The experiment was performed as an intensive online writing course in Czech; the participants were students preparing for a B1-level exam. The investigation aimed at the quality of students' comments and at the implementation degree of their suggestions shows limited effectivity of using peer-to-peer assessment in teaching Czech on B1-level. However, based on the data, the study managed to identify several shortcomings in the way the peer-to-peer assessment method was used and it suggested recomendations for their elimination. If the tutors decide to follow these suggestions this could lead to a more succesful use of the investigated method in teaching.

This dissertation thesis is focused on the theme of formative assessment on a secondary level of education. The aim of this thesis is to define formative assessment and to show how it is implemented to the teaching process of the Czech language and literature on a secondary level of education. The theoretical part presents a review based on major international and national sources. Formative assessment is defined as an evaluation which brings valuable information concerning the knowledge and skills of the given pupil during the process of education. The typology and function of formative assessment, questions linked to its successful implementation are elaborated together with this methods: goals, criteria, feedback, self-assessment and peer assessment. The aim of the empirical part is to define the real form of the implementation of formative assessment in six targeted Czech schools. This thesis also presents the attitude of students towards the implementation of this method. A qualitative research method was chosen and a multiple case study was used as a form of research strategy. The data collection consisted of observation, document analysis and interviews with teachers and pupils. The formative assessment methods were best implemented under the supervision of a skilled mentor. Contrary to...

The focus of this research was to investigate the use of a district created formative benchmark assessment in reading to predict student achievement for 10th-grade students on the Florida Comprehensive Assessment Test (FCAT) in one county in north central Florida. The purpose of the study was to provide information to high school principals and teachers to better understand how students were performing and learning and to maximize use of the formative district benchmark assessment in order to modify instruction and positively impact student achievement. This study expanded a prior limited study which correlated district benchmark assessment scores to FCAT scores for students in grades three through five in five elementary schools in the targeted county. The high correlations suggested further study. This research focused on secondary reading, specifically in 10th grade where both state and targeted county FCAT scores were low in years preceding this research. Investigated were (a) the district formative assessment in reading as a predictor of FCAT Reading scores, (b) differences in strength of correlation and prediction among student subgroups and between high schools, and (c) any relationships between reading formative assessment scores and Mathematics FCAT scores. An additional focus of this study was to determine best leadership practices in schools where there were the highest correlations between the formative assessment and FCAT Reading scores. Research on best practices was reviewed, and principals were interviewed to determine trends and themes in practice. Tenth grade students in the seven Florida targeted district high schools were included in the study. The findings of the study supported the effective use of formative assessments both in instruction and as predictors of students' performance on the FCAT. The results of the study also showed a significant correlation between performance on the reading formative assessment and performance on FCAT Mathematics. The data indicated no significant differences in the strength of correlation between student subgroups or between the high schools included in the study. Additionally, the practices of effective principals in using formative assessment data to inform instruction, gathered through personal interviews, were documented and described.

Le présent mémoire explore l’enseignement en classe inversée, tel que rapporté par 25 enseignants de niveau collégial et universitaire au Québec en 2017-2018. L’étude décrite dans ce mémoire documente l’usage de la classe inversée, dans le but d’en favoriser l’usage par la communauté enseignante. Avec le développement rapide de ressources éducatives en ligne en 2020-2021, le contexte technopédagogique est favorable à un plus grand usage de la classe inversée à tous les niveaux de l’éducation. La classe inversée est étudiée comme piste d’amélioration de la réussite scolaire au postsecondaire, puisque la réussite scolaire est une problématique sociale d’actualité qui a des répercussions sur la société, les établissements et les individus. À l’aide d’approches pédagogiques recourant à l’apprentissage actif, les enseignants peuvent concevoir des activités motivantes et engageantes pour les étudiants ce qui a un effet positif sur la motivation. L’approche de la classe inversée, devenue très populaire au postsecondaire, exploite les avantages motivationnels connu de l’apprentissage actif sans sacrifier la capacité à transmettre beaucoup de contenus théoriques durant une session de cours. Le présent mémoire fait l’inventaire des pratiques technopédagogiques en classe inversée et comporte 3 angles d’analyse : l’adoption de la classe inversée, les pratiques d’enseignement et la gestion de classe. Nous avons recours à un modèle de développement professionnel afin de comprendre le contexte de la classe inversée, ses particularités et l’évolution des pratiques des répondants. Ces questions spécifiques sont répondues à l’aide d’une approche où l’on compare 25 différents cas d’usage de la classe inversée pour en faire émerger les convergences. Des entrevues semi-dirigées d’une heure et demie sont analysées à trois reprises pour répondre spécifiquement à ces trois questions, permettant une triangulation par l’analyse qui contribue à dresser un portrait compréhensif de la classe inversée. Premièrement les résultats démontrent que l’adoption de la classe inversée est faite par l’accumulation des pratiques d’enseignement qui la composent au fil du temps. D’un point de départ magistrocentré, on constate que cette accumulation débute plus souvent par l’adoption de nouveaux dispositifs technologiques, puis ensuite de nouvelles approches pédagogiques, plutôt que l’adoption des pédagogies actives, précédant les dispositifs technologiques (Figure 6 à Figure 8). Deuxièmement les résultats permettent d’observer que les pratiques d’enseignement de chaque phase de la séquence pédagogique sont régies par des logiques de conception pédagogique sous-jacentes propres à la classe inversée. La préparation et l’amorce d’un cours sont complémentaires et jouent le rôle de l’acquisition et de la consolidation-réactivation des acquis. Les activités sont davantage magistrocentré et médiatisés hors du cours. Les activités d’apprentissage actif et l’enseignement magistrocentré sont complémentaires en classe, cependant l’enseignement est davantage pédocentré et moins médiatisé hors de la classe. Finalement la conclusion des séances de classe permet généralement aux étudiants de recevoir une rétroaction (ANNEXE VIII). Troisièmement, les résultats indiquent que la gestion de classe en classe inversée se présente sous diverses formes mais favorise généralement le style interactionniste, avec quelques aspects plus interventionnistes ou plus non-interventionnistes (Figure 11). L’étude de la gestion de classe met en lumière les pratiques de contrôle comportemental des étudiants par l’évaluation en classe inversée, en plus de son rôle principal évaluatif et formatif (Figure 12). / This thesis explores the use of flipped classroom teaching, as reported by 25 college and university teachers in Quebec in 2017-2018. The study described in this thesis documents the use of the flipped classroom, with the aim of promoting its use by the teaching community. With the rapid development of online educational resources in 2020-2021, the technopedagogical context is favorable to greater use of the flipped classroom at all levels of education. The flipped classroom is being studied as an avenue for improving post-secondary academic success, since academic success is a topical social issue that has repercussions on society, institutions and individuals. Using active learning pedagogical approaches, teachers can design motivating and engaging activities for students which has a positive effect on motivation. The flipped classroom approach, which has become very popular in post-secondary education, harnesses the known motivational benefits of active learning without sacrificing the ability to convey a lot of theoretical content during a class session. This thesis takes stock of technopedagogical practices in the flipped classroom and includes 3 angles of analysis: its adoption, teaching practices and classroom management. We use a professional development framework to understand the context of the flipped classroom, its peculiarities and the evolution of respondent practices. These specific questions are answered using an approach where we compare 25 different use cases to bring out the convergences. One-and-a-half-hour semi-structured interviews are analyzed three times to specifically answer these three questions, allowing analytical triangulation that helps paint a comprehensive portrait of the flipped classroom. Firstly, the results show that the adoption of the flipped classroom is made by the accumulation of teaching practices that compose it over time. From a teacher-centered point of origin, this accumulation begins more often with the adoption of new technology, then new pedagogical approaches, rather than the adoption of active learning preceding the use of technologies (Figure 6 to Figure 8). Secondly, the results allow us to observe that the teaching practices of each phase of the teaching sequence are governed by underlying teaching design logics specific to the flipped classroom. The preparation and primer of a course are complementary and enable acquisition and consolidationreactivation of acquired knowledge. The activities are more lecture-oriented outside the course. Active learning activities and formal teaching are complementary in the classroom, however teaching is more student-centric in the classroom. Classes concludes generally with feedback (APPENDIX VIII). Thirdly and lastly, the results on classroom management states a variety of approaches but generally favors the interactionist style, with some more interventionist or non-interventionist aspects (Figure 11). Exploring classroom management sheds light on student behavioral control practices through assessments, in addition to its primary evaluative and formative role (Figure 12).