This dissertation explores the use of natural language processing (NLP) and large language models (LLMs) to analyze student self-regulated learning (SRL) strategies in response to exam wrappers. Exam wrappers are structured reflection activities that prompt students to practice SRL after they get their graded exams back. The dissertation consists of three manuscripts that compare traditional qualitative analysis with NLP-assisted approaches using transformer-based models including GPT-3.5, a state-of-the-art LLM. The data set comprises 3,800 student responses from an engineering physics course. The first manuscript develops two NLP-assisted codebooks for identifying learning strategies related to SRL in exam wrapper responses and evaluates the agreement between them and traditional qualitative analysis. The second manuscript applies a novel NLP technique called zero-shot learning (ZSL) to classify student responses into the codes developed in the first manuscript and assesses the accuracy of this method by evaluating a subset of the full dataset. The third manuscript identifies the distribution and differences of learning strategies and SRL constructs among students of different exam performance profiles using the results from the second manuscript. The dissertation demonstrates the potential of NLP and LLMs to enhance qualitative research by providing scalable, robust, and efficient methods for analyzing large corpora of textual data. The dissertation also contributes to the understanding of SRL in engineering education by revealing the common learning strategies, impediments, and SRL constructs that students report they use while preparing for exams in a first-year engineering physics course. The dissertation suggests implications, limitations, and directions for future research on NLP, LLMs, and SRL. / Doctor of Philosophy / This dissertation is about using artificial intelligence (AI) to help researchers and teachers understand how students learn from their exams. Exams are not only a way to measure what students know, but also a chance for students to reflect on how they studied and what they can do better next time. One way that students can reflect is by using exam wrappers, which are short questions that students answer after they get their graded exams back. A type of AI called natural language processing (NLP) is used in this dissertation, which can analyze text and find patterns and meanings in it. This study also uses a powerful AI tool called GPT-3.5, which can generate text and answer questions. The dissertation has three manuscripts that compare the traditional way of analyzing exam wrappers, which is done by hand, with the new way of using NLP and GPT-3.5, evaluate a specific promising NLP method, and use this method to try and gain a deeper understanding in students self-regulated learning (SRL) while preparing for exams. The data comes from 3,800 exam wrappers from a physics course for engineering students. The first manuscript develops a way of using NLP and GPT-3.5 to find out what learning strategies and goals students talk about in their exam wrappers and compares it to more traditional methods of analysis. The second manuscript tests how accurate a specific NLP technique is in finding these strategies and goals. The third manuscript looks at how different students use different strategies and goals depending on how well they did on the exams using the NLP technique in the second manuscript. I found that NLP and GPT-3.5 can aid in analyzing exam wrappers faster and provide nuanced insights when compared with manual approaches. The dissertation also shows what learning strategies and goals are most discussed for engineering students as they prepare for exams. The dissertation gives some suggestions, challenges, and ideas for future research on AI and learning from exams.

Thesis advisor: Rebecca Schendel / Higher education institutes have seen massive growth over the decades in terms of student enrollment. To maintain the quality of education provided to students, scholars, and practitioners have advocated for students to submit an entrance exam as a part of the admission application. This study analyzed trends in all national and standardized entrance exams (N= 119) to see which areas of knowledge and methods of assessments are most commonly utilized in entrance exams around the world. The areas of knowledge that were looked at in this study: English, Language Arts, Math, History, Government Studies, Religion, and Hard Science. The methods of assessment that were looked at in this study: Supplied response, Selected response, Practical, and Oral. Data was collected through publicly available documents and entered into a database to then analyze and recognize trends. The findings of this study show that the most common area of knowledge that is tested is Language Arts (N = 107) and the most common method of assessment is supplied response (N = 81). / Thesis (MA) — Boston College, 2023. / Submitted to: Boston College. Lynch School of Education. / Discipline: Educational Leadership and Higher Education.

Context: Collegiate male athletes have a higher risk of testicular cancer due to their age group, an increased risk of testicular contusions, and a lack of secondary prevention education. As the athletic training profession increases emphasis on evidence-based practice, it is important for athletic trainers to understand testicular cancer and testicular-self examination as it is outlined within their scope of practice. A general understanding of testicular cancer and the prevention techniques will be important for athletic trainers to promote awareness and health behavior practices. Objective: To examine the athletic trainers' actual knowledge, concern, perceived responsibility, training, feeling of embarrassment, and professional/personal practices. Design: Cross sectional survey. Participants: 249 randomly selected athletic trainers employed in collegiate settings. 65.6% of the respondents reported being between the ages of 21 and 35 years old. Intervention: Actual knowledge, concerned, perceived responsibility, trained, embarrassed, and personal and professional practice behavior scores served as dependent variables. Main Outcome Measures: A Pearson correlation coefficient was calculated between participants' actual knowledge, perceived responsibility, and concerned scores. Two one-way MANOVAs were conducted to determine if there was a difference in actual knowledge, perceived responsibility, and concerned scores that was dependent upon participants' age and gender. Results: Athletic trainers in collegiate settings had a fairly high actual knowledge of testicular cancer (X=7.62[plus or minus]1.42 out of 10). Athletic trainers reported that they should be concerned about testicular cancer in male athletes (X=7.26[plus or minus].167 out of 10). Athletic trainers had a low feeling of responsibility suggested by their reported score (X=3.93[plus or minus]0.18 out of 10). A weak correlation (r(169)=.199, P[less than].009) was found between the actual knowledge and perceived responsibility scores, and between the actual knowledge and concerned scores (r(169)=.285, P[less than]<.001). A medium to strong correlation (r(169)=.486, P[less than].001) was found between the concerned and perceived responsibility scores. Athletic trainers reported a decreased feeling of training about testicular cancer and testicular selfexamination (X=2.28[plus or minus]2.10 out of 10). Also, athletic trainers reported (X=2.71[plus or minus]2.42 out of 10) that they were not embarrassed to discuss testicular cancer. Athletic trainers reported performing either a testicular self-exam or breast-self examination on themselves (X=76%). Conclusions: College athletic trainers have a low feeling of embarrassment, adequate knowledge, and a high feeling of concern regarding testicular cancer, but report a low feeling of perceived responsibility and training.

No description available.

No description available.

Depuis 2000, l’examen professionnel de l’Ordre des infirmières et infirmiers du Québec (OIIQ) est constitué d’une partie écrite et d’une partie pratique sous la forme d’un examen clinique objectif structuré (ECOS). Des programmes de préparation à l’examen professionnel ont été mis en place par différents milieux afin d’aider les candidates à l’exercice de la profession infirmière (CEPI) à se préparer pour cet examen, toutefois aucune évaluation de ces programmes n’a encore été effectuée. Cette étude avait pour but d’évaluer qualitativement un programme de préparation à l’examen professionnel. Des groupes de discussion ont été organisés avec des CEPI ayant participé au programme de préparation afin d’obtenir leur perception quant à l’impact de ce programme sur leur expérience de l’examen professionnel. La méthode d’analyse par questionnement analytique de Paillé et Mucchielli (2008) a été utilisée pour réaliser l’analyse des données. L’approche par compétences de deuxième génération développée par Goudreau, Pepin, Dubois, Boyer, Larue et Legault (2009) fut utilisée comme cadre de référence pour cette étude. L’analyse des données a permis d’identifier les attentes des CEPI à l’égard du programme de préparation et les impacts de celui-ci sur leur expérience à l’examen professionnel. Ainsi, le programme permettrait aux CEPI d’évaluer leurs forces et leurs faiblesses afin d’orienter l’étude préparatoire, tout en diminuant le stress relié à l’inconnu et en augmentant leur motivation et leur confiance en elles. Le parcours ECOS et la rétroaction sont les activités du programme jugées les plus pertinentes dans une perspective d’approche par compétences. L’examen écrit serait toutefois moins représentatif de l’examen professionnel et moins pertinent pour le développement des compétences. Des suggestions d’amélioration pour le programme de préparation découlent des entrevues avec les participantes et des recommandations pour la gestion, la pratique, la formation et la recherche en sciences infirmières ont également été émises à la suite de cette étude. / Since 2000, the professional licensure exam of the Ordre des infirmières et infirmiers du Québec (OIIQ) is comprised of two parts; a written exam and a practical exam in the form of an objectively structured clinical examination (OSCE). Various establishments have created preparation programs in order to help candidates study for the exam; however, these programs have never been evaluated. The goal of this study was to qualitatively assess a preparation program for the professional licensure exam. Focus groups were held with candidates who had participated in a preparation program. The groups explored the program’s impact on the candidates’ experience during the professional licensure exam, as perceived by the candidates themselves. The data analysis was performed based on the method of analysis by analytical questioning of Paillé and Mucchielli (2008). The second-generation competency-based approach developed by Goudreau, Pepin, Dubois, Boyer, Larue and Legault (2009) was used as a framework for this study. Analysis of the interviews identified candidates’ expectations with regards to the preparation program and its impact on their experience during the professional licensure exam. The program allowed candidates to assess their strengths and weaknesses in order to guide their study sessions, while reducing the stress linked to the fear of the unknown and increasing their motivation and confidence. The OSCE and the feedback provided after OSCE are the activities perceived as being the most relevant, both by the candidates and from a competency approach perspective. On the other hand, the written exam was perceived as being less relevant for the exam preparation and less relevant to competency development. Suggestions to improve the preparation program were gathered from the interviews with the participants and recommendations for management, practice, education and research in nursing science were made as a result of this study.

Depuis 2000, l’examen professionnel de l’Ordre des infirmières et infirmiers du Québec (OIIQ) est constitué d’une partie écrite et d’une partie pratique sous la forme d’un examen clinique objectif structuré (ECOS). Des programmes de préparation à l’examen professionnel ont été mis en place par différents milieux afin d’aider les candidates à l’exercice de la profession infirmière (CEPI) à se préparer pour cet examen, toutefois aucune évaluation de ces programmes n’a encore été effectuée. Cette étude avait pour but d’évaluer qualitativement un programme de préparation à l’examen professionnel. Des groupes de discussion ont été organisés avec des CEPI ayant participé au programme de préparation afin d’obtenir leur perception quant à l’impact de ce programme sur leur expérience de l’examen professionnel. La méthode d’analyse par questionnement analytique de Paillé et Mucchielli (2008) a été utilisée pour réaliser l’analyse des données. L’approche par compétences de deuxième génération développée par Goudreau, Pepin, Dubois, Boyer, Larue et Legault (2009) fut utilisée comme cadre de référence pour cette étude. L’analyse des données a permis d’identifier les attentes des CEPI à l’égard du programme de préparation et les impacts de celui-ci sur leur expérience à l’examen professionnel. Ainsi, le programme permettrait aux CEPI d’évaluer leurs forces et leurs faiblesses afin d’orienter l’étude préparatoire, tout en diminuant le stress relié à l’inconnu et en augmentant leur motivation et leur confiance en elles. Le parcours ECOS et la rétroaction sont les activités du programme jugées les plus pertinentes dans une perspective d’approche par compétences. L’examen écrit serait toutefois moins représentatif de l’examen professionnel et moins pertinent pour le développement des compétences. Des suggestions d’amélioration pour le programme de préparation découlent des entrevues avec les participantes et des recommandations pour la gestion, la pratique, la formation et la recherche en sciences infirmières ont également été émises à la suite de cette étude. / Since 2000, the professional licensure exam of the Ordre des infirmières et infirmiers du Québec (OIIQ) is comprised of two parts; a written exam and a practical exam in the form of an objectively structured clinical examination (OSCE). Various establishments have created preparation programs in order to help candidates study for the exam; however, these programs have never been evaluated. The goal of this study was to qualitatively assess a preparation program for the professional licensure exam. Focus groups were held with candidates who had participated in a preparation program. The groups explored the program’s impact on the candidates’ experience during the professional licensure exam, as perceived by the candidates themselves. The data analysis was performed based on the method of analysis by analytical questioning of Paillé and Mucchielli (2008). The second-generation competency-based approach developed by Goudreau, Pepin, Dubois, Boyer, Larue and Legault (2009) was used as a framework for this study. Analysis of the interviews identified candidates’ expectations with regards to the preparation program and its impact on their experience during the professional licensure exam. The program allowed candidates to assess their strengths and weaknesses in order to guide their study sessions, while reducing the stress linked to the fear of the unknown and increasing their motivation and confidence. The OSCE and the feedback provided after OSCE are the activities perceived as being the most relevant, both by the candidates and from a competency approach perspective. On the other hand, the written exam was perceived as being less relevant for the exam preparation and less relevant to competency development. Suggestions to improve the preparation program were gathered from the interviews with the participants and recommendations for management, practice, education and research in nursing science were made as a result of this study.

The topic of this thesis are problematic types of test items in the listening sub test of the didactic test of common part of Maturita school - leaving exam in French language. Introductory chapter of the theoretical part of the thesis focuses on the description of Maturita school-leaving exam, its origin and contemporary model. The following chapters deal with characteristics of the didactic test, its parts and individual test items. These chapters also provide a description of the examined sample, concise characteristics of grammar schools (Gymnasiums) where the research was conducted, and introduces the analytical tools which were used when examining the listening subtest in French language. On the basis of students' answers the most difficult test items of the sub test were pinpointed. KEYWORDS: didactic test, listening sub test, French language, Maturita school-leaving exam, state high school leaving exam, test item

This diploma thesis deals with speaking skills and their development for the purposes of the Maturita exam. It is divided into 6 chapters. The theoretical part deals with speaking as a skill, basic definitions of the Maturita exam and, more specifically, it gives a detailed overview and comparison of the Maturita oral performance and interactive skills at both levels. The practical part covers the analysis of three sets of books that are used during the English lessons in secondary education. Key words: Maturita exam, basic level, higher level, evaluation, course books

En vue de l’importance de l’évaluation dans l’enseignement et l’apprentissage des langues étrangères, notre projet vise à comparer et analyser les pratiques d’évaluation utilisées dans les cours de langue étrangère au sein de trois pays : l’Iran, la France et les États-Unis. L’objectif de cette recherche n’est pas de mettre en valeur un système éducatif à proprement parlé, mais de comprendre si les pratiques évaluatives de ces contextes sont universelles ou culturellement spécifiques. In fine, il convient de comprendre quelle est la dynamique qui met en jeu leurs spécificités, c'est-à-dire les réalités sociales, politiques ou économiques qui les forment. Pour la réalisation de cette étude, nous avons mené des entretiens compréhensifs semi-directifs auprès de 48 enseignants de langues (français, anglais et persan) dans les trois pays. À travers ces entretiens nous avons extrait les représentations de ces enseignants autour de quatre concepts clés de pratiques d’évaluation, c'est-à-dire les objectifs d’apprentissage des langues, les pratiques et les instruments évaluatifs, l’examen, et la perception des enseignants des langues (étrangères). L’analyse qualitative de ces entretiens nous a montré que les pratiques d’évaluation utilisées dans les différents contextes académiques et universitaires dans ces trois pays, sont plutôt universelles ; fait principalement dû aux règles et exigences particulières du contexte institutionnel d’évaluation que les enseignants doivent respecter et satisfaire, ainsi qu’au niveau élémentaire des étudiants, qui nécessite l’intégration des activités évaluatives des niveaux inférieurs de la Taxonomie de Bloom. En même temps, nous avons également constaté des différences significatives qui relèvent des particularités culturelles telles que les traditions d’enseignement et d’évaluation, les rôles de l’enseignant, de l’apprenant et de l’institution, ainsi que des contextes d’emploi de la langue, situations qui dépendent, à leur tour, des contextes et des enjeux politiques, économiques, sociaux et géographiques de chacun des trois pays étudiés. / Given the importance of assessment in the teaching and the learning of foreign languages, this dissertation aims at comparing and analyzing the assessment practices used in university-level foreign language classes in three countries: Iran, France and the United States. The goal of this research is not to valorize one of these countries’ educational systems, but to understand if the assessment practices used in these countries are universal or culturally specific and if specific, what social, political or economic realities are involved. For the completion of this study, a set of semi-directive comprehensive interviews were conducted with 48 language instructors (French, English and Persian languages) in these three countries. These interviews provided us with the representations of the participants of their assessment practices, which enabled us to establish four images that reflect the key concepts relating to assessment: these images were course objectives, assessment tools and practices, exam and instructors’ perception of (foreign) languages. Through a qualitative analysis of these interviews revealed that assessment practices in academic (university) settings were in part universal, due to their institutional contexts, which require the implementation of certain rules and regulations, as well as the elementary level of students involved in these assessment practices, which favored the use of the activities that lie in the inferior levels of the Bloom’s Taxonomy. At the same time, we also able to observe significant differences among the assessment practices used in these three countries, which raise from the specificities of these three cultures and education/teaching traditions, as well as the uses of the language in question, which depend on the political, economic, social and geographic situations of each of these three countries.