This dissertation explores the use of natural language processing (NLP) and large language models (LLMs) to analyze student self-regulated learning (SRL) strategies in response to exam wrappers. Exam wrappers are structured reflection activities that prompt students to practice SRL after they get their graded exams back. The dissertation consists of three manuscripts that compare traditional qualitative analysis with NLP-assisted approaches using transformer-based models including GPT-3.5, a state-of-the-art LLM. The data set comprises 3,800 student responses from an engineering physics course. The first manuscript develops two NLP-assisted codebooks for identifying learning strategies related to SRL in exam wrapper responses and evaluates the agreement between them and traditional qualitative analysis. The second manuscript applies a novel NLP technique called zero-shot learning (ZSL) to classify student responses into the codes developed in the first manuscript and assesses the accuracy of this method by evaluating a subset of the full dataset. The third manuscript identifies the distribution and differences of learning strategies and SRL constructs among students of different exam performance profiles using the results from the second manuscript. The dissertation demonstrates the potential of NLP and LLMs to enhance qualitative research by providing scalable, robust, and efficient methods for analyzing large corpora of textual data. The dissertation also contributes to the understanding of SRL in engineering education by revealing the common learning strategies, impediments, and SRL constructs that students report they use while preparing for exams in a first-year engineering physics course. The dissertation suggests implications, limitations, and directions for future research on NLP, LLMs, and SRL. / Doctor of Philosophy / This dissertation is about using artificial intelligence (AI) to help researchers and teachers understand how students learn from their exams. Exams are not only a way to measure what students know, but also a chance for students to reflect on how they studied and what they can do better next time. One way that students can reflect is by using exam wrappers, which are short questions that students answer after they get their graded exams back. A type of AI called natural language processing (NLP) is used in this dissertation, which can analyze text and find patterns and meanings in it. This study also uses a powerful AI tool called GPT-3.5, which can generate text and answer questions. The dissertation has three manuscripts that compare the traditional way of analyzing exam wrappers, which is done by hand, with the new way of using NLP and GPT-3.5, evaluate a specific promising NLP method, and use this method to try and gain a deeper understanding in students self-regulated learning (SRL) while preparing for exams. The data comes from 3,800 exam wrappers from a physics course for engineering students. The first manuscript develops a way of using NLP and GPT-3.5 to find out what learning strategies and goals students talk about in their exam wrappers and compares it to more traditional methods of analysis. The second manuscript tests how accurate a specific NLP technique is in finding these strategies and goals. The third manuscript looks at how different students use different strategies and goals depending on how well they did on the exams using the NLP technique in the second manuscript. I found that NLP and GPT-3.5 can aid in analyzing exam wrappers faster and provide nuanced insights when compared with manual approaches. The dissertation also shows what learning strategies and goals are most discussed for engineering students as they prepare for exams. The dissertation gives some suggestions, challenges, and ideas for future research on AI and learning from exams.

Of the emerging photovoltaic technologies, perovskite solar cells (PSCs) are arguably among the most promising candidates for commercialization. Worldwide interest has prompted researchers to produce tens of thousands of studies on the topic, making PSCs one of the most active research topics of the past decade. Unfortunately, the rapid output of a substantial number of publications has made the traditional literature review process and research plans cumbersome tasks for both the novice and expert. In this dissertation, a data-driven analysis utilizing a novel text mining and natural language processing pipeline is applied on the perovskite literature to help decipher the field, uncover emerging research trends, and delineate an experimental research plan of action for this dissertation. The analysis led to the selection and exploration of two experimental projects on single-crystal PSCs, which are devices based on micrometers-thick grain-boundary-free monocrystalline films with long charge carrier diffusion lengths and enhanced light absorption (relative to polycrystalline films). First, a low-temperature crystallization approach is devised to improve the quality of methylammonium lead iodide (MAPbI3) single-crystal films, leading to devices with markedly enhanced open-circuit voltages (1.15 V vs 1.08 V for controls) and power conversion efficiencies (PCEs) of up to 21.9%, among the highest reported for MAPbI3-based devices. Second, mixed-cation formamidinium (FA)-based single-crystal PSCs are successfully fabricated with PCEs of up to 22.8% and short-circuit current values exceeding 26 mA cm-2, achieved by a significant expansion of the external quantum efficiency band edge, which extends past that of the state-of-the-art polycrystalline FAPbI3-based solar cells by about 50 meV — only 60 meV larger than that of the top-performing photovoltaic material, GaAs. These figures of merit not only set new record values for SC-PSCs, but also showcase the potential of adopting data-driven techniques to guide the research process of a data-rich field.

This paper investigates the conversational capabilities of voice-controlled virtual assistants with respect to biased questions and answers. Three commercial virtual assistants (Google Assistant, Alexa and Siri) are tested for the presence of three cognitive biases (wording, framing and confirmation) in the answers given. The results show that all assistants are susceptible to wording and framing biases to varying degrees, and have limited ability to recognise questions designed to induce cognitive biases. The paper describes the different response strategies available to voice user interfaces, the differences between them, and discusses the role of strategy in relation to biased content.

Committee meetings are a fundamental part of the legislative process in whichconstituents, lobbyists, and legislators alike can speak on proposed bills at thelocal and state level. Oftentimes, unspoken “rules” or standards are at play inpolitical processes that can influence the trajectory of a bill, leaving constituentswithout a political background at an inherent disadvantage when engaging withthe legislative process. The work done in this thesis aims to explore the extent towhich the language and phraseology of a general public testimony can influence avote, and examine how this information can be used to promote civic engagement. The Digital Democracy database contains digital records for over 40,000 realtestimonies by non-legislator public persons presented at California Legislaturecommittee meetings 2015-2018, along with the speakers’ desired vote outcomeand individual legislator votes in that discussion. With this data, we conduct alinguistic analysis that is then leveraged by the Constituent phraseology AnalysisTool (CPAT) to generate a user-based intelligent statistical comparison betweena proposed testimony and language patterns that have previously been successful. The following questions are at the core of this research: Which (if any) lan-guage features are correlated with persuasive success in a legislative context?Does the committee’s topic of discussion impact the language features that canlend to a testimony’s success? Can mirroring a legislator’s speech patterns changethe probability of the vote going your way? How can this information be used tolevel the playing field for constituents who want their voices heard? Given the 33 linguistic features developed in this research, supervised classifi-cation models were able to predict testimonial success with up to 85.1% accuracy,indicating that the new features had a significant impact on the prediction ofsuccess. Adding these features to the 16 baseline linguistic features developedin Gundala’s [18] research improved the prediction accuracy by up to 2.6%. Wealso found that balancing the dataset of testimonies drastically impacted theprediction performance metrics, with 93% accuracy achieved for the imbalanceddataset and 60% accuracy after balancing. The Constituent Phraseology AnalysisTool showed promise in the generation of linguistic analysis based on previouslysuccessful language patterns, but requires further development before achievingtrue usability. Additionally, predicting success based on linguistic similarity to alegislator on the committee produced contradictory results. Experiments yieldeda 4% increase in predictive accuracy when adding comparative language featuresto the feature set, but further experimentation with weight distributions revealedonly marginal impacts from comparative features.

This thesis explores potential strategies to enhance user control and satisfaction within the movie selection process, with a particular focus on the utilization of conversational generative artificial intelligence, such as ChatGPT, for personalized movie recommendations. The study adopts a qualitative user-centered design thinking approach, aiming to compre-hensively understand user needs, goals, and behavior. In-depth interviews were conducted, utilizing the "Thinking aloud"method and trigger materials to elicit rich user feedback. Participants interacted with ChatGPT and various prototypes, providing valuable insights into their experiences. The study found that participants felt more in control when given the opportunity to specify wishes. In addition, the users found that the experience of receiving recommendations through ChatGPT was more satisfying than their usual way of receiving recommendations for movies. Furthermore, participants expressed a desire for additional information about recommended movies and more novel suggestions. The prototypes, designed as triggers for user feedback, were generally well-received, providing an engaging and fun user experience. Despite some participants expressing challenges in specifying movie choices based on an emotion, this new approach to movie selection was viewed positively. Despite limitations concerning the study’s validity, reliability, and testing situation, the findings suggest the potential of generative artificial intelligence conversational agents in enhancing the movie selection process. It is concluded that iterative design improvements and further research is necessary to fully leverage the potential of natural language processing technologies in recommendation systems. The study serves as a preliminary investigation into improving movie recommendations using generative artificial intelligence and offers valuable insights for future developments.

Process systems engineering (PSE) involves a systems-level approach to solving problems in chemical engineering related to process modeling, design, control, and optimization and involves modeling interactions between various systems (and subsystems) governing the process. This requires using a combination of mathematical methods, physical intuition, and recently machine learning techniques. Recently, language models have seen tremendous advances due to new and more efficient model architectures (such as transformers), computing power, and large volumes of training data. Many of these language models could be appropriately adapted to solve several PSE-related problems. However, language models are inherently complex and are often characterized by several million parameters, which could only be trained efficiently in data-rich areas, unlike PSE. Moreover, PSE is characterized by decades of rich process knowledge that must be utilized during model training to avoid mismatch between process knowledge and data-driven language models. This thesis presents a framework for building domain-informed language models for several central problems in PSE spanning multiple scales. Specifically, the frameworks presented include molecular property prediction, forward and retrosynthesis reaction outcome prediction, chemical flowsheet representation and generation, pharmaceutical information extraction, and reaction classification. Domain knowledge is integrated with language models using custom model architectures, standard and custom-built ontologies, linguistics-inspired chemistry and process flowsheet grammar, adapted problem formulations, graph theory techniques, and so on. This thesis is intended to provide a path for future developments of domain-informed language models in process systems engineering that respect domain knowledge, but leverage their computational advantages.

The available body of biomedical literature is increasing at a high pace, exceeding the ability of researchers to promptly leverage this knowledge-rich amount of information. Although the outstanding progress in natural language processing (NLP) we observed in the past few years, current technological advances in the field mainly concern newswire and web texts, and do not directly translate in good performance on highly specialized domains such as biomedicine due to linguistic variations along surface, syntax and semantic levels. Given the advances in NLP and the challenges the biomedical domain exhibits, and the explosive growth of biomedical knowledge being currently published, in this thesis we contribute to the biomedical NLP field by providing efficient means for extracting semantic relational information from biomedical literature texts. To this end, we made the following contributions towards the real-world adoption of knowledge extraction methods to support biomedicine: (i) we propose a symbolic high-precision biomedical relation extraction approach to reduce the time-consuming manual curation efforts of extracted relational evidence (Chapter 3), (ii) we conduct a thorough cross-domain study to quantify the drop in performance of deep learning methods for biomedical edge detection shedding lights on the importance of linguistic varieties in biomedicine (Chapter 4), and (iii) we propose a fast and accurate end-to-end solution for biomedical event extraction, leveraging sequential transfer learning and multi-task learning, making it a viable approach for real-world large-scale scenarios (Chapter 5). We then outline the conclusions by highlighting challenges and providing future research directions in the field.

Automatic Post-Editing (APE) aims to correct systematic errors in a machine translated text. This is primarily useful when the machine translation (MT) system is not accessible for improvement, leaving APE as a viable option to improve translation quality as a downstream task - which is the focus of this thesis. This field has received less attention compared to MT due to several reasons, which include: the limited availability of data to perform a sound research, contrasting views reported by different researchers about the effectiveness of APE, and limited attention from the industry to use APE in current production pipelines. In this thesis, we perform a thorough investigation of APE as a down- stream task in order to: i) understand its potential to improve translation quality; ii) advance the core technology - starting from classical methods to recent deep-learning based solutions; iii) cope with limited and sparse data; iv) better leverage multiple input sources; v) mitigate the task-specific problem of over-correction; vi) enhance neural decoding to leverage external knowledge; and vii) establish an online learning framework to handle data diversity in real-time. All the above contributions are discussed across several chapters, and most of them are evaluated in the APE shared task organized each year at the Conference on Machine Translation. Our efforts in improving the technology resulted in the best system at the 2017 APE shared task, and our work on online learning received a distinguished paper award at the Italian Conference on Computational Linguistics. Overall, outcomes and findings of our work have boost interest among researchers and attracted industries to examine this technology to solve real-word problems.

Uppgifter för behandling av naturliga språk (NLP) har under de senaste åren visat sig vara särskilt effektiva när man använder förtränade språkmodeller som BERT. Det enorma kravet på datorresurser som krävs för att träna sådana modeller gör det dock svårt att använda dem i verkligheten. För att lösa detta problem har komprimeringsmetoder utvecklats. I det här projektet studeras, genomförs och testas några av dessa metoder för komprimering av neurala nätverk för textbearbetning. I vårt fall var den mest effektiva metoden Knowledge Distillation, som består i att överföra kunskap från ett stort neuralt nätverk, som kallas läraren, till ett litet neuralt nätverk, som kallas eleven. Det finns flera varianter av detta tillvägagångssätt, som skiljer sig åt i komplexitet. Vi kommer att titta på två av dem i det här projektet. Den första gör det möjligt att överföra kunskap mellan ett neuralt nätverk och en mindre dubbelriktad LSTM, genom att endast använda resultatet från den större modellen. Och en andra, mer komplex metod som uppmuntrar elevmodellen att också lära sig av lärarmodellens mellanliggande lager för att utvinna kunskap. Det slutliga målet med detta projekt är att ge företagets datavetare färdiga komprimeringsmetoder för framtida projekt som kräver användning av djupa neurala nätverk för NLP. / Natural language processing (NLP) tasks have proven to be particularly effective when using pre-trained language models such as BERT. However, the enormous demand on computational resources required to train such models makes their use in the real world difficult. To overcome this problem, compression methods have emerged in recent years. In this project, some of these neural network compression approaches for text processing are studied, implemented and tested. In our case, the most efficient method was Knowledge Distillation, which consists in transmitting knowledge from a large neural network, called teacher, to a small neural network, called student. There are several variants of this approach, which differ in their complexity. We will see two of them in this project, the first one which allows a knowledge transfer between any neural network and another smaller bidirectional LSTM, using only the output of the larger model. And a second, more complex approach that encourages the student model to also learn from the intermediate layers of the teacher model for incremental knowledge extraction. The ultimate goal of this project is to provide the company’s data scientists with ready-to-use compression methods for their future projects requiring the use of deep neural networks for NLP.

The science of making a computer understand text and process it, natural language processing, is a topic of great interest among researchers. This study aims to further that research by comparing the BERT algorithm and classic logistic regression when identifying names of public organizations. The results show that BERT outperforms its competitor in the task from the data which consisted of public state inquiries and reports. Furthermore a literature study was conducted as a way of exploring how a system for NER can be implemented into the management of an organization. The study found that there are many ways of doing such an implementation but mainly suggested three main areas that should be focused to ensure success - recognising the right entities, trusting the system and presentation of data. / Vetenskapen kring hur datorer ska förstå och arbeta med fria texter, språkteknologi, är ett område som blivit populärt bland forskare. Den här uppsatsen vill utvidga det området genom att jämföra BERT med logistisk regression för att undersöka nämnandet av svenska myndigheter genom NER. BERT visar bättre resultat i att identifiera namnen på myndigheter från texter i statliga utredningar och rapporter än modellen med logistisk regression. Det genomfördes även en litteraturstudie för att undersöka hur ett system för NER kan implementeras i en organisation. Studien visade att det finns flera sätt att genomföra detta men föreslår framförallt tre områden som bör fokuseras på för en lyckad implementation - användande av rätt entiteter, trovärdighet i system och presentation av data.