This study investigates the utilization of Large Language Models for Test Case Generation. The study uses the Large Language model and Embedding model provided by Llama, specifically Llama2 of size 7B, to generate test cases given a defined input. The study involves an implementation that uses customization techniques called Retrieval Augmented Generation (RAG) and Prompt Engineering. RAG is a method that in this study, stores organisation information locally, which is used to create test cases. This stored data is used as complementary data apart from the pre-trained data that the large language model has already trained on. By using this method, the implementation can gather specific organisation data and therefore have a greater understanding of the required domains. The objective of the study is to investigate how AI-driven test case generation impacts the overall software quality and development efficiency. This is evaluated by comparing the output of the AI-based system, to manually created test cases, as this is the company standard at the time of the study. The AI-driven test cases are analyzed mainly in the form of coverage and time, meaning that we compare to which degree the AI system can generate test cases compared to the manually created test case. Likewise, time is taken into consideration to understand how the development efficiency is affected. The results reveal that by using Retrieval Augmented Generationin combination with Prompt Engineering, the system is able to identify test cases to a certain degree. The results show that 66.67% of a specific project was identified using the AI, however, minor noise could appear and results might differ depending on the project’s complexity. Overall the results revealed how the system can positively impact the development efficiency and could also be argued to have a positive effect on the software quality. However, it is important to understand that the implementation as its current stage, is not sufficient enough to be used independently, but should rather be used as a tool to more efficiently create test cases.

The development of artificial intelligence (AI) has revolutionised the way we interact with technology and each other, both in society and in professional careers. Although they come with great potential for productivity and automation, AI systems have been found to exhibit biases that reflect and perpetuate existing societal inequalities. With the recent rise of artificial intelligence tools exploiting the large language model (LLM) technology, such as ChatGPT, Bing Chat and Bard AI, this research project aims to investigate the extent of AI bias in said tools and explore its ethical implications. By reviewing and analysing responses to carefully crafted prompts generated by three different AI chatbot tools, the author will intend to determine whether the content generated by these tools indeed exhibits patterns of bias related to various social identities, as well as compare the extent to which such bias is present across all three tools. This study will contribute to the growing body of literature on AI ethics and inform efforts to develop more equitable and inclusive AI systems. By exploring the ethical dimensions of AI bias in selected LLMs, this research will shed light on the broader societal implications of AI and the role of technology in shaping our future.

Automatic Speech Recognition (ASR) enables a wide variety of practical applications. However, many applications have their own domain-specific words, creating a gap between training and test data when used in practice. Domain adaptation can be achieved through model fine-tuning, but it requires domain-specific speech data paired with transcripts, which is labor intensive to produce. Fortunately, the dependence on audio data can be mitigated to a certain extent by incorporating text-based language models during decoding. This thesis explores approaches for creating domain-specific 4-gram models for a Swedish open-source wav2vec 2.0 model. The three main approaches extend a social media corpus with domain-specific data to estimate the models. The first approach utilizes a relatively small set of in-domain text data, and the second approach utilizes machine transcripts from another ASR system. Finally, the third approach utilizes Named Entity Recognition (NER) to find words of the same entity type in a corpus to replace with in-domain words. The 4-gram models are evaluated by the error rate (ERR) of recognizing in-domain words in a custom dataset. Additionally, the models are evaluated by the Word Error Rate (WER) on the Common Voice test set to ensure good overall performance. Compared to not having a language model, the base model improves the WER on Common Voice by 2.55 percentage points and the in-domain ERR by 6.11 percentage points. Next, adding in-domain text to the base model results in a 2.61 WER improvement and a 10.38 ERR improvement over not having a language model. Finally, adding in-domain machine transcripts and using the NER approach results in the same 10.38 ERR improvement as adding in-domain text but slightly less significant WER improvements of 2.56 and 2.47, respectively. These results contribute to the exploration of state-of-the-art Swedish ASR and have the potential to enable the adoption of open-source ASR models for more use cases. / Automatisk taligenkänning (ASR) möjliggör en mängd olika praktiska tillämpningar. Men många tillämpningsområden har sin egen uppsättning domänspecifika ord vilket kan skapa problem när en taligenkänningsmodell används på data som skiljer sig från träningsdatan. Taligenkänningsmodeller kan anpassas till nya domäner genom fortsatt träning med taldata, men det kräver tillgång till domänspecifik taldata med tillhörande transkript, vilket är arbetskrävande att producera. Lyckligtvis kan beroendet av ljuddata mildras till viss del genom användande av textbaserade språkmodeller tillsammans med taligenkänningsmodellerna. Detta examensarbete utforskar tillvägagångssätt för att skapa domänspecifika 4-gram-språkmodeller för en svensk wav2vec 2.0-modell som tränats av Kungliga Biblioteket. Utöver en basmodell så används tre huvudsakliga tillvägagångssätt för att utöka en korpus med domänspecifik data att träna modellerna från. Det första tillvägagångssättet använder en relativt liten mängd domänspecifik textdata, och det andra tillvägagångssättet använder transkript från ett annat ASR-system (maskintranskript). Slutligen använder det tredje tillvägagångssättet Named Entity Recognition (NER) för att hitta ord av samma entitetstyp i en korpus som sedan ersätts med domänspecifika ord. Språkmodellerna utvärderas med ett nytt domänspecifikt evalueringsdataset samt på testdelen av Common Voice datasetet. Jämfört med att inte ha en språkmodell förbättrar basmodellen Word Error Rate (WER) på Common Voice med 2,55 procentenheter och Error Rate (ERR) inom domänen med 6,11 procentenheter. Att lägga till domänspecifik text till basmodellens korpus resulterar i en 2,61 WER-förbättringochen10,38 ERR-förbättring jämfört med att inte ha en språkmodell. Slutligen, att lägga till domänspecifika maskintranskript och att använda NER-metoden resulterar i samma 10.38 ERR-förbättringar som att lägga till domänspecifik text men något mindre WER-förbättringar på 2.56 respektive 2.47 procentenheter. Den här studien bidrar till svensk ASR och kan möjliggöra användandet av öppna taligenkänningsmodeller för fler användningsområden.

Much research work has been done to extract data from scientific papers, journals, and articles. However, Electronic Theses and Dissertations (ETDs) remain an unexplored genre of data in the research fields of natural language processing and machine learning. Moreover, much of the related research involved data that is in the English language. Arabic data such as news and tweets have begun to receive some attention in the past decade. However, Arabic ETDs remain an untapped source of data despite the vast number of benefits to students and future generations of scholars. Some ways of improving the browsability and accessibility of data include data annotation, indexing, parsing, translation, and classification. Classification is essential for the searchability and management of data, which can be manual or automated. The latter is beneficial when handling growing volumes of data. There are two main roadblocks to performing automatic subject classification on Arabic ETDs. The first is the unavailability of a public corpus of Arabic ETDs. The second is the Arabic language’s linguistic complexity, especially in academic documents. This research presents the Otrouha project, which aims at building a corpus of key metadata of Arabic ETDs as well as providing a methodology for their automatic subject classification. The first goal is aided by collecting data from the AskZad Digital Library. The second goal is achieved by exploring different machine learning and deep learning techniques. The experiments’ results show that deep learning using pretrained language models gave the highest classification performance, indicating that language models significantly contribute to natural language understanding. / M.S. / An Electronic Thesis or Dissertation (ETD) is an openly-accessible electronic version of a graduate student’s research thesis or dissertation. It documents their main research effort that has taken place and becomes available in the University Library instead of a paper copy. Over time, collections of ETDs have been gathered and made available online through different digital libraries. ETDs are a valuable source of information for scholars and researchers, as well as librarians. With the digitalization move in most Middle Eastern Universities, the need to make Arabic ETDs more accessible significantly increases as their numbers increase. One of the ways to improve their accessibility and searchability is through providing automatic classification instead of manual classification. This thesis project focuses on building a corpus of metadata of Arabic ETDs and building a framework for their automatic subject classification. This is expected to pave the way for more exploratory research on this valuable genre of data.

The loss of senior expertise in power line design poses a critical challenge to the sustainable energy transition. Current methods of knowledge transfer fail to prevent the loss of invaluable knowledge necessary for future junior power line designers. Additionally, the rise of informal deployment of generative language models may also threaten to bury hand-written knowledge documents before this knowledge can be extracted, structured, and preserved for future guidance. This thesis proposes a framework where large language models are integrated into knowledge transfer and decision-making guidance for an engineering enterprise. Using this framework, this thesis further explores how data-driven knowledge tools can assist junior design engineers by supporting information retrieval and directing to knowledge sources. The ability of a large language model to retrieve relevant knowledge from an engineering design document was validated by comparing the process of human designers manually completing a similar task. In this evaluation involving six participants and the large language model, responses to questions on mechanical dimensioning of stays for utility poles were ranked by experts. The results showed that the large language model responses were ranked similarly to the junior designers on average. Additionally, a small-scale demonstrative knowledge tool, insights from interviews, literature studies as well as the results from the validation study lead to the conclusion that large language models can assist power line designers via a knowledge tool. Beyond power line design, this thesis contributes to the understanding of how data-driven language models can assist knowledge retrieval and decision-making across other engineering design domains. This work utilizes a professional education document on the mechanical dimensioning of wooden power line poles including an analysis on the wind and weight span’s affect on the dimension of the pole, developed parallel to this work. The original design data from the document supported the tests conducted in this thesis. The professional education document on the mechanical dimensioning of wooden power line poles was developed in parallel to this thesis as a case study supporting the tests with original design data on power line design knowledge. The work also discusses risks and ethical aspects when implementing such a knowledge tool. Risks such as leakage of classified information are emphasized and need comprehensive systems and methods to be avoided. It is therefore highlighted how important it is to carry out the project with care and expertise to avoid damage to companies and society. Local language models or highly trusted AI system providers are recommended to ensure that no sensitive information is leaked to an unwanted third-party. With a high degree of caution and consideration of risks, an effective knowledge tool can contribute to increased efficiency, faster and more sustainable development of power line infrastructure, and thus an faster energy transition. / Förlusten av senior expertis inom kraftledningskonstruktion utgör en kritisk utmaning för den hållbara energiomställningen. Nuvarande metoder för kunskapsöverföring är otillräcklig för att förhindra förlusten av ovärderlig kunskap som är nödvändig för framtida juniora kraftledningsprojektörer. Dessutom kan den ökade informella användingen av generativa språkmodeller hota att begrava mänskligt skrivna kunskapsdokument. Detta arbete presenterar ett ramverk d¨ar storskaliga språkmodeller används för att underlätta kunskapsöverföring och tillhandahålla vägledning vid beslutsfattande inom ingenjörsföretag. Med hjälp av detta ramverk utforskar arbetet ytterligare hur datadrivna kunskapsverktyg kan hjälpa juniora kraftledningskonstrukt¨orer genom att stödja informationsinhämtning med hänvisning till kunskapskällorna. En storskalig språkmodells förmåga att hämta relevant kunskap från ett tekniskt designdokument validerades genom att jämföra processen för mänskliga designers som manuellt slutförde en liknande uppgift. I denna utv¨ardering, som involverade sex deltagare och den storskaliga spr˚akmodellen, rankades svaren på frågor om mekanisk dimensionering av stag för kraftledningsstolpar av experter. Resultaten visade att den storskaliga språkmodellens svar i genomsnitt rankades på liknade nivå som de juniora ingenjörerna. Tillsammans med  ett småskaligt demonstrativt kunskapsverktyg, insikter från intervjuer med kraftledningskonstruktörer, litteraturstudier samt resultat från valideringsstudien dras slutsatsen att storskaliga språkmodeller kan stödja kraftledningskonstruktörer via ett kunskapsverktyg. Utöver kraftledningskonstruktion bidrar detta arbete till förståelsen av hur datadrivna språkmodeller kan hjälpa till med kunskapsinhämtning och beslutsfattande  inom andra tekniska designområden. Arbetet använder ett professionellt utbildningsunderlag om mekanisk dimensionering av kraftledningsstolpar i träkonstruktion, inklusive en analys av vertikala- och horistontella linspannets påverkan på stolpens dimension, utvecklat parallellt med detta arbete. Orginaldesigndata från underlaget stödde de tester som genomfördes. Arbetet belyser även risker och etiska aspekter vid implementering av ett sådant kunskapsverktyg. Risker som läckage av sekretessbelagd information betonas, och omfattande system och metoder behövs för att undvika dem. Därför understryks hur viktigt det är att genomföra liknande projekt med noggrannhet, försiktighet och expertis för att undvika skador på företag och samhälle. Lokala språkmodeller eller API-leverantörer med högt förtroende rekommenderas för att minimera risken att känslig information läcker ut till en oönskad tredje part. Med stor försiktighet och hänsyn till riskerna kan ett effektivt kunskapsverktyg bidra till ökad effektivitet, snabbare och mer hållbar utveckling av kraftledningsinfrastruktur, och därmed en snabbare energiomställning.

Denna kvasiexperimentella studie syftade till att undersöka hur en lektionsserie kan struktureras och implementeras med mål att utveckla gymnasieelevers förmåga att använda sig av generativ artificiell intelligens som ett pedagogiskt verktyg. För att möta detta syfte genomfördes tre lektioner om artificiell intelligens, maskininlärning, neurala nätverk och stora språkmodeller med fokus på utveckling av teknisk kunskap och praktiska färdigheter med inslag av etik och kritik. Valet av dessa teman grundades i ett tidigare etablerat ramverk för undervisning inom AIläskunnighet. Vidare teman tas dessa teman upp som del av teknikprogrammet och den kommande AI-kursen enligt Skolverkets förslag. Lektionsseriens påverkan kvantifierades med hjälp av två enkäter – en innan och en efter genomförandet av lektionsserien. Lektionsserien presenterades för två gymnasieklasser vilka bestod av totalt ungefär 50 elever. Urvalet av gymnasieklasserna grundades i deras anslutning till den uppdragsgivande läraren. Vidare valdes respondenterna till enkäten utifrån de elever som fysiskt deltog på den första och sista lektionen och frivilligt valde att svara på enkäten. Dessutom intervjuades fyra tekniklärare för att bättre anpassa lektionsinnehållet till målgruppen. Analysen av svarsfrekvensen till enkätfrågorna visade att lektionsserien hade en statistiskt signifikant påverkan på elevernas tekniska kunskaper, men dess påverkan på elevernas praktiska färdigheter var i stort statistiskt insignifikant. Samtidigt påvisade frekvensanalysen att gymnasieeleverna i regel överskattade sin förmåga att kritiskt granska datorgenererad text och var i stort omedvetna om relevanta etiska frågeställningar. Explorativa faktoranalysen visade att det existerar åtminstone två typer av elever. En elevgrupp av okänd storlek använder sig av stora språkmodeller för att accelerera sina studier genom att lösa problem de annars inte kunde lösa. I detta fall har artificiell intelligens en multiplicerande effekt på elevernas produktivitet. En annan elevgrupp av okänd storlek har i stället som mål att förbättra sina skolresultat genom att använda sig av stora språkmodeller för att lösa deras problem åt dem. Samtidigt överskattar dessa elever sin förmåga att granska datorgenererad text. I detta fall har artificiell intelligens en dämpande effekt på elevernas lärande. Studiens slutsats är att det i dagsläget finns behov för undervisning av gymnasieelever på teknikprogrammet om artificiell intelligens. Detta utrymme kan i stort uppfyllas av en tre lektioner lång lektionsserie. Dock erkänner studien att det finns ytterligare utrymme för praktiska moment där läraren handleder eleverna i deras användning av verktyg såsom ChatGPT. Vidare finns det utrymme för kontinuerligt arbete med kritik och etik, möjligtvis som del av de tidigare nämnda praktiska momenten. / This quais-experimental study aimed to investigate how a series of lessons could be structured and implemented with the goal of developing secondary level students’ ability to use generative artificial intelligence as an educational tool. To meet this goal three lessons on artificial intelligence, machine learning, neural networks, and large language models were conducted, focusing on the development of technical knowledge and practical skills with the inclusion of ethics and critical thinking. The choice of these topics was based on a previously established framework for AI-literacy education. Further, these topics are brought up as part of the Swedish upper secondary school technology programme as well as the upcoming AI-course as per the proposal made by the Swedish Agency for Education. The impact of the lesson series was quantified using two form surveys – one before and one after the implementation of the lesson series. The lesson series was presented to two student classes totalling roughly 50 students. The selection of student classes were based on their affiliation with the assigning teacher. Further, the survey respondents were sampled from the students who physically attended the first and last lesson and voluntarily elected to respond. Additionally, four technology teachers were interviewed to better adapt the teaching material to the student demographic. Response analysis showed that the lesson series had a statistically significant impact on students’ technical knowledge, but its impact on students’ practical skills was largely statistically insignificant. At the same time, the frequency analysis indicated that students generally overestimated their ability to critically evaluate computer-generated text and were largely unaware of relevant ethical issues. Exploratory factor analysis had shown that there exist at least two types of students. A student group of unknown size use large language models to accelerate their studies through solving problems they could not otherwise solve. In this case, artificial intelligence has a multiplying effect on the students’ productivity. Another group of students of unknown size instead use large language models to solve their problems for them with the goal of improving their academic performance. At the same time, these students overestimate their ability to evaluate computer-generated text critically. In this case, artificial intelligence has a dampening effect on the students’ learning. The study concludes that there is a need for teaching secondary level students from the technology programme about artificial intelligence. This space can largely be fulfilled by a series of three lessons. However, the study acknowledges that there remains room for practical activities where the teacher guides students in their use of tools such as ChatGPT. Furthermore, there is room for ongoing work on critical thinking and ethics, possibly as part of the aforementioned practical activities.

Large language models (LLMs) are very large deep learning models which are retrained on a huge amount of data. Among the LLMs are sentence bidirectional encoder representations from transformers (SBERT) where advanced training methods such as transformer-based denoising autoEncoder (TSDAE), generative query network (GenQ) and an adaption of generative pseudo labelling (GPL) can be applied. This thesis project aims to develop a semantic search tool for Swedish legal judgments in order to overcome the limitations of traditional keyword searches in legal document retrieval. For this aim, a model adept at understanding the semantic nuances of legal language has been developed by leveraging natural language processing (NLP) and fine- tuning LLMs like SBERT, using advanced training methods such as TSDAE, GenQ, and an adaption of GPL. To generate labeled data out of unlabelled data, a GPT3.5 model was used after it was fine-tuned. The generation of labeled data with the use of a generative model was crucial for this project to train the SBERT efficiently. The search tool has been evaluated. The evaluation demonstrates that the search tool can accurately retrieve relevant documents based on semantic queries and simnifically improve the efficiency and accuracy of legal research. GenQ has been shown to be the most efficient training method for this use case.

his thesis researches the complex intersection of Data Protection and Intelligent Chatbots (IC)at Scania Group. Developing intelligent chatbots in a secure and GDPR compliant way is highlycomplicated and multifaceted task. The purpose of this research is to provide Scania withorganizational knowledge on how this can be achieved. This study utilizes the Action DesignResearch framework to develop an artifact which integrates technological solutions with dataprotection considerations. By conducting a literature review and semi-structured interviews withemployees at Scania, three potential solutions are identified evaluated: ChatGPT Enterprise, theSecured AI Knowledge Repository (SAIKR), and Techtalker. Each solution offers differentcapabilities and compliance strategies: ChatGPT Enterprise, while practical, relies on contractualassurances for GDPR compliance with data stored in the USA. SAIKR, on the other hand, offersmore control with data stored and encrypted in Sweden, allowing for the use of advancedprivacy-preserving techniques. Techtalker, which is hosted directly by Scania, provides enhancedsecurity measures tailored to specific technical use cases. Based on the artifact and conclusionsof this research, generalized design principles for developing intelligent chatbots within acorporate structure are formulated. These four design principles encourages the utilization ofRAG and LLMs, safe and legal data localization, strong contractual safeguards with third-partyproviders, and a comprehensive risk analysis with stringent security measures.

Computational Linguistics has a long history of applying mathematics to the grammatical and syntactic structure of language; however, applying math to the more complex aspects of language, such as narrative, plot, scenes, character relations, causation, etc. remains a difficult topic. The goal of my research is to bridge the narrative humanities with mathematics, to computationally grasp at these difficult topic, and help develop the field of Narrative Engineering. I view narrative and story with the same mathematical scrutiny as other engineering fields, to take the creativity and fluidity of story and encode it in mathematical representations that have meaning beyond probability and statistical predictions that are the primary function of modern large language models. Included in this research is how stories and narratives are structured, evolve, and change, implying that there exists an inherent narrative computation that we as humans do to merge and combine ideas into new and novel ones. Our thoughts and knowledge and opinions determine the stories we tell, as a combination of everything we have seen, read, heard, and otherwise experienced. Narratives have the ability to inform and change those thoughts and opinions, which then lead to the creation of new and novel narratives. In essence, stories can be seen as a programming language for people. My dissertation, then, is to better understand stories and the environments in which stories are shared. I do this through developing tools that detect, extract, and model aspects of stories and their environments; developing mathematical models of stories and their spread environments; and investigating the impact and effects on stories and their spread environments. I then finish with a discussion on the ethical concerns of research in narrative influence and opinion control.

Digital libraries hold vast and diverse content, with electronic theses and dissertations (ETDs) being among the most diverse. ETDs span multiple disciplines and include unique terminology, making achieving clear and coherent topic representations challenging. Existing topic modeling techniques often struggle with such heterogeneous collections, leaving a gap in providing interpretable and meaningful topic labels. This thesis addresses these challenges through a three-step framework designed to improve topic modeling outcomes for ETD metadata. First, we developed a custom preprocessing pipeline to enhance data quality and ensure consistency in text analysis. Second, we applied and optimized multiple topic modeling techniques to uncover latent themes, including LDA, ProdLDA, NeuralLDA, Contextualized Topic Models, and BERTopic. Finally, we integrated Large Language Models (LLMs), such as GPT-4, using prompt engineering to augment traditional topic models, refining and interpreting their outputs without replacing them. The framework was tested on a large corpus of ETD metadata, including through preliminary testing on a small subset. Quantitative metrics and user studies were used to evaluate performance, focusing on the clarity, accuracy, and relevance of the generated topics. The results demonstrated significant improvements in topic coherence and interpretability, with user study participants highlighting the value of the enhanced representations. These findings underscore the potential of combining customized preprocessing, advanced topic modeling, and LLM-driven refinements to better represent themes in complex collections like ETDs, providing a foundation for downstream tasks such as searching, browsing, and recommendation. / Master of Science / Digital libraries store vast information, including books, research papers, and electronic theses and dissertations (ETDs). ETDs are incredibly diverse, covering most academic fields and using highly specialized language. This diversity makes it challenging to create clear and meaningful summaries of the main themes within these collections. Our study addresses this challenge by developing a three-step framework and applying it to ETDs. First, we cleaned and standardized the data to make it easier to analyze. Second, we used advanced techniques to uncover patterns and group similar topics together. Finally, we improved these topics using powerful tools like GPT-4, which helped make the themes more precise, more accurate, and easier to interpret. We tested this framework on both a small and a large collection of ETDs. Combining quantitative evaluations and user feedback showed that our methods significantly improved how the topics represented the content. This work lays the foundation for more effective future tools to help people search, explore, and navigate large collections of academic works.