Documentation for programming projects can vary both in quality and availability. The availability of documentation can vary more for a closed working environment, since fewer developers will read the documentation. Documenting programming projects can be demanding on worker hours and unappreciated among developers. It is a common conception that developers rather invest time on developing a project than documenting a project, and making the documentation process more effective would benefit developers. To move towards a more automated process of writing documentation, this work generated documentation for repositories which attempts to summarize the repositories in their use cases and functionalities. Two different implementations are created to generate documentation using an on-premise hosted large language model (LLM) as a tool. First, the embedded solution processes all available code in a project and creates the documentation based on multiple summarizations of files and folders. Second, the RAG solution attempts to use only the most important parts of the code and lets the LLM create the documentation on a smaller set of the codebase. The results show that generating documentation is possible, but unreliable and must be controlled by a person with knowledge about the codebase. The embedded solution seems to be more reliable and produce better results, but is more costly compared to the RAG solution.

This study investigates the potential of Large Language Models (LLMs) to automate and enhance software maintenance tasks, focusing on bug detection and code refactoring. Traditional software maintenance, which includes debugging and code optimization, is time-consuming and prone to human error. With advancements in artificial intelligence, LLMs like ChatGPT and Copilot offer promising capabilities for automating these tasks. Through a series of quasi-experiments, we evaluate the effectiveness of ChatGPT 3.5, ChatGPT 4 (Grimoire GPT), and GitHub Copilot. Each model was tested on various code snippets to measure their ability to identify and correct bugs and refactor code while maintaining its original functionality. The results indicatethat ChatGPT 4 (Grimoire GPT) outperforms the other models, demonstrating superior accuracy and effectiveness, with success percentages of 87.5% and 75% in bug detection and code refactoring respectively. This research highlights the potential of advanced LLMs to significantly reduce the time and cost associated with software maintenance, though human oversight is still necessary to ensure code integrity. The findings contribute to the understanding of LLM capabilities in real-world software engineering tasks and pave the way for more intelligent and efficient software maintenance practices.

This study investigates the capacity of Large Language Models (LLMs), specifically ChatGPT 3.5 and 4, to evaluate citations in scholarly papers. Given the importance of accurate citations in academic writing, the goal was to determine how well these models can assist in verifying citations. A series of experiments were conducted using a dataset of our own creation. This dataset includes the three main citation categories: Direct Quotation, Paraphrasing, and Summarising, along with subcategories such as minimal and long source text.  In the preliminary experiment, ChatGPT 3.5 demonstrated perfect accuracy, while ChatGPT 4 showed a tendency towards false positives. Further experiments with an extended dataset revealed that ChatGPT 4 excels in correctly identifying valid citations, particularly with longer and more complex texts, but is also more prone to wrong predictions. ChatGPT 3.5, on the other hand, provided a more balanced performance across different text lengths, with both models achieving an accuracy rate of 90.7%. The reliability experiments indicated that ChatGPT 4 is more consistent in its responses compared to ChatGPT 3.5, although it also had a higher rate of consistent wrong predictions.  This study highlights the potential of LLMs to assist scholars in citation verification, suggesting a hybrid approach where ChatGPT 4 is used for initial scans and ChatGPT 3.5 for final verification, paving the way for automating this process. Additionally, this study contributes a dataset that can be further expanded and tested on, offering a valuable resource for future research in this domain.

The Thermo-Calc software is a key tool in the research process for many material engineers. However, integrating multiple modules in Thermo-Calc requires the user to write code in a Python-based language, which can be challenging for novice programmers. This project aims to enable the generation of such code from user prompts by using existing generative AI models. In particular, we use a retrieval-augmented generation architecture applied to LLaMA and Mistral models. We use Code LLaMA-Instruct models with 7, 13, and 34 billion parameters, and a Mistral-Instruct model with 7 billion parameters. These models are all based on LLaMA 2. We also use a LLaMA 3-Instruct model with 8 billion parameters. All these models are instruction-tuned, which suggests that they have the capability to interpret natural language and identify appropriate options for a command-line program such as Python. In our testing, the LLaMA 3-Instruct model performed best, achieving 53% on the industry benchmark HumanEval and 49% on our internal adequacy assessment at pass@1, which is the expected probability of getting a correct solution when generating a response. This indicates that the model generates approximately every other answer correct. Due to GPU memory limitations, we had to apply quantisation to process the 13 and 34 billion parameter models. Our results revealed a mismatch between model size and optimal levels of quantisation, indicating that reduced precision adversely affects the performance of these models. Our findings suggest that a properly customised large language model can greatly reduce the coding effort of novice programmers, thereby improving productivity in material research.

This bachelor’s thesis investigates the efficacy of cutting-edge Large Language Models (LLMs) — GPT-4, Code Llama Instruct (7B parameters), and Gemini 1.0 — in detecting and correcting bugs in Java and Python code. Through a controlled experiment using standardized prompts and the QuixBugs dataset, each model's performance was analyzed and compared. The study highlights significant differences in the ability of these LLMs to correctly identify and fix programming bugs, showcasing a comparative advantage in handling Python over Java. Results suggest that while all these models are capable of identifying bugs, their effectiveness varies significantly between models. The insights gained from this research aim to aid software developers and AI researchers in selecting appropriate LLMs for integration into development workflows, enhancing the efficiency of bug management processes.

Användningen av Artificiell Intelligens (AI) är utbredd bland verksamma företag idag, likväl privatpersoner. Det har blivit en integrerad del av vårt samhälle som ofta går obemärkt förbi. Allt från face recognition, självkörande bilar och automatisering inom arbetsrelaterade områden, har AI onekligen påverkat omvärlden. I takt med att AI-modeller fortsätter att utvecklas tillkommer även farhågor om dess påverkan på jobb, tillhörande säkerhetsrisker och etiska dilemman. Uppsatsens litteratur hjälper till att skildra AI historiskt, i nutid, men även ge en uppfattning om vart den är på väg. Den AI-modell som i nuläget har väckt störst uppmärksamhet är ChatGPT. Dess potential tycks inte ha några gränser, därmed uppstod relevansen för att öka kunskapen kring AI-modellen. Vidare gjordes en avgränsning, där fokusområdet var att undersöka hur ChatGPT kan generera kodkommentarer och potentiellt agera som ett hjälpmedel vid kommentering av källkod. I samband med avgränsningen och fokusområdet bildades även forskningsfrågan: Large Language Models: Bedömning av ChatGPT:s potential som verktyg för kommentering av kod För att besvara forskningsfrågan har avhandlingen varit baserat på en kvalitativ ansats, där urvalet av respondenter har varit programmerare. Den primära datainsamlingen har genomförts via två semistrukturerade intervjuer, varav den inledande innefattade initiala känslor kring ChatGPT och övergripande fakta om respektive intervjuobjekt. Vidare gjordes det en observation för att få en inblick i hur AI-modellen används av programmerare, för att avslutningsvis göra en uppföljande intervju post-observation i syfte att samla tankarna från intervjuobjekten efter användning av ChatGPT för att generera kodkommentarer. Baserat på den insamlade empirin kunde studien konkludera vissa begränsningar i den nuvarande modellen, inte minst behovet av tydliga instruktioner. Trots brister visar ChatGPTs framställning potential att vara en betydande resurs för kommentering av kod i framtiden. Resultaten indikerar att modellen kan generera relativt passande kommentarer i de analyserade kodkodstycken. Emellertid uttryckte deltagarna under de avslutande intervjuerna generellt sett att kommentarerna var redundanta och saknade betydande värde för att öka förståelsen av källkoden. Respondenterna diskuterade dock möjligheterna att använda ChatGPT i framtiden, men underströk behovet av förbättringar för att göra det till en tillförlitlig metod inom arbetsrelaterade situationer. / The usage of Artificial Intelligence (AI) is widespread among both companies and individuals today. It has become an integrated part of our society, often going unnoticed. From face recognition and self-driving cars to automation in work-related areas, AI has undeniably impacted the world. As AI models continue to evolve, concerns about their impact on jobs, associated security risks, and ethical dilemmas arise. The literature in this essay helps portray AI historically, in the present, and provides an insight into its future direction. The AI model that has currently garnered the most attention is ChatGPT. Its potential seems limitless, which prompted the relevance of increasing knowledge about the AI model. Furthermore, a delimitation was made, where the focus area was to investigate how ChatGPT can generate code comments and potentially act as a tool for commenting source code. As part of the research focus and scope, the research question was formulated: "Large Language Models: Assessment of ChatGPT's Potential as a Tool for Code Commenting." To answer the research question, the thesis adopted a qualitative approach, with programmers as the selected respondents. The primary data collection was conducted through two semi-structured interviews, where the initial interview involved capturing initial impressions of ChatGPT and gathering general information about the interviewees. Additionally, an observation was carried out to gain insights into how programmers utilize the AI model, followed by a post-observation interview to gather the interviewees' thoughts after using ChatGPT to generate code comments. Based on the collected empirical data, the study was able to conclude certain limitations in the current model, particularly the need for clear instructions. Despite these limitations, ChatGPT's performance demonstrates the potential to be a significant resource for code commenting in the future. The results indicate that the model can generate relatively suitable comments in the analyzed code snippets. However, during the concluding interviews, participants generally expressed that the comments were redundant and lacked significant value in enhancing the understanding of the source code. Nevertheless, the respondents 2 discussed the possibilities of using ChatGPT in the future, while emphasizing the need for improvements to establish it as a reliable method in work-related situations.

The knowledge of tests conducted on a problematic vehicle is essential for enhancing the efficiency of mechanics. Therefore, identifying the tests performed in each repair case is of utmost importance. This thesis explores techniques for extracting data from unstructured repair reports to identify component tests. The main emphasis is on developing a supervised multi-class classifier to categorize data and extract sentences that describe repair diagnoses and actions. It has been shown that incorporating a category-aware contrastive learning objective can improve the repair report classifier’s performance. The proposed approach involves training a sentence representation model based on a pre-trained model using a category-aware contrastive learning objective. Subsequently, the sentence representation model is further trained on the classification task using a loss function that combines the cross-entropy and supervised contrastive learning losses. By applying this method, the macro F1-score on the test set is increased from 90.45 to 90.73. The attempt to enhance the performance of the repair report classifier using a noisy data classifier proves unsuccessful. The noisy data classifier is trained using a prompt-based fine-tuning method, incorporating open-ended questions and two examples in the prompt. This approach achieves an F1-score of 91.09 and the resulting repair report classification datasets are found easier to classify. However, they do not contribute to an improvement in the repair report classifier’s performance. Ultimately, the repair report classifier is utilized to aid in creating the input necessary for identifying component tests. An information retrieval method is used to conduct the test identification. The incorporation of this classifier and the existing labels when creating queries leads to an improvement in the mean average precision at the top 3, 5, and 10 positions by 0.62, 0.81, and 0.35, respectively, although with a slight decrease of 0.14 at the top 1 position.

As advancements in chatbots and Large Language Models (LLMs) such as GPT-3.5 and GPT-4 continue, their applications in diverse fields, including psychology, expand. This study investigates the effectiveness of LLMs optimized through prompt engineering, aiming to enhance their performance in psychological applications. To this end, two distinct versions of a GPT-3.5-based chatbot were developed: a version similar to the base model, and a version equipped with a more extensive system prompt detailing expected behavior. A panel of professional psychologists evaluated these models based on a predetermined set of questions, providing insight into their potential future use as psychological tools. Our results indicate that an overly prescriptive system prompt can unintentionally limit the versatility of the chatbot, making a careful balance in instruction specificity essential. Furthermore, while our study suggests that current LLMs such as GPT-3.5 are not capable of fully replacing human psychologists, they can provide valuable assistance in tasks such as basic question answering, consolation and validation, and triage. These findings provide a foundation for future research into the effective integration of LLMs in psychology and contribute valuable insights into the promising field of AI-assisted psychological services. / I takt med att framstegen inom chatbots och stora språkmodeller (LLMs) som GPT-3.5 och GPT-4 fortsätter utvidgas deras potentiella tillämpningar inom olika områden, inklusive psykologi. Denna studie undersöker effektiviteten av LLMs optimerade genom prompt engineering, med målet att förbättra deras prestanda inom psykologiska tillämpningar. I detta syfte utvecklades två distinkta versioner av en chatbot baserad på GPT-3.5: en version som liknar bas-modellen, och en version utrustad med en mer omfattande systemprompt som detaljerar förväntat beteende. En panel av professionella psykologer utvärderade dessa modeller baserat på en förbestämd uppsättning frågor, vilket ger inblick i deras potentiella framtida användning som psykologiska verktyg. Våra resultat tyder på att en överdrivet beskrivande systemprompt kan ofrivilligt begränsa chatbotens mångsidighet, vilket kräver en noggrann balans i specificiteten av prompten. Vidare antyder vår studie att nuvarande LLMs som GPT-3.5 inte kan ersätta mänskliga psykologer helt och hållet, men att de kan ge värdefull hjälp i uppgifter som grundläggande frågebesvaring, tröst och bekräftelse, samt triage. Dessa resultat ger en grund för framtida forskning om effektiv integration av LLMs inom psykologi och bidrar med värdefulla insikter till det lovande fältet av AI-assisterade psykologtjänster.

The engineering design process can underutilize nature-based solutions during infrastructure development. Instances of nature within the built environment are reflections of the human-nature connection, which may alter how designers ideate solutions to a given design task, especially through virtual reality (VR) as an embodied perspective taking platform. Embodied VR helps designers "see" as an end-user sees, inclusive of the natural environment through the uptake of an avatar, such as a bird or fish. Embodied VR emits empathy toward the avatar, e.g., to see as a bird in VR, one tends to feel and think as a bird. Furthermore, embodied VR also impacts altruistic behavior toward the environment, specifically through proenvironmental behaviors. However, limited research discovers the impact of embodied VR on the human-nature connection and if embodied VR has any impact on how designers ideate, specifically surrounding nature-based solutions as a form of a proenvironmental behavior during the design process. This research first presents a formal measurement of embodied VR's impact on the human-nature connection and maps this impact toward design-related proenvironmental behaviors through design ideas, i. e., tracking changes in nature-based design choices. The design study consisted of three groups of engineering undergraduate students which were given a case study and plan review: a VR group embodying a bird (n=35), a self-lens VR group (n=34), and a control group (n=33). The case study was about a federal mandate to minimize combined sewer overflow in a neighborhood within Cincinnati, OH. Following the plan review, VR groups were given a VR walkthrough or flythrough of the case study area of interest as a selected avatar (embodied:bird, self-lens:oneself). Participants were tested for their connectedness to nature and a mock-design charrette was held to measure engineering design ideas. Verbal protocol analysis was followed, instructing participants to think aloud. Design ideation sessions were recorded and manually transcribed. The results of the study indicated that embodiment impacts the human-nature connection based on participants' perceived connection to nature. Only the bird group witnessed an increase in connectedness to nature, whereas the self-lens and control groups did not report any change. This change in connectedness to nature was also confirmed by engineering design ideas. The bird group was more likely to ideate green-thinking designs to solve the stormwater issue and benefit both nature and socioeconomic conditions, whereas the control group mostly discussed gray designs as the catalyst for minimizing combined sewer overflows. The self-lens group also mentioned green design ideas as well as socioeconomic change, but mostly placed the beneficiary of the design toward people rather than nature in the bird group. The mode of analysis for these findings was driven by thematic content analysis, an exploration of design space as a function of semantic distance, and large language models (LLMs) to synthesize design ideas and themes. An LLM's performance lent accuracy to the design ideas in comparison to thematic content analysis, but struggled to cross-compare groups to provide generalizable findings. This research is intended to benefit the engineering design process with a) the benefit of perspective-taking on design ideas based on lenses of embodied VR and b) various methods to supplement thematic content analysis for coding design ideas. / Doctor of Philosophy / The use of nature in the constructed world, such as rain gardens and natural streams for moving stormwater, is underused during the design process. Virtual reality (VR) programs, like embodiment, have the potential to increase the incorporation of nature and nature-based elements during design. Embodiment is the process of taking on the vantage point of another being or avatar, such as a bird, fish, insect, or other being, in order to see and move as the avatar does. Embodied VR increases the likelihood that the VR participant will act favorably to the subject, specifically when the natural environment is involved. For example, embodying another individual cutting down trees in a virtual forest increased the likelihood that individuals would act favorably to the environment, such as through recycling or conserving energy (Ahn and Bailenson, 2012). Ultimately, this research measures the level of connection participants feel with the environment after an embodied VR experience and motions to discover if this change in connection to nature impacts how participants might design a solution to a problem. This design experiment is based on a case study, which all participants were provided alongside supplemental plan documents of the case. The case study used is about stormwater issues and overflows from infrastructure in a neighborhood in Cincinnati, OH, where key decision-makers were mandated by the federal government to minimize the overflows. The bird group (a bird avatar) performed a fly-through in the area of interest in VR, whereas the self-lens group (first-person, embodying oneself) walked through the same area. The control group received no VR intervention. Following the intervention, participants were asked to re-design the neighborhood and orate their recorded solution. Then, participants were required to score a questionnaire measuring their connectedness to nature. The results show that when people experience the space as a bird in virtual reality, they felt more connected to nature and also included more ideas related to nature in their design. More specifically, ideas involving green infrastructure (using nature-based elements, e.g., rain gardens and streams) and socioeconomic benefits were brought up by the bird group. This research presents embodiment as a tool that can change how engineers design. As stormwater policy has called for more use of green infrastructure (notably, through the Environmental Protection Agency), embodiment may be used during the design process to meet this call from governmental programs. Furthermore, this research impacts how embodiment's effects on design can be interpreted, specifically through quantitative methods through natural language processing and the use of large language models to analyze data and report back on design-related findings. This research is intended to benefit the design process with a) using different avatars in embodiment to impact design ideas and b) a comparison of thematic content analysis and large language models in summarizing design ideas and themes.

This thesis aims to explore and gain an understanding of the current AI landscape within Swedish Cultural Heritage using purposive interviews with five cultural heritage institutions with ongoing AI projects. This study fills a knowledge gap in the practical implementation of AI at Swedish institutions in addition to the sustainable use of technologies for cultural heritage. The overarching discussion further includes related topics of ethical AI and long-term sustainability, framing it from a perspective of Information Practices and a socio-material entanglement. Findings show that AI technologies can play an important part in cultural heritage, with a range of practical applications if certain issues are overcome. Moreover, the utilisation of AI will increase. The study also indicates a need for regulations, digitisation efforts, and increased investments in resources to adopt the technologies into current practices sustainably. The conclusion highlights a need for the cultural heritage sector to converge and find collectively applicable solutions for implementing AI.