Using AI to automatically describe images is a challenging task. The aim of this study has been to compare the use of character-based language models with one of the current state-of-the-art token-based language models, im2txt, to generate image captions, with focus on morphological correctness. Previous work has shown that character-based language models are able to outperform token-based language models in morphologically rich languages. Other studies show that simple multi-layered LSTM-blocks are able to learn to replicate the syntax of its training data. To study the usability of character-based language models an alternative model based on TensorFlow im2txt has been created. The model changes the token-generation architecture into handling character-sized tokens instead of word-sized tokens. The results suggest that a character-based language model could outperform the current token-based language models, although due to time and computing power constraints this study fails to draw a clear conclusion. A problem with one of the methods, subsampling, is discussed. When using the original method on character-sized tokens this method removes characters (including special characters) instead of full words. To solve this issue, a two-phase approach is suggested, where training data first is separated into word-sized tokens where subsampling is performed. The remaining tokens are then separated into character-sized tokens. Future work where the modified subsampling and fine-tuning of the hyperparameters are performed is suggested to gain a clearer conclusion of the performance of character-based language models.

Les modèles de langue pré-entraînés ont montré leur efficacité dans beaucoup de tâches de traitement de la langue naturelle. Ces modèles peuvent capter des régularités générales d'une langue à partir d'un grand ensemble de textes, qui sont utiles dans la plupart des applications en traitement de langue naturelle. Dans ce mémoire, nous étudions les problèmes de dialogue, i.e. générer une réponse à un énoncé de l'utilisateur. Nous exploitons les modèles de langue pré-entraînés pour traiter différents aspects des systèmes de dialogue. Premièrement, les modèles de langue pré-entraînés sont entraînés and utilisés dans les systèmes de dialogue de différentes façons. Il n'est pas clair quelle façon est la plus appropriée. Pour le dialogue orienté-tâche, l’approche de l'état de l'art pour le suivi de l'état de dialogue (Dialogue State Tracking) utilise BERT comme encodeur et empile un autre réseau de neurones récurrent (RNN) sur les sorties de BERT comme décodeur. Dans ce cas, seul l'encodeur peut bénéficier des modèles de langue pré-entraînés. Dans la première partie de ce mémoire, nous proposons une méthode qui utilise un seul modèle BERT pour l'encodeur et le décodeur, permettant ainsi un ajustement de paramètres plus efficace. Notre méthode atteint une performance qui dépasse l'état de l'art. Pour la tâche de génération de réponses dans un chatbot, nous comparons 4 approches communément utilisées. Elles sont basées sur des modèles pré-entraînés et utilisent des objectifs et des mécanismes d'attention différents. En nous appuyant sur des expérimentations, nous observons l'impact de deux types de disparité qui sont largement ignorées dans la littérature: disparité entre pré-entraînement et peaufinage, et disparité entre peaufinage et génération de réponse. Nous montrons que l'impact de ces disparités devient évident quand le volume de données d’entraînement est limité. Afin de remédier à ce problème, nous proposons deux méthodes qui réduisent les disparités, permettant d'améliorer la performance. Deuxièmement, même si les méthodes basées sur des modèles pré-entraînés ont connu de grands succès en dialogue général, nous devons de plus en plus traiter le problème de dialogue conditionné, c'est-à-dire dialogue en relation à une certaine condition (qui peut désigner un personnage, un sujet, etc.). Des chercheurs se sont aussi intéressés aux systèmes de chatbot avec des habiletés de conversation multiples, i.e. chatbot capable de confronter différentes situations de dialogues conditionnés. Ainsi, dans la seconde partie de ce mémoire, nous étudions le problème de génération de dialogue conditionné. D'abord, nous proposons une méthode générale qui exploite non seulement des données de dialogues conditionnées, mais aussi des données non-dialogues (textes) conditionnées. Ces dernières sont beaucoup plus faciles à acquérir en pratique. Ceci nous permet d'atténuer le problème de rareté de données. Ensuite, nous proposons des méthodes qui utilisent le concept d'adaptateur proposé récemment dans la littérature. Un adaptateur permet de renforcer un système de dialogue général en lui donnant une habileté spécifique. Nous montrons que les adaptateurs peuvent encoder des habiletés de dialogue conditionné de façon stricte ou flexible, tout en utilisant seulement 6% plus de paramètres. Ce mémoire contient 4 travaux sur deux grands problèmes de dialogue: l'architecture inhérente du modèle de dialogue basé sur des modèles de langue pré-entraînés, et l'enrichissement d'un système de dialogue général pour avoir des habiletés spécifiques. Ces travaux non seulement nous permettent d'obtenir des performances dépassant de l'état de l'art, mais aussi soulignent l'importance de concevoir l'architecture du modèle pour bien correspondre à la tâche, plutôt que simplement augmenter le volume de données d'entraînement et la puissance de calcul brute. / Pre-trained language models (LMs) have shown to be effective in many NLP tasks. They can capture general language regularities from a large amount of texts, which are useful for most applications related to natural languages. In this thesis, we study the problems of dialogue, i.e. to generate a response to a user's utterance. We exploit pre-trained language models to deal with different aspects of dialogue systems. First, pre-trained language models have been trained and used in different ways in dialogue systems and it is unclear what is the best way to use pre-trained language models in dialogue. For task-oriented dialogue systems, the state-of-the-art framework for Dialogue State Tracking (DST) uses BERT as the encoder and stacks an RNN upon BERT outputs as the decoder. Pre-trained language models are only leveraged for the encoder. In the first part of the thesis, we investigate methods using a single BERT model for both the encoder and the decoder, allowing for more effective parameter updating. Our method achieves new state-of-the-art performance. For the task of response generation in generative chatbot systems, we further compare the 4 commonly used frameworks based on pre-trained LMs, which use different training objectives and attention mechanisms. Through extensive experiments, we observe the impact of two types of discrepancy: pretrain-finetune discrepancy and finetune-generation discrepancy (i.e. differences between pre-training and fine-tuning, and between fine-tuning and generation), which have not been paid attention to. We show that the impact of the discrepancies will surface when limited amount of training data is available. To alleviate the problem, we propose two methods to reduce discrepancies, yielding improved performance. Second, even though pre-training based methods have shown excellent performance in general dialogue generation, we are more and more faced with the problem of conditioned conversation, i.e. conversation in relation with some condition (persona, topic, etc.). Researchers are also interested in multi-skill chatbot systems, namely equipping a chatbot with abilities to confront different conditioned generation tasks. Therefore, in the second part of the thesis, we investigate the problem of conditioned dialogue generation. First, we propose a general method that leverages not only conditioned dialogue data, but also conditioned non-dialogue text data, which are much easier to collect, in order to alleviate the data scarcity issue of conditioned dialogue generation. Second, the concept of Adapter has been recently proposed, which adapts a general dialogue system to enhance some dialogue skill. We investigate the ways to learn a dialogue skill. We show that Adapter has enough capacity to model a dialogue skill for either loosely-conditioned or strictly-conditioned response generation, while using only 6% more parameters. This thesis contains 4 pieces of work relating to the two general problems in dialogue systems: the inherent architecture for dialogue systems based on pre-trained LMs, and enhancement of a general dialogue system for some specific skills. The studies not only propose new approaches that outperform the current state of the art, but also stress the importance of carefully designing the model architecture to fit the task, instead of simply increasing the amount of training data and the raw computation power.

Teknologiutvecklingen av stora språkmodeller har på senaste tiden blivit uppmärksammad genom lanseringen av OpenAI:s ChatGPT. Det har förekommit diskussioner om vad detta innebär för samhället i stort men också hur utbildningen på lärosäten påverkas. Syftet med denna studie var att studera hur stor påverkan dessa verktyg har på utbildningen på Uppsala universitet. Fem studierektorer från olika institutioner har intervjuats. Datan analyserades sedan med hjälp av teorin för socialt disruptiva teknologier för att undersöka hur stor påverkansgraden är. Resultatet visar att det främst är examinationer som har påverkats, där vissa studierektorer har behövt ta bort eller kommer att ta bort hemuppgifter som konsekvens av ChatGPT. Skillnader i förändringsarbetet finns mellan olika institutioner, vilket tycks delvis grunda sig i brist på riktlinjer, men även i utbildningsstruktur och personligt engagemang. Det går dock inte att fastslå några systematiska skillnader mellan universitetets olika delar. Vidare har det diskuterats bredare frågor om studenternas lärande och hur man som studierektor kan förhålla sig till utvecklingen. / The technology development of large language models has recently received attention through the launch of OpenAI’s ChatGPT. There have been discussions of what this means for society overall, but also how the education at universities is affected. The purpose of this study was to study how much impact these tools have on education at Uppsala University. Five directors of studies from different departments have been interviewed. The data was then analyzed using the theory of socially disruptive technologies to investigate the degree of impact. The result shows that it is mainly examinations that have been affected, where some principals have had to remove or will remove homework assignments as a consequence of ChatGPT. Differences in change management exist between different institutions, which seem to be partly due to the lack of guidelines, but also due to educational structure and personal commitment. However, no systematic differences can be determined between the different parts of the university. Furthermore, there have been discussions about broader questions about the students' learning and how one should relate to the development as a director of studies.

Chatbots are becoming increasingly popular in various industries, and thereare many options available for businesses and organisations. Several studieshave investigated open-source chatbots and identified their core strengths,implementation, and integration capabilities however few have investigatedopen-source chatbot frameworks and libraries in a specific use case such asmedicine. The project's objective was to evaluate a selection of chatbots ormore specifically two frameworks: Botkit and Rasa, and two libraries:ChatterBot, and Natural which was utilised together with Botkit and alanguage model which is DialoGPT. The evaluation focuses specifically onaccuracy, consistency, and response time. Frequently asked questions fromthe World Health Organization and COVID-19 related Dialogue Datasetfrom GitHub were utilised to test the chatbots' abilities in handling differentqueries and accuracy was measured through metrics like Jaccard similarity,bilingual evaluation understudy (BLEU), and recall oriented gistingevaluation (ROUGE) scores, consistency through Jaccard similarity betweenthe generated responses and response time was taken to be the average timefor a response in seconds. The analysis revealed unique strengths andlimitations for each chatbot model. Rasa displayed robust performance inaccuracy, consistency, and customisation capabilities if the chatbot works ina particular topic with acceptable response times. DialoGPT demonstratedstrong conversational abilities and contextually relevant responses withtrade-offs in consistency. ChatterBot showed consistency, though sometimesstruggled with advanced queries, and Botkit with Natural stood out for itsquick response times, albeit with limitations in accuracy and scalability.Despite implementation challenges, these open-source frameworks, libraries,and models offer promising solutions for organisations intending to harnessconversational agents' technology. The study suggests encouraging furtherexploration and refinement in this rapidly evolving field.

Text-to-SQL, which involves translating natural language into Structured Query Language (SQL), is crucial for enabling broad access to structured databases without expert knowledge. However, designing models for such tasks is challenging due to numerous factors, including the presence of ’noise,’ such as ambiguous questions and syntactical errors. This thesis provides an in-depth analysis of the distribution and types of noise in the widely used BIRD-Bench benchmark and the impact of noise on models. While BIRD-Bench was created to model dirty and noisy database values, it was not created to contain noise and errors in the questions and gold queries. We found after a manual evaluation that noise in questions and gold queries are highly prevalent in the financial domain of the dataset, and a further analysis of the other domains indicate the presence of noise in other parts as well. The presence of incorrect gold SQL queries, which then generate incorrect gold answers, has a significant impact on the benchmark’s reliability. Surprisingly, when evaluating models on corrected SQL queries, zero-shot baselines surpassed the performance of state-of-the-art prompting methods. The thesis then introduces the concept of classifying noise in natural language questions, aiming to prevent the entry of noisy questions into text-to-SQL models and to annotate noise in existing datasets. Experiments using GPT-3.5 and GPT-4 on a manually annotated dataset demonstrated the viability of this approach, with classifiers achieving up to 0.81 recall and 80% accuracy. Additionally, the thesis explored the use of LLMs for automatically correcting faulty SQL queries. This showed a 100% success rate for specific query corrections, highlighting the potential for LLMs in improving dataset quality. We conclude that informative noise labels and reliable benchmarks are crucial to developing new Text-to-SQL methods that can handle varying types of noise.

This thesis explores the opportunities and challenges of integrating recent Natural Language Processing (NLP) Artificial Intelligence (AI) advancements into IT project management and delivery (PM&D). Using a qualitative design through hermeneutic phenomenology strategy, the study employs a semi-systematic literature review and semi-structured interviews to delve into NLP AI's potential impacts in IT PM&D, from both theoretical and practical standpoints. The results revealed numerous opportunities for NLP AI application across Project Performance Domains, enhancing areas such as stakeholder engagement, team productivity, project planning, performance measurement, project work, delivery, and risk management. However, challenges were identified in areas including system integration, value definition, team and stakeholder-related issues, environmental considerations, and ethical concerns. In-house and third-party model usage also presented their unique set of challenges, emphasizing cost implications, data privacy and security, result quality, and dependence issues. The research concludes the immense potential of NLP AI in IT PM&D is tempered by these challenges, and calls for robust strategies, sound ethics, comprehensive training, new ROI evaluation frameworks, and responsible AI usage to effectively manage these issues. This thesis provides valuable insights to academics, practitioners, and decision-makers navigating the rapidly evolving landscape of NLP AI in IT PM&D.

In recent years, artificial intelligence (AI) has become increasingly popular. An area where AI technology is used and has received much attention during the past year is chatbots. They can simulate an understanding of human language and form text responses to questions asked. Apart from generating text responses, they can also generate programming code, making them useful for tasks such as testing. Although testing is considered a crucial part of software development, many find it tedious and time-consuming. There are currently limited AI-powered tools for generating unit tests in general and even fewer for the programming language Java. The thesis tackles the problem of the lack of tools for generating unit tests in Java that explore the capabilities of AI, and a research question is introduced thereafter. The purpose of this thesis is to address the issue by creating a prototype for generating unit tests in Java based on the AI model, GPT-3.5-Turbo. The goal is to provide a basis for other professionals to create tools for generating unit tests, which was done by experimenting with different prompts and values of a randomness parameter and then suggesting the prototype JUTAI. A quantitative research method with an experimental and comparative approach was used to evaluate the results. A comparison model with three criteria was brought forward to evaluate the results. The findings reveal that JUTAI outperformed the general-purpose AI tool, ChatGPT, across all three criteria and indicate that the goal of this thesis is achieved and the research question answered. / Intresset för artificiell intelligens (AI) har ökat de senaste åren. Ett område där AI- teknologi används och som har fått mycket uppmärksamhet under det senaste året är chattbottar. De kan simulera en förståelse för mänskligt språk och svara på frågor i textformat. Utöver det kan de även generera programkod. Tack vare förmågan att generera kod kan de användas för testning. Även om testning anses vara en viktig del av mjukvaruutveckling, tycker många att det är tråkigt och tidskrävande. För närvarande finns det ett begränsat antal verktyg som kan generera enhetstester, och det finns ännu färre verktyg som kan göra detta i Java. Detta examensarbete tog sig an problemet med bristen på AI-verktyg för enhetstestning i Java genom att besvara på forskningsfrågan som ställdes. Syftet med examensarbetet är att föreslå en lösning på problemet genom att utveckla en prototyp som använder sig av AI- modellen GPT-3.5-Turbo för att generera enhetstester i Java. Målet är att ge en grund för andra yrkesverksamma att skapa verktyg för att generera enhetstester, vilket gjordes genom att experimentera med olika instruktionstrukturer och värden för en slumpmässighetsparameter, och sedan föreslå protypen JUTAI. En kvantitativ forskningsmetod tillsammans med en experimentell och jämförande ansats användes för att utvärdera resultaten. En jämförelsemodell med tre kriterier togs fram för att utvärdera resultaten. Resultaten visar att JUTAI presterade bättre än AI-verktyget ChatGPT i de tre kriterierna och indikerar att målet med detta examensarbete uppnåddes och forskningsfrågan besvarades.

Auto-completion is a task that requires an algorithm to give suggestions for completing sentences. Specifically, the history of live chat and the words already typed by the agents are provided to the algorithm for outputting the suggestions to finish the sentences. This study aimed to investigate if the above task can be handled by fine-tuning a pre-trained T5 model on the target dataset. In this thesis, both an English and a Portuguese dataset were selected. Then, T5 and its multilingual version mT5were fine-tuned on the target datasets. The models were evaluated with different metrics (log perplexity, token level accuracy, and multi-word level accuracy), and the results are compared to those of the baseline methods. The results on these different metrics show that a method based on pre-trained T5 is a promising approach to handle the target task. / Automatisk komplettering är en uppgift som kräver en algoritm för att ge förslag på hur man kan slutföra meningar. Specifikt levereras historien om livechatt och de ord som redan har skrivits av agenterna till algoritmen för att mata ut förslagen för att avsluta meningarna. Denna studie syftade till att undersöka om ovanstående uppgift kan hanteras genom att finjustera en förtränad T5-modell på måldatamängden. I denna avhandling valdes både en engelsk och en portugisisk datamängd. Därefter finjusterades T5 och dess flerspråkiga version mT5 på måldatauppsättningarna. Modellerna utvärderades med olika mätvärden (log-perplexitet, precision på ordnivå och flerordsnivå), och resultaten jämförs med baslinjemetoderna. Resultaten på dessa olika mätvärden visar att en metod baserad på en förtränad T5 är ett lovande tillvägagångssätt för att hantera uppgiften.

<p dir="ltr">Semantic pairs, which consist of related entities or concepts, serve as the foundation for comprehending the meaning of language in both written and spoken forms. These pairs enable to grasp the nuances of relationships between words, phrases, or ideas, forming the basis for more advanced language tasks like entity recognition, sentiment analysis, machine translation, and question answering. They allow to infer causality, identify hierarchies, and connect ideas within a text, ultimately enhancing the depth and accuracy of automated language processing.</p><p dir="ltr">Nevertheless, the task of extracting semantic pairs from sentences poses a significant challenge, necessitating the relevance of syntactic dependency patterns (SDPs). Thankfully, semantic relationships exhibit adherence to distinct SDPs when connecting pairs of entities. Recognizing this fact underscores the critical importance of extracting these SDPs, particularly for specific semantic relationships like hyponym-hypernym, meronym-holonym, and cause-effect associations. The automated extraction of such SDPs carries substantial advantages for various downstream applications, including entity extraction, ontology development, and question answering. Unfortunately, this pivotal facet of pattern extraction has remained relatively overlooked by researchers in the domains of natural language processing (NLP) and information retrieval.</p><p dir="ltr">To address this gap, I introduce an attention-based supervised deep learning model, ASPER. ASPER is designed to extract SDPs that denote semantic relationships between entities within a given sentential context. I rigorously evaluate the performance of ASPER across three distinct semantic relations: hyponym-hypernym, cause-effect, and meronym-holonym, utilizing six datasets. My experimental findings demonstrate ASPER's ability to automatically identify an array of SDPs that mirror the presence of these semantic relationships within sentences, outperforming existing pattern extraction methods by a substantial margin.</p><p dir="ltr">Second, I want to use the SDPs to extract semantic pairs from sentences. I choose to extract cause-effect entities from medical literature. This task is instrumental in compiling various causality relationships, such as those between diseases and symptoms, medications and side effects, and genes and diseases. Existing solutions excel in sentences where cause and effect phrases are straightforward, such as named entities, single-word nouns, or short noun phrases. However, in the complex landscape of medical literature, cause and effect expressions often extend over several words, stumping existing methods, resulting in incomplete extractions that provide low-quality, non-informative, and at times, conflicting information. To overcome this challenge, I introduce an innovative unsupervised method for extracting cause and effect phrases, PatternCausality tailored explicitly for medical literature. PatternCausality employs a set of cause-effect dependency patterns as templates to identify the key terms within cause and effect phrases. It then utilizes a novel phrase extraction technique to produce comprehensive and meaningful cause and effect expressions from sentences. Experiments conducted on a dataset constructed from PubMed articles reveal that PatternCausality significantly outperforms existing methods, achieving a remarkable order of magnitude improvement in the F-score metric over the best-performing alternatives. I also develop various PatternCausality variants that utilize diverse phrase extraction methods, all of which surpass existing approaches. PatternCausality and its variants exhibit notable performance improvements in extracting cause and effect entities in a domain-neutral benchmark dataset, wherein cause and effect entities are confined to single-word nouns or noun phrases of one to two words.</p><p dir="ltr">Nevertheless, PatternCausality operates within an unsupervised framework and relies heavily on SDPs, motivating me to explore the development of a supervised approach. Although SDPs play a pivotal role in semantic relation extraction, pattern-based methodologies remain unsupervised, and the multitude of potential patterns within a language can be overwhelming. Furthermore, patterns do not consistently capture the broader context of a sentence, leading to the extraction of false-positive semantic pairs. As an illustration, consider the hyponym-hypernym pattern <i>the w of u</i> which can correctly extract semantic pairs for a sentence like <i>the village of Aasu</i> but fails to do so for the phrase <i>the moment of impact</i>. The root cause of this limitation lies in the pattern's inability to capture the nuanced meaning of words and phrases in a sentence and their contextual significance. These observations have spurred my exploration of a third model, DepBERT which constitutes a dependency-aware supervised transformer model. DepBERT's primary contribution lies in introducing the underlying dependency structure of sentences to a language model with the aim of enhancing token classification performance. To achieve this, I must first reframe the task of semantic pair extraction as a token classification problem. The DepBERT model can harness both the tree-like structure of dependency patterns and the masked language architecture of transformers, marking a significant milestone, as most large language models (LLMs) predominantly focus on semantics and word co-occurrence while neglecting the crucial role of dependency architecture.</p><p dir="ltr">In summary, my overarching contributions in this thesis are threefold. First, I validate the significance of the dependency architecture within various components of sentences and publish SDPs that incorporate these dependency relationships. Subsequently, I employ these SDPs in a practical medical domain to extract vital cause-effect pairs from sentences. Finally, my third contribution distinguishes this thesis by integrating dependency relations into a deep learning model, enhancing the understanding of language and the extraction of valuable semantic associations.</p>

[pt] No contexto de comércio eletrônico, descrições de produtos exercem grande influência na experiência de compra. Descrições bem feitas devem idealmente informar um potencial consumidor sobre detalhes relevantes do produto, esclarecendo potenciais dúvidas e facilitando a compra. Gerar boas descrições, entretanto, é uma atividade custosa, que tradicionalmente exige esforço humano. Ao mesmo tempo, existe uma grande quantidade de produtos sendo lançados a cada dia. Nesse contexto, este trabalho apresenta uma nova metodologia para a geração automatizada de descrições de produtos, usando as avaliações deixadas por usuários como fonte de informações. O método proposto é composto por três etapas: (i) a extração de sentenças adequadas para uma descrição a partir das avaliações (ii) a seleção de sentenças dentre as candidatas (iii) a geração da descrição de produto a partir das sentenças selecionadas usando um Large Language Model (LLM) de forma zero-shot. Avaliamos a qualidade das descrições geradas pelo nosso método comparando-as com descrições de produto reais postadas pelos próprios anunciantes. Nessa avaliação, contamos com a colaboração de 30 avaliadores, e verificamos que nossas descrições são preferidas mais vezes do que as descrições originais, sendo consideradas mais informativas, legíveis e relevantes. Além disso, nessa mesma avaliação replicamos um método da literatura recente e executamos um teste estatístico comparando seus resultados com o nosso método, e dessa comparação verificamos que nosso método gera descrições mais informativas e preferidas no geral. / [en] In the context of e-commerce, product descriptions have a great influence on the shopping experience. Well-made descriptions should ideally inform a potential consumer about relevant product details, clarifying potential doubt sand facilitating the purchase. Generating good descriptions, however, is a costly activity, which traditionally requires human effort. At the same time, there are a large number of products being launched every day. In this context, this work presents a new methodology for the automated generation of product descriptions, using reviews left by users as a source of information. The proposed method consists of three steps: (i) the extraction of suitable sentences for a description from the reviews (ii) the selection of sentences among the candidates (iii) the generation of the product description from the selected sentences using a Large Language Model (LLM) in a zero-shot way. We evaluate the quality of descriptions generated by our method by comparing them to real product descriptions posted by sellers themselves. In this evaluation, we had the collaboration of 30 evaluators, and we verified that our descriptions are preferred more often than the original descriptions, being considered more informative, readable and relevant. Furthermore, in this same evaluation we replicated a method from recent literature and performed a statistical test comparing its results with our method, and from this comparison we verified that our method generates more informative and preferred descriptions overall.