In Norstedts Comprehensive English-Swedish Dictionary (2000) it is said that the numbered list of senses under each headword is frequency ordered. Thus, the aim of this study is to see whether this frequency order of senses agrees with the frequencies appearing in the British National Corpus (BNC). Five English, polysemous verbs were studied. For each verb, a simple search in the corpus was carried out, displaying 50 random occurrences. Each collocate was encoded with the most compatible sense from the numbered list of senses in the dictionary. The encoded tokens were compiled and listed in frequency order. This list was compared to the dictionary's list of senses. Only two of the verbs reached agreement between the highest ranked dictionary sense and the most frequent sense in the BNC simple search. None of the verbs' dictionary orders agreed completely with the emerged frequency order of the corpus occurrences, why complementary collocational learning is advocated.

Raw text data online has increased the need for designing artificial systems capable of processing raw data efficiently and at a low cost in the field of natural language processing (NLP). A well-developed morphological analysis is an important cornerstone of NLP, in particular when word look-up is an important stage of processing. Morphological analysis has many advantages, including reducing the number of word forms to be stored computationally, as well as being cost-efficient and time-efficient. NLP is relevant in the field of medicine, especially in automatic text analysis, which is a relatively young field in Swedish medical texts. Much of the stored information is highly unstructured and disorganized. Using raw corpora, this paper aims to contribute to automatic morphological segmentation by experimenting with state-of-art-tools for unsupervised and semi-supervised word segmentation of Swedish words in medical texts. The results show that a reasonable segmentation is more dependent on a high number of word types, rather than a special type of corpora. The results also show that semi-supervised word segmentation in the form of annotated training data greatly increases the performance. / Rå textdata online har ökat behovet för artificiella system som klarar av att processa rå data effektivt och till en låg kostnad inom språkteknologi (NLP). En välutvecklad morfologisk analys är en viktig hörnsten inom NLP, speciellt när ordprocessning är ett viktigt steg. Morfologisk analys har många fördelar, bland annat reducerar den antalet ordformer som ska lagras teknologiskt, samt så är det kostnadseffektivt och tidseffektivt. NLP är av relevans för det medicinska ämnet, speciellt inom textanalys som är ett relativt ungt område inom svenska medicinska texter. Mycket av den lagrade informationen är väldigt ostrukturerat och oorganiserat. Genom att använda råa korpusar ämnar denna uppsats att bidra till automatisk morfologisk segmentering genom att experimentera med de för närvarande bästa verktygen för oövervakad och semi-övervakad ordsegmentering av svenska ord i medicinska texter. Resultaten visar att en acceptabel segmentering beror mer på ett högt antal ordtyper, och inte en speciell sorts korpus. Resultaten visar också att semi-övervakad ordsegmentering, dvs. annoterad träningsdata, ökar prestandan markant.

This thesis focuses on finding an end-to-end unsupervised solution to solve a two-step problem of extracting semantically meaningful topics and trend analysis of these topics from a large temporal text corpus. To achieve this, the focus is on using the latest develop- ments in Natural Language Processing (NLP) related to pre-trained language models like Google’s Bidirectional Encoder Representations for Transformers (BERT) and other BERT based models. These transformer-based pre-trained language models provide word and sentence embeddings based on the context of the words. The results are then compared with traditional machine learning techniques for topic modeling. This is done to evalu- ate if the quality of topic models has improved and how dependent the techniques are on manually defined model hyperparameters and data preprocessing. These topic models provide a good mechanism for summarizing and organizing a large text corpus and give an overview of how the topics evolve with time. In the context of research publications or scientific journals, such analysis of the corpus can give an overview of research/scientific interest areas and how these interests have evolved over the years. The dataset used for this thesis is research articles and papers from a journal, namely ’Journal of Cleaner Productions’. This journal has more than 24000 research articles at the time of working on this project. We started with implementing Latent Dirichlet Allocation (LDA) topic modeling. In the next step, we implemented LDA along with document clus- tering to get topics within these clusters. This gave us an idea of the dataset and also gave us a benchmark. After having some base results, we explored transformer-based contextual word and sentence embeddings to evaluate if this leads to more meaningful, contextual, and semantic topics. For document clustering, we have used K-means clustering. In this thesis, we also discuss methods to optimally visualize the topics and the trend changes of these topics over the years. Finally, we conclude with a method for leveraging contextual embeddings using BERT and Sentence-BERT to solve this problem and achieve semantically meaningful topics. We also discuss the results from traditional machine learning techniques and their limitations.

News value assessment has been done forever in the news media industry and is today often done in real-time without any documentation. Editors take a lot of different qualitative aspects into consideration when deciding what news stories will make it to the first page. This thesis explores how the complex news value assessment process can be translated into a quantitative model, and also how those news values can be predicted in an effective way using machine learning and NLP. Two models for news value were constructed, for which the correlation between modeled and manual news values was measured, and the results show that the more complex model gives a higher correlation. For prediction, different types of features are extracted, Random Forest and SVM are used, and the predictions are evaluated with accuracy, F1-score, RMSE, and MAE. Random Forest shows the best results for all metrics on all datasets, the best result being on the largest dataset, probably due to the smaller datasets having a less even distribution between classes.

The research of open-domain, knowledge-grounded dialogue systems has been advancing rapidly due to the paradigm shift introduced by large language models (LLMs). While the strides have improved the performance of the dialogue systems, the scope is mostly monolingual and English-centric. The lack of multilingual in-task dialogue data further discourages research in this direction. This thesis explores the use of transfer learning techniques to extend the English-centric dialogue systems to multiple languages. In particular, this work focuses on five typologically diverse languages, of which well-performing models could generalize to the languages that are part of the language family as the target languages, hence widening the accessibility of the systems to speakers of various languages. I propose two approaches: Multilingual Retrieval-Augmented Dialogue Model (xRAD) and Multilingual Generative Dialogue Model (xGenD). xRAD is adopted from a pre-trained multilingual question answering (QA) system and comprises a neural retriever and a multilingual generation model. Prior to the response generation, the retriever fetches relevant knowledge and conditions the retrievals to the generator as part of the dialogue context. This approach can incorporate knowledge into conversational agents, thus improving the factual accuracy of a dialogue model. In addition, xRAD has advantages over xGenD because of its modularity, which allows the fusion of QA and dialogue systems so long as appropriate pre-trained models are employed. On the other hand, xGenD takes advantage of an existing English dialogue model and performs a zero-shot cross-lingual transfer by training sequentially on English dialogue and multilingual QA datasets. Both automated and human evaluation were carried out to measure the models' performance against the machine translation baseline. The result showed that xRAD outperformed xGenD significantly and surpassed the baseline in most metrics, particularly in terms of relevance and engagingness. Whilst xRAD performance was promising to some extent, a detailed analysis revealed that the generated responses were not actually grounded in the retrieved paragraphs. Suggestions were offered to mitigate the issue, which hopefully could lead to significant progress of multilingual knowledge-grounded dialogue systems in the future.

Finding the appropriate level of text for students with varying reading abilities is an important and demanding task for teachers. Radar chart visualisations of text complexity could potentially be an aid in that process, but they need to be evaluated to see if they are intuitive and if they say something about the complexity of a text. This study explores how visualisations of text complexity, in the format of radar charts, are interpreted, what measures they should include and what information they should contain in order to be intelligible for teachers who work with people who have language and/or reading diffi- culties. A preliminary study and three focus group sessions were conducted with teachers from special education schools for adults and gymnasium level. Through thematic analysis of the sessions, five themes were generated and it was found that the visualisations are intelligible to some extent, but they need to be adapted to the target group by making sure the measures are relevant, and that the scale, colours, categories and measures are clearly explained. / Det är både en viktig och krävande uppgift för lärare att hitta lämplig textnivå för elever med varierande läsförmågor. Radardiagramsvisualiseringar av textkomplexitet kan poten- tiellt stötta den processen, men de måste utvärderas för att undersöka om de är intuitiva, vilka mått som bör inkluderas samt om de säger något om komplexiteten av en text. Den här studien utforskar hur visualiseringar av textkomplexitet i form av radardiagram tolkas, vilka mått de bör inkludera samt vilken information de bör innehålla i syfte att vara begripliga för lärare som jobbar med elever med språk och/eller lässvårigheter. En förundersökning och tre fokusgruppsessioner utfördes, med lärare från särgymnasium och särvuxskolor. Efter tematisk analys av data från fokusgrupperna genererades fem teman. Reultaten visade att visualiseringarna var begripliga till viss del, men de behöver anpassas till målgruppen genom att se till att måtten är relevanta samt att skalan, färgerna, kategorierna och måtten är tydligt förklarade.

The domain of rhetoric has changed dramatically since its inception as the art of persuasion. It has adapted to encompass many forms of digital media, including, for example, data visualization and coding as a form of literature, but the approach has frequently been that of an outsider looking in. The use of comprehensive computational tools as a part of rhetorical analysis has largely been lacking. In this report, we attempt to address this lack by means of three case studies in natural language processing tasks, all of which can be used as part of a computational approach to rhetoric. At this same moment in time, it is becoming all the more important to transition to renewable energy in order to keep global warming under 1.5 degrees Celsius and ensure that countries meet the conditions of the Paris Agreement. Thus, we make use of speech data on climate policy from the Swedish parliament to ground these three analyses in semantic textual similarity, topic modeling, and political party attribution. We find that speeches are, to a certain extent, consistent within parties, given that a slight majority of most semantically similar speeches come from the same party. We also find that some of the most common topics discussed in these speeches are nuclear energy and the Swedish Green party, purported environmental risks due to renewable energy sources, and the job market. Finally, we find that though pairs of speeches are semantically similar, party rhetoric on the whole is generally not unique enough for speeches to be distinguishable by party. These results then open the door for a broader exploration of computational rhetoric for Swedish political science in the future.

In Automatic Speech Recognition, there is an extensive need for time-aligned data. Manual speech segmentation has been shown to be more laborious than manual transcription, especially when dealing with tens of hours of speech. Forced alignment is a technique for matching a signal with its orthographic transcription with respect to the duration of linguistic units. Most forced aligners, however, are language-dependent and trained on English data, whereas under-resourced languages lack the resources to develop an acoustic model required for an aligner, as well as manually aligned data. An alternative solution to the training of new models can be cross-language forced alignment, in which an aligner trained on one language is used for aligning data in another language.  This thesis aimed to evaluate state-of-the-art forced alignment algorithms available for Swedish and test whether a Swedish model could be applied for aligning Norwegian. Three approaches for forced aligners were employed: (1) one forced aligner based on Dynamic Time Warping and text-to-speech synthesis Aeneas, (2) two forced aligners based on Hidden Markov Models, namely the Munich AUtomatic Segmentation System (WebMAUS) and the Montreal Forced Aligner (MFA) and (3) Connectionist Temporal Classification (CTC) segmentation algorithm with two pre-trained and fine-tuned Wav2Vec2 Swedish models. First, small speech test sets for Norwegian and Swedish, covering different types of spontaneousness in the speech, were created and manually aligned to create gold-standard alignments. Second, the performance of the Swedish dataset was evaluated with respect to the gold standard. Finally, it was tested whether Swedish forced aligners could be applied for aligning Norwegian data. The performance of the aligners was assessed by measuring the difference between the boundaries set in the gold standard from that of the comparison alignment. The accuracy was estimated by calculating the proportion of alignments below a particular threshold proposed in the literature. It was found that the performance of the CTC segmentation algorithm with Wav2Vec2 (VoxRex) was superior to other forced alignment systems. The differences between the alignments of two Wav2Vec2 models suggest that the training data may have a larger influence on the alignments, than the architecture of the algorithm. In lower thresholds, the traditional HMM approach outperformed the deep learning models. Finally, findings from the thesis have demonstrated promising results for cross-language forced alignment using Swedish models to align related languages, such as Norwegian.

In this project, we have investigated how to perform rule-based machine translation of sets of keywords between two languages. The goal was to translate an input set, which contains one or more keywords in a source language, to a corresponding set of keywords, with the same number of elements, in the target language. However, some words in the source language may have several senses and may be translated to several, or no, words in the target language. If ambiguous translations occur, the best translation of the keyword should be chosen with respect to the context. In traditional machine translation, a word's context is determined by a phrase or sentences where the word occurs. In this project, the set of keywords represents the context. By investigating traditional approaches to machine translation (MT), we designed and described models for the specific purpose of keyword- translation. We have proposed a solution, based on direct translation for translating keywords between English and Swedish. In the proposed solu- tion, we also introduced a simple graph-based model for solving ambigu- ous translations. / I detta projekt har vi undersökt hur man utför regelbaserad maskinöver- sättning av nyckelord mellan två språk. Målet var att översätta en given mängd med ett eller flera nyckelord på ett källspråk till en motsvarande, lika stor mängd nyckelord på målspråket. Vissa ord i källspråket kan dock ha flera betydelser och kan översättas till flera, eller inga, ord på målsprå- ket. Om tvetydiga översättningar uppstår ska nyckelordets bästa över- sättning väljas med hänsyn till sammanhanget. I traditionell maskinö- versättning bestäms ett ords sammanhang av frasen eller meningen som det befinner sig i. I det här projektet representerar den givna mängden nyckelord sammanhanget. Genom att undersöka traditionella tillvägagångssätt för maskinöversätt- ning har vi designat och beskrivit modeller specifikt för översättning av nyckelord. Vi har presenterat en direkt maskinöversättningslösning av nyckelord mellan engelska och svenska där vi introducerat en enkel graf- baserad modell för tvetydiga översättningar.

The use of Machine Learning (ML) in business has increased significantly over the past years. Creating high quality and robust models requires a lot of data, which is at times infeasible to obtain. As more people are becoming concerned about their data being misused, data privacy is increasingly strengthened. In 2018, the General Data Protection Regulation (GDPR), was announced within the EU. Models that use either sensitive or personal data to train need to obtain that data in accordance with the regulatory rules, such as GDPR. One other data related issue is that enterprises who wish to collaborate on model building face problems when it requires them to share their private corporate data [36, 38]. In this thesis we will investigate how one might overcome the issue of directly accessing private data when training ML models by employing Federated Learning (FL) [38]. The concept of FL is to allow several silos, i.e. separate parties, to train models with the same objective, using their local data and then with the learned model parameters create a central model. The objective of the central model is to obtain the information learned by the separate models, without ever accessing the raw data itself. This is achieved by averaging the separate models’ weights into the central model. FL thus facilitates opportunities to train a model on large amounts of data from several sources, without the need of having access to the data itself. If one can create a model with this methodology, that is not significantly worse than a model trained on the raw data, then positive effects such as strengthened data privacy, cross-enterprise collaboration and more could be attainable. In this work we have used a financial data set consisting of 25242 equity research reports, provided by Skandinaviska Enskilda Banken (SEB). Each report has a recommendation label, either Buy, Sell or Hold, making this a multi-class classification problem. To evaluate the feasibility of FL we fine-tune the pre-trained Transformer model AlbertForSequenceClassification [37] on the classification task. We create one baseline model using the entire data set and an FL model with different experimental settings, for which the data is distributed both uniformly and non-uniformly. The baseline model is used to benchmark the FL model. Our results indicate that the best FL setting only suffers a small reduction in performance. The baseline model achieves an accuracy of 83.5% compared to 82.8% for the best FL model setting. Further, we find that with an increased number of clients, the performance is worsened. We also found that our FL model was not sensitive to non-uniform data distributions. All in all, we show that FL results in slightly worse generalisation compared to the baseline model, while strongly improving on data privacy, as the central model never accesses the clients’ data. / Företags nyttjande av maskininlärning har de senaste åren ökat signifikant och för att kunna skapa högkvalitativa modeller krävs stora mängder data, vilket kan vara svårt att insamla. Parallellt med detta så ökar också den allmänna förståelsen för hur användandet av data missbrukas, vilket har lätt till ett ökat behov av starkare datasäkerhet. 2018 så trädde General Data Protection Regulation (GDPR) i kraft inom EU, vilken bland annat ställer krav på hur företag skall hantera persondata. Företag med maskininlärningsmodeller som på något sätt använder känslig eller personlig data behöver således ha fått tillgång till denna data i enlighet med de rådande lagar och regler som omfattar datahanteringen. Ytterligare ett datarelaterat problem är då företag önskar att skapa gemensamma maskininlärningsmodeller som skulle kräva att de delar deras bolagsdata [36, 38]. Denna uppsats kommer att undersöka hur Federerad Inlärning [38] kan användas för att skapa maskinlärningsmodeller som överkommer dessa datasäkerhetsrelaterade problem. Federerad Inlärning är en metod för att på ett decentraliserat vis träna maskininlärningsmodeller. Detta omfattar att låta flera aktörer träna en modell var. Varje enskild aktör tränar respektive modell på deras isolerade data och delar sedan endast modellens parametrar till en central modell. På detta vis kan varje enskild modell bidra till den gemensamma modellen utan att den gemensamma modellen någonsin haft tillgång till den faktiska datan. Givet att en modell, skapad med Federerad Inlärning kan uppnå liknande resultat som en modell tränad på rådata, så finns många positiva fördelar så som ökad datasäkerhet och ökade samarbeten mellan företag. Under arbetet har ett dataset, bestående av 25242 finansiella rapporter tillgängliggjort av Skandinaviska Ensilda Banken (SEB) använts. Varje enskild rapport innefattar en rekommendation, antingen Köp, Sälj eller Håll, vilket innebär att vi utför muliklass-klassificering. Med datan tränas den förtränade Transformermodellen AlbertForSequence- Classification [37] på att klassificera rapporterna. En Baseline-modell, vilken har tränats på all rådata och flera Federerade modellkonfigurationer skapades, där bland annat varierande fördelningen av data mellan aktörer från att vara jämnt fördelat till vara ojämnt fördelad. Resultaten visar att den bästa Federerade modellkonfigurationen endast presterar något sämre än Baseline-modellen. Baselinemodellen uppnådde en klassificeringssäkerhet på 83.5% medan den bästa Federerade modellen uppnådde 82.8%. Resultaten visar också att den Federerade modellen inte var känslig mot att variera fördelningen av datamängd mellan aktorerna, samt att med ett ökat antal aktörer så minskar klassificeringssäkerheten. Sammanfattningsvis så visar vi att Federerad Inlärning uppnår nästan lika goda resultat som Baseline-modellen, samtidigt så bidrar metoden till avsevärt bättre datasäkerhet då den centrala modellen aldrig har tillgång till rådata.