The impact and dynamics of Internet-based resources for Arabic-speaking users is increasing in significance, depth and breadth at highest pace than ever, and thus requires updated mechanisms for computational processing of Arabic texts. Arabic is a complex language and as such requires in depth investigation for analysis and improvement of available automatic processing techniques such as root extraction methods or text classification techniques, and for developing text collections that are already labeled, whether with single or multiple labels. This thesis proposes new ideas and methods to improve available automatic processing techniques for Arabic texts. Any automatic processing technique would require data in order to be used and critically reviewed and assessed, and here an attempt to develop a labeled Arabic corpus is also proposed. This thesis is composed of three parts: 1- Arabic corpus development, 2- proposing, improving and implementing root extraction techniques, and 3- proposing and investigating the effect of different pre-processing methods on single-labeled text classification methods for Arabic. This thesis first develops an Arabic corpus that is prepared to be used here for testing root extraction methods as well as single-label text classification techniques. It also enhances a rule-based root extraction method by handling irregular cases (that appear in about 34% of texts). It proposes and implements two expanded algorithms as well as an adjustment for a weight-based method. It also includes the algorithm that handles irregular cases to all and compares the performances of these proposed methods with original ones. This thesis thus develops a root extraction system that handles foreign Arabized words by constructing a list of about 7,000 foreign words. The outcome of the technique with best accuracy results in extracting the correct stem and root for respective words in texts, which is an enhanced rule-based method, is used in the third part of this thesis. This thesis finally proposes and implements a variant term frequency inverse document frequency weighting method, and investigates the effect of using different choices of features in document representation on single-label text classification performance (words, stems or roots as well as including to these choices their respective phrases). This thesis applies forty seven classifiers on all proposed representations and compares their performances. One challenge for researchers in Arabic text processing is that reported root extraction techniques in literature are either not accessible or require a long time to be reproduced while labeled benchmark Arabic text corpus is not fully available online. Also, by now few machine learning techniques were investigated on Arabic where usual preprocessing steps before classification were chosen. Such challenges are addressed in this thesis by developing a new labeled Arabic text corpus for extended applications of computational techniques. Results of investigated issues here show that proposing and implementing an algorithm that handles irregular words in Arabic did improve the performance of all implemented root extraction techniques. The performance of the algorithm that handles such irregular cases is evaluated in terms of accuracy improvement and execution time. Its efficiency is investigated with different document lengths and empirically is found to be linear in time for document lengths less than about 8,000. The rule-based technique is improved the highest among implemented root extraction methods when including the irregular cases handling algorithm. This thesis validates that choosing roots or stems instead of words in documents representations indeed improves single-label classification performance significantly for most used classifiers. However, the effect of extending such representations with their respective phrases on single-label text classification performance shows that it has no significant improvement. Many classifiers were not yet tested for Arabic such as the ripple-down rule classifier. The outcome of comparing the classifiers' performances concludes that the Bayesian network classifier performance is significantly the best in terms of accuracy, training time, and root mean square error values for all proposed and implemented representations. / Petra University, Amman (Jordan)

Manual categorization of a mail inbox can often become time-consuming. Therefore many attempts have been made to use machine learning for this task. One essential Natural Language Processing (NLP) task is text classification, which is a big challenge since an NLP engine is not a native speaker of any human language. An NLP engine often fails at understanding sarcasm and underlying intent. One of the NLP challenges is to represent text. Text embeddings can be learned, or they can be generated from a pre-trained model. Google’s pre-trained model Sentence Bidirectional Encoder Representations from Transformers (SBERT) is state-of-the-art for generating pre-trained vector representation of longer text. In this project, different methods of classifying and clustering emails were studied. The performances of three supervised classification models were compared to each other. A Support Vector Machine (SVM) and a Neural Network (NN) were trained with SBERT embeddings, and the third model, a Recurrent Neural Network (RNN) was trained on raw data. The motivation for this experiment was to see whether SBERT embedding is an excellent choice of text representation when combined with simpler classification models in an email classification task. The results show that the SVM and NN perform higher than RNN in the email classification task. Since most real data is unlabeled, this thesis also evaluated how well unsupervised methods could perform in email clustering taking advantage of the available labels and using SBERT embeddings as text representations. Three unsupervised clustering models are reviewed in this thesis: K-Means (KM), Spectral Clustering (SC), and Hierarchical Agglomerative Clustering (HAC). The results show that the unsupervised models all had a similar performance in terms of precision, recall and F1-score, and the performances were evaluated using the available labeled dataset. In conclusion, this thesis gives evidence that in an email classification task, it is better for supervised models to train with pre-trained SBERT embeddings than to train on raw data. This thesis also showed that the output of the clustering methods compared on par with the output of the selected supervised learning techniques. / Manuell kategorisering av en inkorg kan ofta bli tidskrävande. Därför har många försök gjorts att använda maskininlärning för denna uppgift. En viktig uppgift för Natural Language Processing (NLP) är textklassificering, vilket är en stor utmaning eftersom en språkmotor inte talar något mänskligt språk som modersmål. En språkmotor misslyckas ofta med att förstå sarkasm och underliggande avsikt. En av språkmotorns utmaningar är att representera text. Textinbäddningar kan bli inlärda, eller så kan de genereras av en förutbildad modell. Googles förutbildade modell Sentence Bidirectional Encoder Representations from Transformers (SBERT) är den senaste tekniken för att generera förtränade vektorrepresentation av längre text. I detta projekt studerades olika metoder för att klassificera e-postmeddelanden. Prestandan av tre övervakade klassificeringsmodeller jämfördes med varandra, och av dessa var två utbildade med SBERT-inbäddningar: Support Vector Machine (SVM), Neural Network (NN) och den tredje modellen tränades på rådata: Recurrent Neural Network (RNN). Motivationen till detta experiment var att se om SBERT-inbäddningar tillsammans med enklare klassificeringsmodeller är ett bra val av textrepresentation i en e-post klassificeringsuppgift. Resultaten visar att SVM och NN har högre prestanda än RNN i e-postklassificeringsuppgiften. Eftersom mycket verklig data är omärkt utvärderade denna avhandling också hur väl oövervakade metoder kan utföras i samma e-postklassificeringsuppgift med SBERT-inbäddningar som textrepresentationer. Tre oövervakade klustringsmodeller utvärderas i denna avhandling: K-Means (KM), Spectral Clustering (SC) och Hierarchical Agglomerative Clustering (HAC). Resultaten visar att de oövervakade modeller hade liknande prestanda i precision, recall och F1-score, och prestandan var baserad på de tillgängliga klassannoteringarna. Sammanfattningsvis ger denna avhandling bevis på att i en e-postklassificeringsuppgift är det bättre att övervakade modeller tränar med förtränade SBERT-inbäddningar än att träna på rådata. Denna avhandling visade också att resultatet av klustringsmodellerna hade en jämförbar prestanda med resultatet av de valda övervakade inlärningstekniker.

The proposed tool will provide users with a platform to access a side-by-side comparison of classroom assessment and job posting requirements. Using techniques and methodologies from NLP, machine learning, data analysis, and data mining: the employed algorithm analyzes job postings and classroom assessments, extracts and classifies skill units within, then compares sets of skills from different input volumes. This effectively provides a predicted alignment between academic and career sources, both federal and industrial. The compilation of tool results indicates an overall accuracy score of 82%, and an alignment score of only 75.5% between the input assessments and overall job postings. These results describe that the 50 UNT assessments and 5,000 industry and federal job postings examined, demonstrate a compatibility (alignment) of 75.5%; and, that this measure was calculated using a tool operating at an 82% precision rate.

Nous évaluons des méthodes de traitement automatique du langage naturel (TALN) pour la détection d’idées suicidaires, de la dépression et de l’anxiété à partir de publications sur les médias sociaux. Comme les ensembles de données relatifs à la santé mentale sont rares et généralement de petite taille, les méthodes classiques d’apprentissage automatique ont traditionnellement été utilisées dans ce domaine. Nous évaluons l’effet de l’apprentissage multi-tâche sur la détection d’idées suicidaires en utilisant comme tâches auxiliaires des ensembles de données disponibles publiquement pour la détection de la dépression et de l’anxiété, ainsi que la classification d’émotions et du stress. Nous constatons une hausse de la performance de classification pour les tâches de détection d’idées suicidaires, de la dépression et de l’anxiété lorsqu’elles sont entraînées ensemble en raison de similitudes entre les troubles de santé mentale à l’étude. Nous observons que l’utilisation d’ensembles de données publiquement accessibles pour des tâches connexes peut bénéficier à la détection de problèmes de santé mentale. Nous évaluons enfin la performance des modèles ChatGPT et GPT-4 dans des scénarios d’apprentissage zero-shot et few-shot. GPT-4 surpasse toutes les autres méthodes testées pour la détection d’idées suicidaires. De plus, nous observons que ChatGPT bénéficie davantage de l’apprentissage few-shot, car le modèle fournit un haut taux de réponses non concluantes si aucun exemple n’est présenté. Enfin, une analyse des faux négatifs produits par GPT-4 pour la détection d’idées suicidaires conclut qu’ils sont dus à des erreurs d’étiquetage plutôt qu’à des lacunes du modèle. / In this work we explore natural language processing (NLP) methods to suicide ideation, depression, and anxiety detection in social media posts. Since annotated mental health data is scarce and difficult to come by, classical machine learning methods have traditionally been employed on this type of task due to the small size of the datasets. We evaluate the effect of multi-task learning on suicide ideation detection using publicly-available datasets for depression, anxiety, emotion and stress classification as auxiliary tasks. We find that classification performance of suicide ideation, depression, and anxiety is improved when trained together because of the proximity between the mental disorders. We observe that publicly-available datasets for closely-related tasks can benefit the detection of certain mental health conditions. We then perform classification experiments using ChatGPT and GPT-4 using zero-shot and few-shot learning, and find that GPT-4 obtains the best performance of all methods tested for suicide ideation detection. We further observe that ChatGPT benefits the most from few-shot learning as it struggles to give conclusive answers when no examples are provided. Finally, an analysis of false negative results for suicide ideation output by GPT-4 concludes that they are due to labeling errors rather than mistakes from the model.

Pharmacogivilance consists in carefully monitoring drugs in order to re-evaluate their risk for people’s health. The sooner the Adverse Drug Reactions are detected, the sooner one can act consequently. This thesis aims at discovering such reactions in electronical health records under the constraint of lacking annotated data, in order to replicate the scenario of the Regional Center for Pharmacovigilance of Nice. We investigate how in a semi-supervised learning design the unlabeled data can contribute to improve classification scores. Results suggest an excellent recall in discovering adverse reactions and possible classification improvements under specific data distribution. / Läkemedelsövervakningen består i kolla försiktigt läkemedlen så att utvärdera dem för samhällets hälsa. Ju tidigare de läkemedelsrelaterade biverkningarna upptäcks, desto tidigare man får handla dem. Detta exjobb söker att upptäcka de där läkemedelsrelaterade biverkningarnna inom elektroniska hälsopost med få datamärkningar, för att återskapa Nice regionalt läkemedelelsöveraknings-centrumets situationen. Vi undersöker hur en halvväglett lärande lösning kan hjälpa att förbättra klassificeringsresultat. Resultaten visar en god återställning med biverknings-upptäckning och möjliga förbättringar.

The rise of social media and the use of mobile applications has led to increasing concerns regarding the content that is shared through these apps and whether they are being regulated or not. One of the problems that can arise due to a lack of regulation is that chat messages that are inappropriate or of profane nature can be allowed to be shared through these apps. Thus, it is vital to detect whenever these types of chat messages are shared through these mobile applications. In addition to that, there should also be detection of chat messages that can lead to the identity of the users being revealed as that is how the app in this thesis project was intended to be used. One of the most popular approaches to detect chat messages of this nature is to use machine learning techniques that can classify text. We were quick to discover that there were not many machine learning models that were built to classify short text messages in the Swedish language, thus the main problem of our thesis was the lack of evaluation and analysis of machine learning models for text classification in the context of the chat messages in Swedish. Thus, the purpose of our project was mainly to find the best performing models for text classification, implement these models and evaluate them to find the best among the ones we found. After the models were created, a hosting server, as well as an API, was required for the text classifying system to compute and communicate the prediction results to the mobile application in real-time. Therefore, the models were containerized and deployed as a REST API that serves requests upon arrival on a cloud server. The goal of this project was to help future work being done on text classification in the Swedish language by providing the results of this thesis to any parties that are interested in our line of work. From our own experience, we realized how challenging it can be to find and choose the best machine learning models when one has no previous data on which can be the best performing one. Thus, we believe that the results of this thesis project will greatly aid future projects in this area. The chosen research methodology was qualitative and dealt with quantitative data. The results we received showed that the BERT model was the best choice among the three models that we compared. With minute adjustments, this model should be more than capable of detecting the type of chat messages that it is required within the mobile application. / Uppkomsten av social media och användning av mobilapplikationer ledde till ökande oro om innehållet som är delad inom dessa appar och om dem är reglerad eller inte. Ett problem som uppstår på grund av bristande reglering kan vara att chatmeddelanden som är olämplig eller profan kan bli delad med dessa appar. Därför är det viktig att upptäcka när dessa typer av chatmeddelande är delad genom mobilapplikationer. Dessutom det måste finnas ett system som upptäcker chattmeddelanden som kan hjälpa att avslöja användarens identiteter, som den här appen i detta projekt avsedda att användas. En av mest populära sett att upptäcka den typen av chattmeddelanden är användning av mäskinlärning tekniker som kan klassificera text. Vi snart hittade att det finns inte så många mäskinlärning modeller som var byggt att klassificera texter på svenska, alltså huvudproblem med vår exam en var bistrande utvärdering och analys av mäskinlärning modeller för textklassificering i kontext av svenska språket. Så, syftet med vårt projekt var att hitta de bästa presenterande modeller för textklassifikation, genomföra dessa modeller själva och sedan utvärdera dem att hitta den bästa. Därtill, för att textklassificering ska beräkna och kommunicera den förutsägelseresultaten till mobila applikationer i realtid behövs en värdserver samt en API. Därför, modellerna containeriserades och distribuerad es som en REST API som betjänar begäran vid ankomst på en molnserver. Målet med det här projektet var att hjälpa framtidsarbete inom textklassifikation på svenska språket genom att tillhandahålla resultaten till partier som är intresserad i vår arbetslin je. Från vår egen erfarenhet, vi insåg att det var svårt att hitta och välja dem bästa mäskinlärning modeller, specifikt när man har inga data som tidigare visat den med bäst prestanda. Och därför vi anser att den resultaten av den har examen kommer att v ara stor hjälp till framtida projekt i det här området. Den valda forskningsmetodiken var kvalitativ och handlade om kvantitativ data. Resultaten visade att BERT modell var den bästa bland de tre modellerna som vi jämförde med. Med lite justeringen är mod ellen mer än kapable att detektera den typen av krävs inom mobilapplikationen.

The emergence of large language models, such as BERT and GPT-3, has revolutionized natural language processing tasks. However, the development and deployment of these models pose challenges, including concerns about computational resources and environmental impact. This study aims to compare discriminative language models for text classification based on their performance and usage cost. We evaluate the models using a hierarchical multi-label text classification task and assess their performance using primarly F1-score. Additionally, we analyze the usage cost by calculating the Floating Point Operations (FLOPs) required for inference. We compare a baseline model, which consists of a classifier chain with logistic regression models, with fine-tuned discriminative language models, including BERT with two different sequence lengths and DistilBERT, a distilled version of BERT. Results show that the DistilBERT model performs optimally in terms of performance, achieving an F1-score of 0.56 averaged on all classification layers. The baseline model and BERT with a maximal sequence length of 128 achieve F1-scores of 0.51. However, the baseline model outperforms the transformers at the most specific classification level with an F1-score of 0.33. Regarding usage cost, the baseline model significantly requires fewer FLOPs compared to the transformers. Furthermore, restricting BERT to a maximum sequence length of 128 tokens instead of 512 sacrifices some performance but offers substantial gains in usage cost. The code and dataset are available on GitHub. / Uppkomsten av stora språkmodeller, som BERT och GPT-3, har revolutionerat språkteknologi. Dock ger utvecklingen och implementeringen av dessa modeller upphov till utmaningar, bland annat gällande beräkningsresurser och miljöpåverkan. Denna studie syftar till att jämföra diskriminativa språkmodeller för textklassificering baserat på deras prestanda och användningskostnad. Vi utvärderar modellerna genom att använda en hierarkisk textklassificeringsuppgift och bedöma deras prestanda primärt genom F1-score. Dessutom analyserar vi användningskostnaden genom att beräkna antalet flyttalsoperationer (FLOPs) som krävs för inferens. Vi jämför en grundläggande modell, som består av en klassifikationskedja med logistisk regression, med finjusterande diskriminativa språkmodeller, inklusive BERT med två olika sekvenslängder och DistilBERT, en destillerad version av BERT. Resultaten visar att DistilBERT-modellen presterar optimalt i fråga om prestanda och uppnår en genomsnittlig F1-score på 0,56 för alla klassificeringsnivåer. Den grundläggande modellen och BERT med en maximal sekvenslängd på 128 uppnår ett F1-score på 0,51. Dock överträffar den grundläggande modellen transformermodellerna på den mest specifika klassificeringsnivån med en F1-score på 0,33. När det gäller användningskostnaden kräver den grundläggande modellen betydligt färre FLOPs jämfört med transformermodellerna. Att begränsa BERT till en maximal sekvenslängd av 128 tokens ger vissa prestandaförluster men erbjuder betydande besparingar i användningskostnaden. Koden och datamängden är tillgängliga på GitHub.

Le danmaku désigne les commentaires synchronisés qui s’affichent et défilent directement en surimpression sur des vidéos au fil du visionnement. Bien que les danmakus proposent à l’audience une manière originale de partager leur sentiments, connaissances, compréhensions et prédictions sur l’histoire d’une série, etc., et d’interagir entre eux, la façon dont les commentaires s’affichent peut nuire à l’expérience de visionnement, lorsqu’une densité excessive de commentaires dissimule complètement les images de la vidéo ou distrait l’audience. Actuellement, les sites de vidéo chinois emploient principalement des méthodes par mots-clés s’appuyant sur des expressions régulières pour éliminer les commentaires non désirés. Ces approches risquent fortement de surgénéraliser en supprimant involontairement des commentaires intéressants contenant certains mots-clés ou, au contraire, de sous-généraliser en étant incapables de détecter ces mots lorsqu’ils sont camouflés sous forme d’homophones. Par ailleurs, les recherches existantes sur la classification automatique du danmaku se consacrent principalement à la reconnaissance de la polarité des sentiments exprimés dans les commentaires. Ainsi, nous avons cherché à regrouper les commentaires par classes fonctionnelles, à évaluer la robustesse d’une telle classification et la possibilité de l’automatiser dans la perspective de développer de meilleurs systèmes de filtrage des commentaires. Nous avons proposé une nouvelle taxonomie pour catégoriser les commentaires en nous appuyant sur la théorie des actes de parole et la théorie des gratifications dans l’usage des médias, que nous avons utilisées pour produire un corpus annoté. Un fragment de ce corpus a été co-annoté pour estimer un accord inter-annotateur sur la classification manuelle. Enfin, nous avons réalisé plusieurs expériences de classification automatique. Celles-ci comportent trois étapes : 1) des expériences de classification binaire où l’on examine si la machine est capable de faire la distinction entre la classe majoritaire et les classes minoritaires, 2) des expériences de classification multiclasses à granularité grosse cherchant à classifier les commentaires selon les catégories principales de notre taxonomie, et 3) des expériences de classification à granularité fine sur certaines sous-catégories. Nous avons expérimenté avec des méthodes d’apprentissage automatique supervisé et semi-supervisé avec différents traits. / Danmaku denotes synchronized comments which are displayed and scroll directly on top of videos as they unfold. Although danmaku offers an innovative way to share their sentiments, knowledge, predictions on the plot of a series, etc., as well as to interact with each other, the way comments display can have a negative impact on the watching experience, when the number of comments displayed in a given timespan is so high that they completely hide the pictures, or distract audience. Currently, Chinese video websites mainly ressort to keyword approaches based on regular expressions to filter undesired comments. These approaches are at high risk to overgeneralize, thus deleting interesting comments coincidentally containing some keywords, or, to the contrary, undergeneralize due to their incapacity to detect occurrences of these keywords disguised as homophones. On another note, existing research focus essentially on recognizing the polarity of sentiments expressed within comments. Hence, we have sought to regroup comments into functional classes, evaluate the robustness of such a classification and the feasibility of its automation, under an objective of developping better comments filtering systems. Building on the theory of speech acts and the theory of gratification in media usage, we have proposed a new taxonomy of danmaku comments, and applied it to produce an annotated corpus. A fragment of the corpus has been co-annotated to estimate an interannotator agreement for human classification. Finally, we performed several automatic classification experiments. These involved three steps: 1) binary classification experiments evaluating whether the machine can distinguish the most frequent class from all others, 2) coarse-grained multi-class classification experiments aiming at classifying comments within the main categories of our taxonomy, and 3) fine-grained multi-class classification experiments on specific subcategories. We experimented both with supervised and semi-supervised learning algorithms with diffrent features.

[pt] [Contexto] A importância das revisões sistemáticas da literatura (RSLs) para encontrar e sintetizar novas evidências para Engenharia de Software (ES) é bem conhecida, mas realizar e manter as RSLs atualizadas ainda é um grande desafio. Uma das atividades mais exaustivas durante uma RSL é a seleção de estudos, devido ao grande número de estudos a serem analisados. Além disso, para evitar viés, a seleção de estudos deve ser conduzida por mais de um revisor. [Objetivo] Esta dissertação tem como objetivo avaliar o uso de modelos de classificação de texto de machine learning (ML) para apoiar a seleção de estudos em atualizações de RSL e verificar se tais modelos podem substituir um revisor adicional. [Método] Reproduzimos a seleção de estudos de uma atualização de RSL realizada por três pesquisadores experientes, aplicando os modelos de ML ao mesmo conjunto de dados que eles utilizaram. Utilizamos dois algoritmos de ML supervisionado com configurações diferentes (Random Forest e Support Vector Machines) para treinar os modelos com base na RSL original. Calculamos a eficácia da seleção de estudos dos modelos de ML em termos de precisão, recall e f-measure. Também comparamos o nível de semelhança e concordância entre os estudos selecionados pelos modelos de ML e os revisores originais, realizando uma análise de Kappa e da Distância Euclidiana. [Resultados] Em nossa investigação, os modelos de ML alcançaram um f-score de 0.33 para a seleção de estudos, o que é insuficiente para conduzir a tarefa de forma automatizada. No entanto, descobrimos que tais modelos poderiam reduzir o esforço de seleção de estudos em 33.9 por cento sem perda de evidências (mantendo um recall de 100 por cento), descartando estudos com baixa probabilidade de inclusão. Além disso, os modelos de ML alcançaram em média um nível de concordância moderado com os revisores, com um valor médio de 0.42 para o coeficiente de Kappa. [Conclusões] Os resultados indicam que o ML não está pronto para substituir a seleção de estudos por revisores humanos e também pode não ser usado para substituir a necessidade de um revisor adicional. No entanto, há potencial para reduzir o esforço de seleção de estudos das atualizações de RSL. / [en] [Context] The importance of systematic literature reviews (SLRs) to find and synthesize new evidence for Software Engineering (SE) is well known, yet performing and keeping SLRs up-to-date is still a big challenge. One of the most exhaustive activities during an SLR is the study selection because of the large number of studies to be analyzed. Furthermore, to avoid bias, study selection should be conducted by more than one reviewer. [Objective] This dissertation aims to evaluate the use of machine learning (ML) text classification models to support the study selection in SLR updates and verify if such models can replace an additional reviewer. [Method] We reproduce the study selection of an SLR update performed by three experienced researchers, applying the ML models to the same dataset they used. We used two supervised ML algorithms with different configurations (Random Forest and Support Vector Machines) to train the models based on the original SLR. We calculated the study selection effectiveness of the ML models in terms of precision, recall, and f-measure. We also compared the level of similarity and agreement between the studies selected by the ML models and the original reviewers by performing a Kappa Analysis and Euclidean Distance Analysis. [Results] In our investigation, the ML models achieved an f-score of 0.33 for study selection, which is insufficient for conducting the task in an automated way. However, we found that such models could reduce the study selection effort by 33.9 percent without loss of evidence (keeping a 100 percent recall), discarding studies with a low probability of being included. In addition, the ML models achieved a moderate average kappa level of agreement of 0.42 with the reviewers. [Conclusion] The results indicate that ML is not ready to replace study selection by human reviewers and may also not be used to replace the need for an additional reviewer. However, there is potential for reducing the study selection effort of SLR updates.

This study aims to construct a state-of-the-art classifier model and compare it against a largelanguage model. A variation of SVM called LinearSVC was utilised and the BERT model usingbert-base-uncased was used. The data, provided by the Swedish Research Council, consisted ofresearch grant applications. The research grant applications were divided into two groups, whichwere further divided into several subgroups. The subgroups represented research fields such ascomputer science and applied physics. Significant class imbalances were present, with someclasses having only a tenth of the applications of the largest class. To address these imbalances,a new dataset was created using data that had been randomly oversampled. The models weretrained and tested on their ability to correctly assign a subgroup to a research grant application.Results indicate that the BERT model outperformed the SVM model on the original dataset,but not on the balanced dataset . Furthermore, the BERT model’s performance decreased whentransitioning from the original to the balanced dataset, due to overfitting or randomness. / Denna studie har som mål att bygga en state-of-the-art klassificerar model och sedan jämföraden mot en stor språkmodel. SVM modellen var en variation av SVM vid namn LinearSVC ochför BERT användes bert-base-uncased. Data erhölls från Vetenskapsrådet och bestod av forskn-ingsbidragsansökningar. Forskningsbidragsansökningarna var uppdelade i två grupper, som varytterligare uppdelade i ett flertal undergrupper. Dessa undergrupper representerar forsknings-fält såsom datavetenskap och tillämpad fysik. I den data som användes i studien fanns storaskillnader mellan klasserna, där somliga klasser hade en tiondel av ansökningarna som de storaklasserna hade. I syfte att lösa dessa klassbalanseringsproblem skapades en datamängd somundergått slumpmässig översampling. Modellerna tränades och testades på deras förmåga attkorrekt klassificera en forskningsbidragsansökan in i rätt undergrupp. Studiens fynd visade attBERT modellen presterade bättre än SVM modellen på både den ursprungliga datamängden,dock inte på den balanserade datamängden. Tilläggas kan, BERTs prestanda sjönk vid övergångfrån den ursprungliga datamängden till den balanserade datamängden, något som antingen berorpå överanpassning eller slump.