As our lives become more and more digital, our exposure to certain phenomena increases, one of which is hate speech. Thus, automatic hate speech identification is needed. This thesis explores three strategies for hate speech detection for cross­-domain scenarios: using a model trained on annotated data for a previous domain, a model trained on data from a novel methodology of automatic data derivation (with cross­-domain scenarios in mind), and using ChatGPT as a domain-­agnostic classifier. Results showed that cross-­domain scenarios remain a challenge for hate speech detection, results which are discussed out of both technical and ethical considera­tions. / I takt med att våra liv blir allt mer digitala ökar vår exponering för vissa fenomen, varav ett är näthat. Därför behövs automatisk identifikation av näthat. Denna uppsats utforskar tre strategier för att upptäcka hatretorik för korsdomänscenarion: att använda inferenserna av en modell trä­nad på annoterad data för en tidigare domän, att använda inferenserna av en modell tränad på data från en ny metodologi för automatisk dataderivatisering som föreslås (för denna avhandling), samt att använda ChatGPT som klassifierare. Resultaten visade att korsdomänscenarion fort­farande utgör en utmaning för upptäckt av näthat, resultat som diskuteras utifrån både tekniska och etiska överväganden.

A method for performing robust optimal design that combines the efficiency of experimental designs and the accuracy of nonlinear programming (NLP) has been developed, called Search-and-Zoom. Two case studies from the RF and communications industry, a high-frequency micro-strip band-pass filter (BPF) and a rectangular, directional patch antenna, were used to show that sensitivity optimization could be effectively performed in this industry and to compare the computational efficiency of traditional NLP methods (using fmincon solver in MATLAB R2013a) and they hybrid method Search-and-Zoom. The sensitivity of the BPF's S11 response was reduced from 0.06666 at the (non-robust) nominal optimum to 0.01862 at the sensitivity optimum. Feasibility in the design was improved by reducing the likelihood of violating constraints from 20% to nearly 0%, assuming RSS (i.e., normally-distributed) input tolerances and from 40% to nearly 0%, assuming WC (i.e., uniformly-distributed) input tolerances. The sensitivity of the patch antenna's S11 function was also improved from 0.02068 at the nominal optimum to 0.0116 at the sensitivity optimum. Feasibility at the sensitivity optimum was estimated to be 100%, and thus did not need to be improved. In both cases, the computation effort to reach the sensitivity optima, as well as the sensitivity optima with RSS and WC feasibility robustness, was reduced by more than 80% (average) by using Search-and-Zoom, compared to the NLP solver.

[pt] Este trabalho tem como objetivo identificar uma terminologia e expressões relevantes do domínio de óleo e gás (OeG) e estruturá-la como uma taxonomia, tendo em vista o levantamento de itens para a anotação de entidades dentro do domínio. Para tanto, foi construída uma lista de termos relevantes da área, com base em diversas fontes, e, em seguida, a lista foi estruturada hierarquicamente por meio de regras. O processo de elaboração da taxonomia seguiu aspectos teóricometodológicos utilizados por diversos trabalhos semelhantes dentro da área. O trabalho procura evidenciar que a identificação de uma terminologia de um domínio técnico e a sua estruturação como taxonomia podem servir como a primeira etapa do levantamento de entidades de um domínio. Por conta disso, o trabalho também se propõe a discutir estratégias para identificação de entidade mencionada (EM) e possibilitar um diálogo entre duas áreas: Processamento de Linguagem Natural (PLN) e Linguística. De maneira geral, espera-se que a taxonomia ajudar a suprir, mesmo que de forma modesta, a escassez de recursos linguísticos para as técnicas do Processamento de Linguagem Natural (PLN) e da Extração de Informação (EI), dentro da área de óleo e gás. / [en] This work aims to identify a terminology and relevant expressions of the oil and gas domain and structure it as a taxonomy. To this end, a list of relevant terms in the area was built, based on various sources, and then the list was structured hierarchically by rules. The taxonomy elaboration process followed theoretical and methodological aspects used by several similar works within the area. The work tries to show that the identification of a technical domain terminology and its structuring as a taxonomy can serve as the first stage of the identification of entities in a domain. Because of this, the work also proposes to discuss strategies for identifying named entity and to enable a dialogue between two areas: Natural Language Processing (NLP) and Linguistics. In general, the taxonomy presented is expected to supply, at least in a modest way, the lack of linguistic resources for techniques of Natural Language Processing (NLP) and Information Extraction (EI), within the area of oil and gas.

This thesis aims to evaluate the current state of the art for unconditional text generation and compare established models with novel approaches in the task of generating texts, after being trained on texts written by political parties from the Swedish Riksdag. First, the progression of language modeling from n-gram models and statistical models to neural network models is presented. This is followed by theoretical arguments for the development of adversarial training methods,where a generator neural network tries to fool a discriminator network, trained to distinguish between real and generated sentences. One of the methods in the research frontier diverges from the adversarial idea and instead uses cooperative training, where a mediator network is trained instead of a discriminator. The mediator is then used to estimate a symmetric divergence measure between the true distribution and the generator’s distribution, which is to be minimized in training. A set of experiments evaluates the performance of cooperative training and adversarial training, and finds that they both have advantages and disadvantages. In the experiments, the adversarial training increases the quality of generated texts, while the cooperative training increases the diversity. The findings are in line with the theoretical expectation. / Denna uppsats utvärderar några nyligen föreslagna metoder för obetingad textgenerering, baserade på s.k. “Generative Adversarial Networks” (GANs). Den jämför etablerade modeller med nya metoder för att generera text, efter att ha tränats på texter från de svenska Riksdagspartierna. Utvecklingen av språkmodellering från n-gram-modeller och statistiska modeller till modeller av neurala nätverk presenteras. Detta följs upp av teoretiska argument för utvecklingen av GANs, för vilka ett generatornätverk försöker överlista ett diskriminatornätverk, som tränas skilja mellan riktiga och genererade meningar. En av de senaste metoderna avviker från detta angreppssätt och introducerar istället kooperativ träning, där ett mediatornätverk tränas istället för en diskriminator. Mediatorn används sedan till att uppskatta ett symmetriskt divergensmått mellan den sanna distributionen och generatorns distribution, vilket träningen syftar till att minimera. En serie experiment utvärderar hur GANs och kooperativ träning presterar i förhållande till varandra, och finner att de båda har för- och nackdelar. I experimenten ökar GANs kvaliteten på texterna som genereras, medan kooperativ träning ökar mångfalden. Resultaten motsvarar vad som kan förväntas teoretiskt.

Automatically detecting anomalies in text is of great interest for surveillance entities as vast amounts of data can be analysed to find suspicious activity. In this thesis, three distinct machine learning algorithms are evaluated as a chat message classifier is being implemented for the purpose of market surveillance. Naive Bayes and Support Vector Machine belong to the statistical class of machine learning algorithms being evaluated in this thesis and both require feature selection, a side objective of the thesis is thus to find a suitable feature selection technique to ensure mentioned algorithms achieve high performance. Long Short-Term Memory network is the deep learning algorithm being evaluated in the thesis, rather than depend on feature selection, the deep neural network will be evaluated as it is trained using word embeddings. Each of the algorithms achieved high performance but the findings ofthe thesis suggest Naive Bayes algorithm in conjunction with a feature counting feature selection technique is the most suitable choice for this particular learning problem. / Att automatiskt kunna upptäcka anomalier i text har stora implikationer för företag och myndigheter som övervakar olika sorters kommunikation. I detta examensarbete utvärderas tre olika maskininlärningsalgoritmer för chattmeddelandeklassifikation i ett marknadsövervakningsystem. Naive Bayes och Support Vector Machine tillhör båda den statistiska klassen av maskininlärningsalgoritmer som utvärderas i studien och bådar kräver selektion av vilka särdrag i texten som ska användas i algoritmen. Ett sekundärt mål med studien är således att hitta en passande selektionsteknik för att de statistiska algoritmerna ska prestera så bra som möjligt. Long Short-Term Memory Network är djupinlärningsalgoritmen som utvärderas i studien. Istället för att använda en selektionsteknik kommer djupinlärningsalgoritmen nyttja ordvektorer för att representera text. Resultaten visar att alla utvärderade algoritmer kan nå hög prestanda för ändamålet, i synnerhet Naive Bayes tillsammans med termfrekvensselektion.

This bachelor's thesis in Computer Science explores the efficiency of an incremental re-tokenization algorithm in the context of BPE-trained SentencePiece models used in natural language processing. The thesis begins by underscoring the critical role of tokenization in NLP, particularly highlighting the complexities introduced by modifications in tokenized text. It then presents an incremental re-tokenization algorithm, detailing its development and evaluating its performance against a full text re-tokenization. Experimental results demonstrate that this incremental approach is more time-efficient than full re-tokenization, especially evident in large text datasets. This efficiency is attributed to the algorithm's localized re-tokenization strategy, which limits processing to text areas around modifications. The research concludes by suggesting that incremental re-tokenization could significantly enhance the responsiveness and resource efficiency of text-based applications, such as chatbots and virtual assistants. Future work may focus on predictive models to anticipate the impact of text changes on token stability and optimizing the algorithm for different text contexts.

No description available.

Natural Language Processing (NLP) is a branch in data science that marries artificial intelligence with linguistics. Essentially, it tries to program computers to understand human language, both spoken and written. Over the past decade, researchers have applied novel algorithms to gain a better understanding of human sentiment. While no easy feat, incredible improvements have allowed organizations, politicians, governments, and other institutions to capture the attitudes and opinions of the public. It has been particularly constructive for companies who want to check the pulse of a new product or see what the positive or negative sentiments are for their services. NLP has even become useful in boosting sales performance and improving training. Over the years, there have been countless studies on sales performance, both from a psychological perspective, where characteristics of salespersons are explored, and from a data science/AI (Artificial Intelligence) perspective, where text is analyzed to predict sales forecasting (Pai & Liu, 2018) and coach sales agents using AI trainers (Luo et al., 2021). However, few studies have discussed how NLP models can help characterize sales performance using actual sales transcripts. Thus, there is a need to explore to what extent NLP models can inform B2B businesses of the characteristics embodied within their salesforce. This study aims to fill that literature gap. Through a partnership with a medium-sized tech company based out of California, USA, this study conducted an experiment to try and answer to what extent can we characterize sales performance based on real-life sales communication? And in what ways can conversational data inform the sales team at a California-based mid-sized tech company about how top performers communicate with customers? In total, over 5000 sentences containing over 110 000 words were collected and analyzed using two separate rule-based sentiment analysis techniques: TextBlob developed by Steven Loria (2013) and Valence Aware Dictionary and sEntiment Reasoner (VADER) developed by CJ Hutto and Eric Gilbert (2014). A Naïve Bayes classifier was then adopted to test and train each sentiment output from the two rule-based techniques. While both models obtained high accuracy, above 90%, it was concluded that an oversampled VADER approach yields the highest results. Additionally, VADER also tends to classify positive and negative sentences more correctly than TextBlob, when manually reviewing the output, hence making it a better model for the used dataset.

Everyday we are flooded with news from all around the world and this information can be overwhelming. In our study we analyze the possibilities to implement GPT-3 models in the work of news summarization in swedish and automize this process. In this study we also regard the ethic point of view, meaning if we can trust these GPT-3 models and  give them the responsibility to make news summarizations. We studied three different GPT-3 models: ChatGPT, Megatron and GPT-SW3. We used a quantitative survey method where the participants got to rate the news summarizations made by the GPT-3 models. The participants got to rate the news summarizations based on the criterias language, contents and structure. We then took the mean value of the ratings from the survey to see the results. The results showed that ChatGPT was significantly the best of all the three GPT-models on all three criterias, and Megatron and GPT-SW3 performed significantly worse. This shows that these models still need some development to get to the same levels as ChatGPT. Despite ChatGPT being the best performing GPT-3 model it still had its weak sides. We noticed this through one article that had alot of factors included which meant alot of information for the GPT-3 models to condense. Through this study we could confirm that GPT-3 models who are further in their development, like ChatGPT can be used in the work of news summarization but should be used with cautioun of what articles it gets to summarize. This means that GPT-3 models still require human supervision for articles with too much information to condense.

Machine learning has the potential of significantly improving daily operations in health care institutions but many persistent barriers are to be faced in order to ensure its wider acceptance. Among such obstacles are the accuracy and reliability. For a decision support system to be entrusted by the medical staff in clinical situations, it must perform with an accuracy comparable to or surpassing that of human medics, as well ashaving a universal applicability and not being subject to any bias. In this paper the MIMIC-IV Clinical Database will be utilized in order to: (1) Predict patient mortality and its associated risk factors in intensive care units (ICU) and: (2) Assess the reliability of utilizing the database as a basis for a clinical decision system. The cohort consisted of 523,740 hospitalizations, matched with each respective admitting diagnoses in ICD-9 format. The diagnoses were then converted from code to text-format, with the most frequently occurring factors (words) observed in deceased and surviving patients being analyzed with an Natural language Processing (NLP) algorithm. The results concluded that many of the observed risk factors were self-evident while others required further explanation, and that the performance was highly by selection of hyperparameters. Finally, the MIMIC-IV database can serve as a stable foundation for a clinical decision system but its reliability and universality shall also be taken into consideration. / Maskininlärninstekniker har en stor potential att gynna sjukvården men står inför ett flertal hinder för att fullständigt kunna tillämpas. Framförallt bör modellernas tolkningsbarhet och reproducerbarhet beaktas. För att att ett kliniskt beslutstodssystem skall vara fullständigt anförtrott av sjukvårdspersonal måste det kunna prestera med en jämförbar eller högre träffsäkerhet än sjukvårdspersonal, samt kunna tillämpas i åtskilliga sammanhang utan någon subjektivitet. Syftet med denna studie är att: (1) Förutspå patientdödsfall i intensivvårdsavdelningar och utreda dess riskfaktorer genom journalförd information från databasen MIMIC-IV och: 2) Bedöma databasens tillförlitlighet som underlag för ett kliniskt beslutstödssystem. Kohorten bestod av 523,740 insjuknanden som matchades med de diagnoser som ställdes vid deras sjukhusintag. Eftersom diagnoserna inskrevs i ICD-9-format omvandlades dessa till ord och de mest förekommande faktorerna (orden) för avlidna och överlevande patienter analyserades med en NLP-model (Natural Language Processing). Resultaten konkluderade att många av de förutspådda riskfaktorerna var uppenbara medan andra krävde ytterligare klargöranden. Dessutom kunde val av hyperparametrar stort påverka modellens kvalitet. MIMIC-IV-databasen kan utgöra ett gediget underlag för ett kliniskt beslutsystem men dess tillförlitlighet och relevans bör även tas i beaktande. / Kandidatexjobb i elektroteknik 2022, KTH, Stockholm