Yes / The paper presents a text analytics framework that analyses online reviews to explore how consumer-perceived negativity corresponding to the supply chain propagates over time and how it affects car sales. In particular, the framework integrates aspect-level sentiment analysis using SentiWordNet, time-series decomposition, and bias-corrected least square dummy variable (LSDVc) – a panel data estimator. The framework facilitates the business community by providing a list of consumers’ contemporary interests in the form of frequently discussed product attributes; quantifying consumer-perceived performance of supply chain (SC) partners and comparing the competitors; and a model assessing various firms’ sales performance. The proposed framework demonstrated to the automobile supply chain using a review dataset received from a renowned car-portal in India. Our findings suggest that consumer-voiced negativity is maximum for dealers and minimum for manufacturing and assembly related features. Firm age, GDP, and review volume significantly influence car sales whereas the sentiments corresponding to SC partners do not. The proposed research framework can help the manufacturers in inspecting their SC partners; realising consumer-cited critical car sales influencers; and accurately predicting the sales, which in turn can help them in better production planning, supply chain management, marketing, and consumer relationships.

The degree of impact caused by software vulnerabilities is escalating as software systems become increasingly integrated into the everyday lives of human beings. Different methods, such as static and dynamic analysis, are commonly used to classify software vulnerabilities. However, these methods are often plagued by certain limitations, including high false positive and false negative rates. It is crucial to examine C/C++ software vulnerabilities, as C/C++ is widely implemented in many industries and critical infrastructures, where software vulnerabilities could have catastrophic consequences if exploited by malicious actors. This thesis examines the feasibility of utilizing machine learning-based models for automated C/C++ software vulnerability classification. Additionally, the effect of hyperparameter tuning on the predictive performance of the utilized models is explored. The models investigated were divided into two main groups, namely, traditional machine learning models and transformer-based models. All models were trained, evaluated, and compared using a large and diverse C/C++ dataset. The findings suggest that autoregressive large language models, particularly Llama 2 and Code Llama utilizing a decoder-only transformer architecture, demonstrate significant potential for accurate C/C++ vulnerability classification, achieving F1-scores of 0.912 and 0.905, respectively. The results further indicate that hyperparameter tuning has a limited positive effect on predictive performance. Moreover, specific traditional machine learning models, like the SVM model, outperformed many of the transformer-based models, potentially indicating limitations in training procedures and the architectures of many pre-trained language models. Nevertheless, autoregressive large language models exhibit significant potential for precise C/C++ software vulnerability classification and should remain a focal point for future research.

No description available.

This thesis has investigated whether a transformer-based language model can be improved by training the model on context sequences which are input sequences with a larger window of text, by combining a transformer model with a neural network for non-text features, or by domain-adaptive pre-training. Two types of context input sequences are tested: left context and full context. The three modifications are explored by applying BERT models to the Swedish Parliamentary Corpus to classify whether a text sequence is a heading. A standard BERT model is trained for sequence classification alongside a position model which adds an additional feedforward neural network to the model. Each model is trained with- and without context sequences as well as with- and without domain-adaptive pre-training. A standard implementation of the BERT model with domain adaptation achieves an F1 score of 0.9358 on the test set and an accuracy of 0.9940. The best performing standard BERT model with a context input sequence achieves an F1 of 0.9636 and an accuracy of 0.9966 while the best performing position model achieves an F1 of 0.9550 and an accuracy of 0.9957. The best performing model which combines context input sequences with the position model achieves an F1 of 0.9908 and an accuracy of 0.9991 on the test set. Analysis of misclassified sequences suggests that the models with context input sequences and positional features are less likely to misclassify sequences which can appear both as a heading and a non-heading in the corpus. However, a McNemar's exact test indicates that only a position model with left context input sequences differs significantly from its standard BERT counterpart in terms of the number of differing misclassifications at a 5% significance level. Furthermore, there is no experimental evidence that domain-adaptive pre-training improves classification performance on this specific sequence classification task.

<p dir="ltr">The quality of crowdsource annotations has always been a challenge due to the variability in annotators backgrounds, task complexity, the subjective nature of many labeling tasks, and various other reasons. Hence, it is crucial to evaluate these annotations to ensure their reliability. Traditionally, human experts evaluate the quality of crowdsourced annotations, but this approach has its own challenges. Hence, this paper proposes to leverage large language models like ChatGPT-4 to evaluate one of the existing crowdsourced MAVEN dataset and explore its potential as an alternative solution. However, due to stochastic nature of LLMs, it is important to discern when to trust and question LLM responses. To address this, we introduce a novel approach that applies Rubin's framework for identifying and using linguistic cues within LLM responses as indicators of LLMs certainty levels. Our findings reveal that ChatGPT-4 successfully identified 63% of the incorrect labels, highlighting the potential for improving data label quality through human-AI collaboration on these identified inaccuracies. This study underscores the promising role of LLMs in evaluating crowdsourced data annotations offering a way to enhance accuracy and fairness of crowdsource annotations while saving time and costs.</p><p dir="ltr"><br></p>

This study investigates the capacity of Large Language Models (LLMs), specifically ChatGPT 3.5 and 4, to evaluate citations in scholarly papers. Given the importance of accurate citations in academic writing, the goal was to determine how well these models can assist in verifying citations. A series of experiments were conducted using a dataset of our own creation. This dataset includes the three main citation categories: Direct Quotation, Paraphrasing, and Summarising, along with subcategories such as minimal and long source text.  In the preliminary experiment, ChatGPT 3.5 demonstrated perfect accuracy, while ChatGPT 4 showed a tendency towards false positives. Further experiments with an extended dataset revealed that ChatGPT 4 excels in correctly identifying valid citations, particularly with longer and more complex texts, but is also more prone to wrong predictions. ChatGPT 3.5, on the other hand, provided a more balanced performance across different text lengths, with both models achieving an accuracy rate of 90.7%. The reliability experiments indicated that ChatGPT 4 is more consistent in its responses compared to ChatGPT 3.5, although it also had a higher rate of consistent wrong predictions.  This study highlights the potential of LLMs to assist scholars in citation verification, suggesting a hybrid approach where ChatGPT 4 is used for initial scans and ChatGPT 3.5 for final verification, paving the way for automating this process. Additionally, this study contributes a dataset that can be further expanded and tested on, offering a valuable resource for future research in this domain.

The Thermo-Calc software is a key tool in the research process for many material engineers. However, integrating multiple modules in Thermo-Calc requires the user to write code in a Python-based language, which can be challenging for novice programmers. This project aims to enable the generation of such code from user prompts by using existing generative AI models. In particular, we use a retrieval-augmented generation architecture applied to LLaMA and Mistral models. We use Code LLaMA-Instruct models with 7, 13, and 34 billion parameters, and a Mistral-Instruct model with 7 billion parameters. These models are all based on LLaMA 2. We also use a LLaMA 3-Instruct model with 8 billion parameters. All these models are instruction-tuned, which suggests that they have the capability to interpret natural language and identify appropriate options for a command-line program such as Python. In our testing, the LLaMA 3-Instruct model performed best, achieving 53% on the industry benchmark HumanEval and 49% on our internal adequacy assessment at pass@1, which is the expected probability of getting a correct solution when generating a response. This indicates that the model generates approximately every other answer correct. Due to GPU memory limitations, we had to apply quantisation to process the 13 and 34 billion parameter models. Our results revealed a mismatch between model size and optimal levels of quantisation, indicating that reduced precision adversely affects the performance of these models. Our findings suggest that a properly customised large language model can greatly reduce the coding effort of novice programmers, thereby improving productivity in material research.

This study introduces an effective method to address the generalization challenge in pretrained language models (PLMs), which affects their performance on diverse linguistic data beyond their training scope. Improving PLMs' adaptability to out-of-distribution (OOD) data is essential for their reliability and practical utility in real-world applications. Furthermore, we address the ethical imperative of fairness in PLMs, particularly as they become integral to decision-making in sensitive societal sectors. We introduce gender-tuning, to identify and disrupt gender-related biases in training data. This method perturbs gendered terms, replacing them to break associations with other words. Gender-tuning stands as a practical, ethical intervention against gender bias in PLMs. Finally, we present FairAgent, a novel framework designed to imbue small language models (SLMs) with fairness, drawing on the knowledge of large language models (LLMs) without incurring the latter's computational costs. FairAgent operates by enabling SLMs to consult with LLMs, harnessing their vast knowledge to guide the generation of less biased content. This dynamic system not only detects bias in SLM responses but also generates prompts to correct it, accumulating effective prompts for future use. Over time, SLMs become increasingly adept at producing fair responses, enhancing both computational efficiency and fairness in AI-driven interactions.

Submitted by Milena Rubi (milenarubi@ufscar.br) on 2017-10-09T17:30:32Z No. of bitstreams: 1 CARDOSO_Emerson_2017.pdf: 3299853 bytes, checksum: bda5605a1fb8e64f503215e839d2a9a6 (MD5) / Approved for entry into archive by Milena Rubi (milenarubi@ufscar.br) on 2017-10-09T17:30:45Z (GMT) No. of bitstreams: 1 CARDOSO_Emerson_2017.pdf: 3299853 bytes, checksum: bda5605a1fb8e64f503215e839d2a9a6 (MD5) / Approved for entry into archive by Milena Rubi (milenarubi@ufscar.br) on 2017-10-09T17:32:37Z (GMT) No. of bitstreams: 1 CARDOSO_Emerson_2017.pdf: 3299853 bytes, checksum: bda5605a1fb8e64f503215e839d2a9a6 (MD5) / Made available in DSpace on 2017-10-09T17:32:49Z (GMT). No. of bitstreams: 1 CARDOSO_Emerson_2017.pdf: 3299853 bytes, checksum: bda5605a1fb8e64f503215e839d2a9a6 (MD5) Previous issue date: 2017-08-04 / Não recebi financiamento / Before buying a product or choosing for a trip destination, people often seek other people’s opinions to obtain a vision of the quality of what they want to acquire. Given that, opinions always had great influence on the purchase decision. Following the enhancements of the Internet and a huge increase in the volume of data traffic, social networks were created to help users post and view all kinds of information, and this caused people to also search for opinions on the Web. Sites like TripAdvisor and Yelp make it easier to share online reviews, since they help users to post their opinions from anywhere via smartphones and enable product manufacturers to gain relevant feedback quickly in a centralized way. As a result, most people nowadays trust personal recommendations as much as online reviews. However, competition between service providers and product manufacturers have also increased in social media, leading to the first cases of spam reviews: deceptive opinions published by hired people that try to promote or defame products or businesses. These reviews are carefully written in order to look like authentic ones, making it difficult to be detected by humans or automatic methods. Thus, they are used, in a misleading way, in attempt to control the general opinion, causing financial harm to business owners and users. Several approaches have been proposed for spam review detection and most of them use techniques involving machine learning and natural language processing. However, despite all progress made, there are still relevant questions that remain open, which require a criterious analysis in order to be properly answered. For instance, there is no consensus whether the performance of traditional classification methods can be affected by incremental learning or changes in reviews’ features over time; also, there is no consensus whether there is statistical difference between performances of content-based classification methods. In this scenario, this work offers a comprehensive comparison between traditional machine learning methods applied in spam review detection. This comparison is made in multiple setups, employing different types of learning and data sets. The experiments performed along with statistical analysis of the results corroborate offering appropriate answers to the existing questions. In addition, all results obtained can be used as baseline for future comparisons. / Antes de comprar um produto ou escolher um destino de viagem, muitas pessoas costumam buscar por opiniões alheias para obter uma visão da qualidade daquilo que se deseja adquirir. Assim, as opiniões sempre exerceram grande influência na decisão de compra. Com o avanço da Internet e aumento no volume de informações trafegadas, surgiram redes sociais que possibilitam compartilhar e visualizar informações de todo o tipo, fazendo com que pessoas passassem a buscar também por opiniões na Web. Atualmente, sites especializados, como TripAdvisor e Yelp, oferecem um sistema de compartilhamento de opiniões online (reviews) de maneira fácil, pois possibilitam que usuários publiquem suas opiniões de qualquer lugar através de smartphones, assim como também permitem que fabricantes de produtos e prestadores de serviços obtenham feedbacks relevantes de maneira centralizada e rápida. Em virtude disso, estudos indicam que atualmente a maioria dos usuários confia tanto em recomendações pessoais quanto em reviews online. No entanto, a competição entre prestadores de serviços e fabricantes de produtos também aumentou nas redes sociais, o que levou aos primeiros casos de spam reviews: opiniões enganosas publicadas por pessoas contratadas que tentam promover ou difamar produtos ou serviços. Esses reviews são escritos cuidadosamente para parecerem autênticos, o que dificulta sua detecção por humanos ou por métodos automáticos. Assim, eles são usados para tentar, de maneira enganosa, controlar a opinião geral, podendo causar prejuízos para empresas e usuários. Diversas abordagens para a detecção de spam reviews vêm sendo propostas, sendo que a grande maioria emprega técnicas de aprendizado de máquina e processamento de linguagem natural. No entanto, apesar dos avanços já realizados, ainda há questionamentos relevantes que permanecem em aberto e demandam uma análise criteriosa para serem respondidos. Por exemplo, não há um consenso se o desempenho de métodos tradicionais de classificação pode ser afetado em cenários que demandam aprendizado incremental ou por mudanças nas características dos reviews devido ao fator cronológico, assim como também não há um consenso se existe diferença estatística entre os desempenhos dos métodos baseados no conteúdo das mensagens. Neste cenário, esta dissertação oferece uma análise e comparação compreensiva dos métodos tradicionais de aprendizado de máquina, aplicados na detecção de spam reviews. A comparação é realizada em múltiplos cenários, empregando-se diferentes tipos de aprendizado e bases de dados. Os experimentos realizados, juntamente com análise estatística dos resultados, corroboram a oferecer respostas adequadas para os questionamentos existentes. Além disso, os resultados obtidos podem ser usados como baseline para comparações futuras.

Increasing rates of social harm events and plethora of text data demands the need of employing text mining techniques not only to better understand their causes but also to develop optimal prevention strategies. In this work, we study three social harm issues: crime topic models, transitions into drug addiction and homicide investigation chronologies. Topic modeling for the categorization and analysis of crime report text allows for more nuanced categories of crime compared to official UCR categorizations. This study has important implications in hotspot policing. We investigate the extent to which topic models that improve coherence lead to higher levels of crime concentration. We further explore the transitions into drug addiction using Reddit data. We proposed a prediction model to classify the users’ transition from casual drug discussion forum to recovery drug discussion forum and the likelihood of such transitions. Through this study we offer insights into modern drug culture and provide tools with potential applications in combating opioid crises. Lastly, we present a knowledge graph based framework for homicide investigation chronologies that may aid investigators in analyzing homicide case data and also allow for post hoc analysis of key features that determine whether a homicide is ultimately solved. For this purpose<br>we perform named entity recognition to determine witnesses, detectives and suspects from chronology, use keyword expansion to identify various evidence types and finally link these entities and evidence to construct a homicide investigation knowledge graph. We compare the performance over several choice of methodologies for these sub-tasks and analyze the association between network statistics of knowledge graph and homicide solvability. <br>