Social media is a significant factor in information distribution today, and this information landscape contains a lot of different posts that compete for the user’s attention. Different factors can help catch the interest of the user, and one of them is the headline of the message. The headline can be more or less eye-catching, which can make the reader more or less interested in interacting with the post. The theme of this study is the automatized creation of eye-catching headlines that stay truthful to the content of the articles using Automatic Text Summarization. The exact method used consisted of fine-tuning the BART model, which is an existing model for Text Summarization. Other papers have been written using different models to solve this problem with more or less success, however, none have used this method. It was deemed an interesting method as it is less time- and energy-consuming than creating and training a new model entirely from scratch and therefore could be easily replicated if the results were positive. The BartForConditionalGeneration model implemented by the HuggingFace library was fine-tuned, using the Popular News Articles by Web.io. This method showed positive results. The resulting headlines were deemed faithful to the original ones, with a ROUGE-2 recall score of 0.541. They were comparably eye-catching to the human-written headlines, with the human respondents ranking them almost the same, with an average rank of 1.692 for the human-written headlines, and 1.821 for fine-tuned BART, and also getting an average score of 3.31 on a 1 to 5 attractiveness score scale. They were also deemed very comprehensible, with an average score of 0.95 on a scale from 0 to 1. / Sociala medier är idag en viktig faktor i distributionen av information. Detta nya landskap innehåller många olika inlägg som tävlar om användarens uppmärksamhet. Olika faktorer kan hjälpa till att fånga användarens blick till specifika inlägg eller artiklar, och en av dessa faktorer är rubriken. Rubriken kan vara mer eller mindre fängslande, och göra läsaren mer eller mindre intresserad av att interagera med inlägget. Temat för denna studie är att automatiskt skapa iögonfallande och intressanta rubriker, som beskriver innehå llet i artiklarna på ett korrekt sätt. Den valda metoden är automatisk textsamman fattning, och mer specifikt finjusterades BART-modellen, som är en existerande modell för textsammanfattning. Andra metoder har använts tidigare för att lösa denna problematik med mer eller mindre framgång, men ingen studie hade använt den här. Den ansågs vara intressant eftersom den är mindre tids- och energikrävande än vad det skulle vara att skapa en ny modell från grunden, och därför skulle den lätt kunna replikeras om resultatet var positivt. BartForConditionalGeneration-modellen implementerad av HuggingFace-bib lioteket finjusterades därför med hjälp av artiklar och rubriker från datasetet ’Popular News Articles’ av Web.io. Metoden visade positiva resultat. De resulterande rubrikerna ansågs trogna de ursprungliga, med en ROUGE-2 recall score på 0,541. De var jämförbart iögonfallande gentemot de mänskligt skrivna rubrikerna, då respondenterna rankade dem nästan likadant, med en genomsnittlig rankning på 1,692 för de mänskligt skrivna rubrikerna och 1,821 för rubrikerna som finjusterade BART genererade. De fick också ett genomsnittligt betyg av 3,31 på en poängskala från 1 till 5. De ansågs dessutom vara mycket lättbegripliga, med ett medelpoäng på 0,95 på en skala från 0 till 1.

A key part of contract drafting involves thinking of issues that have not been addressedand adding language that will address the missing issues. To assist attorneys with this task, we present a pipeline approach for identifying missing information within a contract section. The pipeline takes a contract section as input and includes 1) identifying sections that are similar to the input section from a corpus of contract sections; and 2) identifying and suggesting information from the similar sections that are missing from the input section. By taking advantage of sentence embedding and principal component analysis, this approach suggests sentences that are helpful for finishing a contract. We show that sentence suggestions are more useful than the state of the art topic suggestion algorithm by synthetic experiments and a user study.

The goal of this thesis is to improve the feasibility of building applied NLP systems for more diverse and niche real-world use-cases of extracting structured information from text. A core factor in determining this feasibility is the cost of manually annotating enough unbiased labeled data to achieve a desired level of system accuracy, and our goal is to reduce this cost. We focus on reducing this cost by making contributions in two directions: (1) easing the annotation burden by leveraging high-level expert knowledge in addition to labeled examples, thus making approaches more annotation-efficient; and (2) mitigating known biases in cheaper, imperfectly labeled real-world datasets so that we may use them to our advantage. A central theme of this thesis is that high-level expert knowledge about the data and task can allow for biased labeling processes that focus experts on only manually labeling aspects of the data that cannot be easily labeled through cheaper means. This combination allows for more accurate models with less human effort. We conduct our research on this general topic through three diverse problems with immediate applications to real-world settings. First, we study an applied problem in biased text classification. We encounter a rare-event text classification system that has been deployed for several years. We are tasked with improving this system's performance using only the severely biased incidental feedback provided by the experts over years of system use. We develop a method that combines importance weighting and an unlabeled data imputation scheme that exploits the selection-bias of the feedback to train an unbiased classifier without requiring additional labeled data. We experimentally demonstrate that this method considerably improves the system performance. Second, we tackle an applied problem in named entity recognition (NER) concerning learning tagging models from data that have very low recall for annotated entities. To solve this issue we propose a novel loss, the Expected Entity Ratio (EER), that uses an uncertain estimate of the proportion of entities in the data to counteract the false-negative bias in the data, encouraging the model to have the correct ratio of entities in expectation. We justify the principles of our approach by providing theory that shows it recovers the true tagging distribution under mild conditions. Additionally we provide extensive empirical results that show it to be practically useful. Empirically, we find that it meets or exceeds performance of state-of-the-art baselines across a variety of languages, annotation scenarios, and amounts of labeled data. We also show that, when combined with our approach, a novel sparse annotation scheme can outperform exhaustive annotation for modest annotation budgets. Third, we study the challenging problem of syntactic parsing in low-resource languages. We approach the problem from a cross-lingual perspective, building on a state-of-the-art transfer-learning approach that underperforms on ``distant'' languages that have little to no representation in the training corpus. Motivated by the field of syntactic typology, we introduce a general method called Expected Statistic Regularization (ESR) to regularize the parser on distant languages according to their expected typological syntax statistics. We also contribute general approaches for estimating the loss supervision parameters from the task formalism or small amounts of labeled data. We present seven broad classes of descriptive statistic families and provide extensive experimental evidence showing that using these statistics for regularization is complementary to deep learning approaches in low-resource transfer settings. In conclusion, this thesis contributes approaches for reducing the annotation cost of building applied NLP systems through the use of high-level expert knowledge to impart additional learning signal on models and cope with cheaper biased data. We publish implementations of our methods and results, so that they may facilitate future research and applications. It is our hope that the frameworks proposed in this thesis will help to democratize access to NLP for producing structured information from text in wider-reaching applications by making them faster and cheaper to build.

Hateful content and offensive language are commonplace on social media platforms. Many surveys prove that high percentages of social media users experience online harassment. Previous efforts have been made to detect and remove online hate content automatically. However, removing users' content restricts free speech. A complementary strategy to address hateful content that does not interfere with free speech is to counter the hate with new content to divert the discourse away from the hate. In this dissertation, we complement the lack of previous work on counterhate arguments by analyzing and detecting them. Firstly, we study the relationships between hateful tweets and replies. Specifically, we analyze their fine-grained relationships by indicating whether the reply counters the hate, provides a justification, attacks the author of the tweet, or adds additional hate. The most obvious finding is that most replies generally agree with the hateful tweets; only 20% of them counter the hate. Secondly, we focus on the hate directed toward individuals and detect authentic counterhate arguments from online articles. We propose a methodology that assures the authenticity of the argument and its specificity to the individual of interest. We show that finding arguments in online articles is an efficient alternative compared to counterhate generation approaches that may hallucinate unsupported arguments. Thirdly, we investigate the replies to counterhate tweets beyond whether the reply agrees or disagrees with the counterhate tweet. We analyze the language of the counterhate tweet that leads to certain types of replies and predict which counterhate tweets may elicit more hate instead of stopping it. We find that counterhate tweets with profanity content elicit replies that agree with the counterhate tweet. This dissertation presents several corpora, detailed corpus analyses, and deep learning-based approaches for the three tasks mentioned above.

Social media is considered a particularly conducive arena for hate speech. Counter speech, which is a "direct response that counters hate speech" is a remedy to address hate speech. Unlike content moderation, counter speech does not interfere with the principle of free and open public spaces for debate. This dissertation focuses on the (a) automatic detection and (b) analyses of the effectiveness of counter speech and its fine-grained strategies in user-generated web content. The first goal is to identify counter speech. We create a corpus with 6,846 instances through crowdsourcing. We specifically investigate the role of conversational context in the annotation and detection of counter speech. The second goal is to assess and predict conversational outcomes of counter speech. We propose a new metric to measure conversation incivility based on the number of uncivil and civil comments as well as the unique authors involved in the discourse. We then use the metric to evaluate the outcomes of replies to hate speech. The third goal is to establish a fine-grained taxonomy of counter speech. We present a theoretically grounded taxonomy that differentiates counter speech addressing the author of hate speech from addressing the content. We further compare the conversational outcomes of different types of counter speech and build models to identify each type. We conclude by discussing our contributions and future research directions on using user-generated counter speech to combat online hatred.

京都大学 / 新制・課程博士 / 博士(情報学) / 甲第24929号 / 情博第840号 / 新制||情||140(附属図書館) / 京都大学大学院情報学研究科知能情報学専攻 / (主査)特定教授 黒橋 禎夫, 教授 河原 達也, 教授 西野 恒 / 学位規則第4条第1項該当 / Doctor of Informatics / Kyoto University / DFAM

<p dir="ltr">The research focuses on improving natural language processing (NLP) models by integrating the hierarchical structure of language, which is essential for understanding and generating human language. The main contributions of the study are:</p><ol><li><b>Hierarchical RNN Model:</b> Development of a deep Recurrent Neural Network model that captures both explicit and implicit hierarchical structures in language.</li><li><b>Hierarchical Attention Mechanism:</b> Use of a multi-level attention mechanism to help the model prioritize relevant information at different levels of the hierarchy.</li><li><b>Latent Indicators and Efficient Training:</b> Integration of latent indicators using the Expectation-Maximization algorithm and reduction of computational complexity with Bootstrap sampling and layered training strategies.</li><li><b>Sequence-to-Sequence Model for Translation:</b> Extension of the model to translation tasks, including a novel pre-training technique and a hierarchical decoding strategy to stabilize latent indicators during generation.</li></ol><p dir="ltr">The study claims enhanced performance in various NLP tasks with results comparable to larger models, with the added benefit of increased interpretability.</p>

Machine Learning models have become more advanced than could have been supposed even a few years ago, often surpassing human performance on many tasks. Large language models (LLM) can produce text indistinguishable from human-produced text. This begs the question, how necessary are humans - even for tasks where humans appear indispensable? Qualitative Analysis (QA) is integral to human-computer interaction research, requiring both human-produced data and human analysis of that data to illuminate human opinions about and experiences with technology. We use GPT-3 and ChatGPT to replace human analysis and then to dispense with human-produced text altogether. We find GPT-3 is capable of automatically identifying themes and generating nuanced analyses of qualitative data arguably similar to those written by human researchers. We also briefly ponder philosophical implications of this research.

Customer calls serve as a valuable source of feedback for financial service providers, potentially containing a wealth of unexplored insights into customer questions and concerns. However, these call data are typically unstructured and challenging to analyze effectively. This thesis project focuses on leveraging Topic Modeling techniques, a sub-field of Natural Language Processing, to extract meaningful customer insights from recorded customer calls to a European financial service provider. The objective of the study is to compare two widely used Topic Modeling algorithms, Latent Dirichlet Allocation (LDA) and BERTopic, in order to categorize and analyze the content of the calls. By leveraging the power of these algorithms, the thesis aims to provide the company with a comprehensive understanding of customer needs, preferences, and concerns, ultimately facilitating more effective decision-making processes.  Through a literature review and dataset analysis, i.e., pre-processing to ensure data quality and consistency, the two algorithms, LDA and BERTopic, are applied to extract latent topics. The performance is then evaluated using quantitative and qualitative measures, i.e., perplexity and coherence scores as well as in- terpretability and usefulness of topic quality. The findings contribute to knowledge on Topic Modeling for customer insights and enable the company to improve customer engagement, satisfaction and tailor their customer strategies.  The results show that LDA outperforms BERTopic in terms of topic quality and business value. Although BERTopic demonstrates a slightly better quantitative performance, LDA aligns much better with human interpretation, indicating a stronger ability to capture meaningful and coherent topics within company’s customer call data.

Indiana University-Purdue University Indianapolis (IUPUI) / The dramatic improvement in information and communication technology (ICT) has made an evolution in learning management systems (LMS). The rapid growth in LMSs has caused users to demand more advanced, automated, and intelligent services. CourseNet working is a next-generation LMS adopting machine learning to add personalization, gamifi cation, and more dynamics to the system. This work tries to come up with two recommender systems that can help improve CourseNetworking services. The first one is a social recommender system helping CourseNetworking to track user interests and give more relevant recommendations. Recently, graph neural network (GNN) techniques have been employed in social recommender systems due to their high success in graph representation learning, including social network graphs. Despite the rapid advances in recommender systems performance, dealing with the dynamic property of the social network data is one of the key challenges that is remained to be addressed. In this research, a novel method is presented that provides social recommendations by incorporating the dynamic property of social network data in a heterogeneous graph by supplementing the graph with time span nodes that are used to define users long-term and short-term preferences over time. The second service that is proposed to add to Rumi services is a hashtag recommendation system that can help users label their posts quickly resulting in improved searchability of content. In recent years, several hashtag recommendation methods are proposed and de veloped to speed up processing of the texts and quickly find out the critical phrases. The methods use different approaches and techniques to obtain critical information from a large amount of data. This work investigates the efficiency of unsupervised keyword extraction methods for hashtag recommendation and recommends the one with the best performance to use in a hashtag recommender system.