Les définitions des mots « sentiment », « opinion » et « émotion » sont toujours très vagues comme l’atteste aussi le dictionnaire qui semble expliquer un mot en utilisant le deux autres. Tout le monde est affecté par les opinions : les entreprises pour vendre les produits, les gens pour les acheter et, plus en général, pour prendre des décisions, les chercheurs en intelligence artificielle pour comprendre la nature de l’être humain. Aujourd’hui on a une quantité d’information disponible jamais vue avant, mais qui résulte peu accessible. Les mégadonnées (en anglais « big data ») ne sont pas organisées, surtout pour certaines langues – dont la difficulté à les exploiter. La recherche française souffre d’une manque de ressources « prêt-à-porter » pour conduire des tests. Cette thèse a l’objectif d’explorer la nature des sentiments et des émotions, dans le cadre du Traitement Automatique du Langage et des Corpus. Les contributions de cette thèse sont plusieurs : création de nouvelles ressources pour l’analyse du sentiment et de l’émotion, emploi et comparaison de plusieurs techniques d’apprentissage automatique, et plus important, l’étude du problème sous différents points de vue : classification des commentaires en ligne en polarité (positive et négative), Aspect-Based Sentiment Analysis des caractéristiques du produit recensé. Enfin, un étude psycholinguistique, supporté par des approches lexicales et d’apprentissage automatique, sur le rapport entre qui juge et l’objet jugé. / "Sentiment", "opinion" and "emotion" are words really vaguely defined; not even the dictionary seems to be of any help, being it the first to define each of the three by using the remaining two. And yet, the civilised world is heavily affected by opinions: companies need them to understand how to sell their products; people use them to buy the most fitting product and, more generally, to weigh their decisions; researchers exploit them in Artificial Intelligence studies to understand the nature of the human being. Today we can count on a humongous amount of available information, though it’s hard to use it. In fact, the so-called “Big data” are not always structured – especially for certain languages. French research suffers from a lack of readily available resources for tests. In the context of Natural Language Processing, this thesis aims to explore the nature of sentiment and emotion. Some of our contributions to the NLP research community are: creation of new resources for sentiment and emotion analysis, tests and comparisons of several machine learning methods to study the problem from different points of view - classification of online reviews using sentiment polarity, classification of product characteristics using Aspect- Based Sentiment Analysis. Finally, a psycholinguistic study - supported by a machine learning and lexical approaches – on the relation between who judges, the reviewer, and the object that has been judged, the product.

Un problème central contribuant à la grande difficulté du traitement du langage naturel par des méthodes statistiques est celui de la parcimonie des données, à savoir le fait que dans un corpus d'apprentissage donné, la plupart des évènements linguistiques n'ont qu'un nombre d'occurrences assez faible, et que par ailleurs un nombre infini d'évènements permis par une langue n'apparaitront nulle part dans le corpus. Les modèles neuronaux ont déjà contribué à partiellement résoudre le problème de la parcimonie en inférant des représentations continues de mots. Ces représentations continues permettent de structurer le lexique en induisant une notion de similarité sémantique ou syntaxique entre les mots. Toutefois, les modèles neuronaux actuellement les plus répandus n'offrent qu'une solution partielle au problème de la parcimonie, notamment par le fait que ceux-ci nécessitent une représentation distribuée pour chaque mot du vocabulaire, mais sont incapables d'attribuer une représentation à des mots hors vocabulaire. Ce problème est particulièrement marqué dans des langues morphologiquement riches, ou des processus de formation de mots complexes mènent à une prolifération des formes de mots possibles, et à une faible coïncidence entre le lexique observé lors de l’entrainement d’un modèle, et le lexique observé lors de son déploiement. Aujourd'hui, l'anglais n'est plus la langue majoritairement utilisée sur le Web, et concevoir des systèmes de traduction automatique pouvant appréhender des langues dont la morphologie est très éloignée des langues ouest-européennes est un enjeu important. L’objectif de cette thèse est de développer de nouveaux modèles capables d’inférer de manière non-supervisée les processus de formation de mots sous-jacents au lexique observé, afin de pouvoir de pouvoir produire des analyses morphologiques de nouvelles formes de mots non observées lors de l’entraînement. / A crucial issue in statistical natural language processing is the issue of sparsity, namely the fact that in a given learning corpus, most linguistic events have low occurrence frequencies, and that an infinite number of structures allowed by a language will not be observed in the corpus. Neural models have already contributed to solving this issue by inferring continuous word representations. These continuous representations allow to structure the lexicon by inducing semantic or syntactic similarity between words. However, current neural models only partially solve the sparsity issue, due to the fact that they require a vectorial representation for every word in the lexicon, but are unable to infer sensible representations for unseen words. This issue is especially present in morphologically rich languages, where word formation processes yield a proliferation of possible word forms, and little overlap between the lexicon observed during model training, and the lexicon encountered during its use. Today, several languages are used on the Web besides English, and engineering translation systems that can handle morphologies that are very different from western European languages has become a major stake. The goal of this thesis is to develop new statistical models that are able to infer in an unsupervised fashion the word formation processes underlying an observed lexicon, in order to produce morphological analyses of new unseen word forms.

Les plongements lexicaux sont un composant standard des architectures modernes de traitement automatique des langues (TAL). Chaque fois qu'une avancée est obtenue dans l'apprentissage de plongements lexicaux, la grande majorité des tâches de traitement automatique des langues, telles que l'étiquetage morphosyntaxique, la reconnaissance d'entités nommées, la recherche de réponses à des questions, ou l'inférence textuelle, peuvent en bénéficier. Ce travail explore la question de l'amélioration de la qualité de plongements lexicaux monolingues appris par des modèles prédictifs et celle de la mise en correspondance entre langues de plongements lexicaux contextuels créés par des modèles préentraînés de représentation de la langue comme ELMo ou BERT.Pour l'apprentissage de plongements lexicaux monolingues, je prends en compte des informations globales au corpus et génère une distribution de bruit différente pour l'échantillonnage d'exemples négatifs dans word2vec. Dans ce but, je précalcule des statistiques de cooccurrence entre mots avec corpus2graph, un paquet Python en source ouverte orienté vers les applications en TAL : il génère efficacement un graphe de cooccurrence à partir d'un grand corpus, et lui applique des algorithmes de graphes tels que les marches aléatoires. Pour la mise en correspondance translingue de plongements lexicaux, je relie les plongements lexicaux contextuels à des plongements de sens de mots. L'algorithme amélioré de création d'ancres que je propose étend également la portée des algorithmes de mise en correspondance de plongements lexicaux du cas non-contextuel au cas des plongements contextuels. / Word embeddings are a standard component of modern natural language processing architectures. Every time there is a breakthrough in word embedding learning, the vast majority of natural language processing tasks, such as POS-tagging, named entity recognition (NER), question answering, natural language inference, can benefit from it. This work addresses the question of how to improve the quality of monolingual word embeddings learned by prediction-based models and how to map contextual word embeddings generated by pretrained language representation models like ELMo or BERT across different languages.For monolingual word embedding learning, I take into account global, corpus-level information and generate a different noise distribution for negative sampling in word2vec. In this purpose I pre-compute word co-occurrence statistics with corpus2graph, an open-source NLP-application-oriented Python package that I developed: it efficiently generates a word co-occurrence network from a large corpus, and applies to it network algorithms such as random walks. For cross-lingual contextual word embedding mapping, I link contextual word embeddings to word sense embeddings. The improved anchor generation algorithm that I propose also expands the scope of word embedding mapping algorithms from context independent to contextual word embeddings.

Avec le développement du commerceélectronique, les clients ont publié de nombreuxcommentaires de produit sur Internet. Ces donnéessont précieuses pour les concepteurs de produit, carles informations concernant les besoins de client sontidentifiables. L'objectif de cette étude est dedévelopper une approche d'analyse automatique descommentaires utilisateurs permettant d'obtenir desinformations utiles au concepteur pour guiderl'amélioration et l'innovation des produits.L’approche proposée contient deux étapes :structuration des données et analyse des données.Dans la structuration des données, l’auteur proposed’abord une ontologie pour organiser les mots et lesexpressions concernant les besoins de client décrientdans les commentaires. Ensuite, une méthode detraitement du langage naturelle basée des règleslinguistiques est proposé pour structurerautomatiquement les textes de commentaires dansl’ontologie proposée.Dans l’analyse des données, deux méthodes sontproposées pour obtenir des idées d’innovation et desvisions sur le changement de préférence d’utilisateuravec le temps. Dans ces deux méthodes, les modèleset les méthodes traditionnelles comme affordancebasedesign, l’analyse conjointe, et le Kano modelsont étudié et appliqué d’une façon innovante.Pour évaluer la praticabilité de l’approche proposéedans la réalité, les commentaires de client de liseusenumérique Kindle sont analysés. Des pistesd’innovation et des stratégies pour améliorer leproduit sont identifiés et construites. / With the development of e-commerce,consumers have posted large number of onlinereviews on the internet. These user-generated dataare valuable for product designers, as informationconcerning user requirements and preference can beidentified.The objective of this study is to develop an approachto guide product design by analyzing automaticallyonline reviews. The proposed approach consists oftwo steps: data structuration and data analytics.In data structuration, the author firstly proposes anontological model to organize the words andexpressions concerning user requirements in reviewtext. Then, a rule-based natural language processingmethod is proposed to automatically structure reviewtext into the propose ontology.In data analytics, two methods are proposed based onthe structured review data to provide designers ideason innovation and to draw insights on the changes ofuser preference over time. In these two methods,traditional affordance-based design, conjointanalysis, the Kano model are studied andinnovatively applied in the context of big data.To evaluate the practicability of the proposedapproach, the online reviews of Kindle e-readers aredownloaded and analyzed, based on which theinnovation path and the strategies for productimprovement are identified and constructed.

Large natural language models (such as GPT-2 or T5) demonstrate impressive abilities across a range of general NLP tasks. Here, we show that the knowledge embedded in such models provides a useful inductive bias, not just on traditional NLP tasks, but also in the nontraditional task of training a symbolic reasoning engine. We observe that these engines learn quickly and generalize in a natural way that reflects human intuition. For example, training such a system to model block-stacking might naturally generalize to stacking other types of objects because of structure in the real world that has been partially captured by the language describing it. We study several abstract textual reasoning tasks, such as object manipulation and navigation, and demonstrate multiple types of generalization to novel scenarios and the symbols that comprise them. We also demonstrate the surprising utility of compositional learning, where a learner dedicated to mastering a complicated task gains an advantage by training on relevant simpler tasks instead of jumping straight to the complicated task.

Indiana University-Purdue University Indianapolis (IUPUI) / Biomedical data exists in the form of journal articles, research studies, electronic health records, care guidelines, etc. While text mining and natural language processing tools have been widely employed across various domains, these are just taking off in the healthcare space. A primary hurdle that makes it difficult to build artificial intelligence models that use biomedical data, is the limited amount of labelled data available. Since most models rely on supervised or semi-supervised methods, generating large amounts of pre-processed labelled data that can be used for training purposes becomes extremely costly. Even for datasets that are labelled, the lack of normalization of biomedical concepts further affects the quality of results produced and limits the application to a restricted dataset. This affects reproducibility of the results and techniques across datasets, making it difficult to deploy research solutions to improve healthcare services. The research presented in this thesis focuses on reducing the need to create labels for biomedical text mining by using unsupervised recurrent neural networks. The proposed method utilizes word embeddings to generate vector representations of biomedical concepts based on semantics and context. Experiments with unsupervised clustering of these biomedical concepts show that concepts that are similar to each other are clustered together. While this clustering captures different synonyms of the same concept, it also captures the similarities between various diseases and the symptoms that those diseases are symptomatic of. To test the performance of the concept vectors on corpora of documents, a document vector generation method that utilizes these concept vectors is also proposed. The document vectors thus generated are used as an input to clustering algorithms, and the results show that across multiple corpora, the proposed methods of concept and document vector generation outperform the baselines and provide more meaningful clustering. The applications of this document clustering are huge, especially in the search and retrieval space, providing clinicians, researchers and patients more holistic and comprehensive results than relying on the exclusive term that they search for. At the end, a framework for extracting clinical information that can be mapped to electronic health records from preventive care guidelines is presented. The extracted information can be integrated with the clinical decision support system of an electronic health record. A visualization tool to better understand and observe patient trajectories is also explored. Both these methods have potential to improve the preventive care services provided to patients.

Indiana University-Purdue University Indianapolis (IUPUI) / Autoimmune Hepatitis (AIH), an incurable chronic condition of unknown cause where the body attacks its own liver, is a rare disease, with a current diagnosed worldwide prevalence of < 150,000. Inadequately treated, AIH can cause progressive liver damage and ultimately liver failure. A wide variety of symptoms are associated with AIH including severe fatigue, joint pain, depression, anxiety, and insomnia. While precision medicine’s genomics has attempted to shed light on the disease, other non-molecular “-omics” approaches can be taken in studying AIH patients, who often utilize social media to gather information from other patients or care providers to apply to their own AIH disease course. It is proposed that these patient-generated social mediomes can create self-report health records for patients – and facets of their lives - otherwise unreachable by conventional research. In this feasibility study, I examined in an exploratory fashion the social mediome of a large (N > 1000) gathering of AIH patients and caregivers as present on a Facebook Group to determine the potential of mining various types health-related user information. The following types of information were mined, with feasible indicating a reliability of F >= 0.670: 1) Types of health information shared and structures of information sharing (Feasible) 2) Types and directionality of support provided by and to users (Portions feasible) 3) Clinical factors (AIH-related and otherwise) disclosed by users a. Medication intake (Feasible) b. Signs and symptoms (including pain and injury) and diagnosed comorbidities (Portions feasible) c. Results of disease monitoring blood tests (Portions feasible) 4) Contextual (non-clinical; environmental; social) factors disclosed by users (Detection of which type of factor discussed occasionally feasible). The resulting knowledge is required to adequately describe the disease not only clinically, but also environmentally and socially, and will form part of the basis for future disease studies.

Indiana University-Purdue University Indianapolis (IUPUI) / Periodontal disease (PD) affects 42% of US population resulting in compromised quality of life, the potential for tooth loss and influence on overall health. Despite significant understanding of PD etiology, limited longitudinal studies have investigated PD change in response to various treatments. A major barrier is the difficulty of conducting randomized controlled trials with adequate numbers of patients over a longer time. Electronic dental record (EDR) data offer the opportunity to study outcomes following various periodontal treatments. However, using EDR data for research has challenges including quality and missing data. In this dissertation, I studied a cohort of patients with PD from EDR to monitor their disease status over time. I studied retrospectively 28,908 patients who received comprehensive oral evaluation at the Indiana University School of Dentistry between January 1st-2009 and December 31st-2014. Using natural language processing and automated approaches, we 1) determined PD diagnoses from periodontal charting based on case definitions for surveillance studies, 2) extracted clinician-recorded diagnoses from clinical notes, 3) determined the number of patients with disease improvement or progression over time from EDR data. We found 100% completeness for age, sex; 72% for race; 80% for periodontal charting findings; and 47% for clinician-recorded diagnoses. The number of visits ranged from 1-14 with an average of two visits. From diagnoses obtained from findings, 37% of patients had gingivitis, 55% had moderate periodontitis, and 28% had severe periodontitis. In clinician-recorded diagnoses, 50% patients had gingivitis, 18% had mild, 14% had moderate, and 4% had severe periodontitis. The concordance between periodontal charting-generated and clinician-recorded diagnoses was 47%. The results indicate that case definitions for PD are underestimating gingivitis and overestimating the prevalence of periodontitis. Expert review of findings identified clinicians relying on visual assessment and radiographic findings in addition to the case definition criteria to document PD diagnosis. / 2021-08-10

Indiana University-Purdue University Indianapolis (IUPUI) / Unmet social needs pose significant risk to both patients and healthcare organizations by increasing morbidity, mortality, utilization, and costs. Health care delivery organizations are increasingly employing social workers to address social needs, given the growing number of policies mandating them to identify and address their patients’ social needs. However, social workers largely document their activities using unstructured or semi-structured textual descriptions, which may not provide information that is useful for modeling, decision-making, and evaluation. Therefore, without the ability to convert these social work documentations into usable information, the utility of these textual descriptions may be limited. While manual reviews are costly, time-consuming, and require technical skills, text mining algorithms such as natural language processing (NLP) and machine learning (ML) offer cheap and scalable solutions to extracting meaningful information from large text data. Moreover, the ability to extract information on social needs and social work interventions from free-text data within electronic health records (EHR) offers the opportunity to comprehensively evaluate the outcomes specific social work interventions. However, the use of text mining tools to convert these text data into usable information has not been well explored. Furthermore, only few studies sought to comprehensively investigate the outcomes of specific social work interventions in a safety-net population. To investigate the role of social workers in addressing patients’ social needs, this dissertation: 1) utilizes NLP, to extract and categorize the social needs that lead to referral to social workers, and market basket analysis (MBA), to investigate the co-occurrence of these social needs; 2) applies NLP, ML, and deep learning techniques to extract and categorize the interventions instituted by social workers to address patients’ social needs; and 3) measures the effects of receiving a specific social work intervention type on healthcare utilization outcomes.

An event is something that happens at a time and location. Events include major life events such as graduating college or getting married, and also simple day-to-day activities such as commuting to work or eating lunch. Most work on event extraction detects events and the entities involved in events. For example, cooking events will usually involve a cook, some utensils and appliances, and a final product. In this work, we target the task of determining whether events result in their expected outcomes. Specifically, we target cooking and baking events, and characterize event outcomes into two categories. First, we distinguish whether something edible resulted from the event. Second, if something edible resulted, we distinguish between perfect, partial and alternative outcomes. The main contributions of this thesis are a corpus of 4,000 tweets annotated with event outcome information and experimental results showing that the task can be automated. The corpus includes tweets that have only text as well as tweets that have text and an image.