No description available.

Sentence labeling is a type of pattern recognition task that involves the assignment of a categorical label to each member of a sentence of observed words. Standard supervised sentence-labeling systems often have poor generalization: it is difficult to estimate parameters for words which appear in the test set, but seldom (or never) appear in the training set, because they only use words as features in their prediction tasks. Representation learning is a promising technique for discovering features that allow a supervised classifier to generalize from a source domain dataset to arbitrary new domains. We demonstrate that features which are learned from distributional representations of unlabeled data can be used to improve performance on out-of-vocabulary words and help the model to generalize. We also argue that it is important for a representation learner to be able to incorporate expert knowledge during its search for helpful features. We investigate techniques for building open-domain sentence labeling systems that approach the ideal of a system whose accuracy is high and consistent across domains. In particular, we investigate unsupervised techniques for language model representation learning that provide new features which are stable across domains, in that they are predictive in both the training and out-of-domain test data. In experiments, our best system with the proposed techniques reduce error by as much as 11.4% relative to the previous system using traditional representations on the Part-of-Speech tagging task. Moreover, we leverage the Posterior Regularization framework, and develop an architecture for incorporating biases from prior knowledge into representation learning. We investigate three types of biases: entropy bias, distance bias and predictive bias. Experiments on two domain adaptation tasks show that our biased learners identify significantly better sets of features than unbiased learners. This results in a relative reduction in error of more than 16% for both tasks with respect to existing state-of-the-art representation learning techniques. We also extend the idea of using additional unlabeled data to improve the system's performance on a different NLP task, word alignment. Traditional word alignment only takes a sentence-level aligned parallel corpus as input and generates the word-level alignments. However, as the integration of different cultures, more and more people are competent in multiple languages, and they often use elements of multiple languages in conversations. Linguist Code Switching (LCS) is such a situation where two or more languages show up in the context of a single conversation. Traditional machine translation (MT) systems treat LCS data as noise, or just as regular sentences. However, if LCS data is processed intelligently, it can provide a useful signal for training word alignment and MT models. In this work, we first extract constraints from this code switching data and then incorporate them into a word alignment model training procedure. We also show that by using the code switching data, we can jointly train a word alignment model and a language model using co-training. Our techniques for incorporating LCS data improve by 2.64 in BLEU score over a baseline MT system trained using only standard sentence-aligned corpora. / Computer and Information Science

Traditional natural language generation systems use a pipelined architecture. Two problems with this architecture are poor task decomposition and the lack of interaction between conceptual and stylistic decisions making. A revision architecture operating in a knowledge intensive environment is proposed as a means to deal with these two problems. In a revision system. text is produced and refined iteratively. A text production cycle consists of two steps. First, the text generators produce initial text. Second, this text is examined for defects by revisors. When defects are found the revisors make suggestions for the regeneration of the text. The text generator/revision cycle continues to polish the text iteratively until no more defects can be found. Although previous research has focused on stylistic revisions only. this paper describes techniques for both stylistic and conceptual revisions. Using revision to produce extended natural language text through a series of drafts provides three significant advantages over a traditional natural language generation system. First, it reduces complexity through task decomposition. Second, it promotes text polishing techniques that benefit from the ability to examine generated text in the context of the underlying knowledge from which it was generated. Third, it provides a mechanism for the integrated handling of conceptual and stylistic decisions. For revision to operate intelligently and efficiently, the revision component must have access to both the surface text and the underlying knowledge from which it was generated. A knowledge intensive architecture with a uniform knowledge base allows the revision software to quickly locate referents, choices made in producing the defective text, alternatives to the decisions made at both the conceptual and stylistic levels, and the intent of the text. The revisors use this knowledge, along with facts about the topic at hand and knowledge about how text is produced. to select alternatives for improving the text. The Kalos system was implemented to illustrate revision processing in a natural language generation system. It produces advanced draft quality text for a microprocessor users' guide from a knowledge base describing the microprocessor. It uses revision techniques in a knowledge intensive environment to iteratively polish its initial generation. The system performs both conceptual and stylistic revisions. Example output from the system, showing both types of revision, is presented and discussed. Techniques for dealing with the computational problems caused by the system's uniform knowledge base are described. / Ph. D.

This research addresses the problem, can unsupervised learning generate a representation that improves on the commonly used term frequency-inverse document frequency (TF-IDF ) representation by capturing semantic relations? The analysis measures the quality of sentence classification using term TF-IDF representations, and finds a practical upper limit to precision and recall in a biomedical text classification task (F1-score of 0.85). Arguably, one could use ontologies to supplement TF-IDF, but ontologies are sparse in coverage and costly to create. This prompts a correlated question: can unsupervised learning capture semantic relations at least as well as existing ontologies, and thus supplement existing sparse ontologies? A shallow neural network implementing the Skip-Gram algorithm is used to generate semantic vectors using a corpus of approximately 2.4 billion words. The ability to capture meaning is assessed by comparing semantic vectors generated with MESH. Results indicate that semantic vectors trained by unsupervised methods capture comparable levels of semantic features in some cases, such as amino acid (92% of similarity represented in MESH), but perform substantially poorer in more expansive topics, such as pathogenic bacteria (37.8% similarity represented in MESH). Possible explanations for this difference in performance are proposed along with a method to combine manually curated ontologies with semantic vector spaces to produce a more comprehensive representation than either alone. Semantic vectors are also used as representations for paragraphs, which, when used for classification, achieve an F1-score of 0.92. The results of classification and analogical reasoning tasks are promising but a formal model of semantic vectors, subject to the constraints of known linguistic phenomenon, is needed. This research includes initial steps for developing a formal model of semantic vectors based on a combination of linear algebra and fuzzy set theory subject to the semantic molecularism linguistic model. This research is novel in its analysis of semantic vectors applied to the biomedical domain, analysis of different performance characteristics in biomedical analogical reasoning tasks, comparison semantic relations captured by between vectors and MESH, and the initial development of a formal model of semantic vectors. / Ph. D.

Arabic news articles in heterogeneous electronic collections are difficult for users to work with. Two problems are: that they are not categorized in a way that would aid browsing, and that there are no summaries or detailed metadata records that could be easier to work with than full articles. To address the first problem, schema mapping techniques were adapted to construct a simple taxonomy for Arabic news stories that is compatible with the subject codes of the International Press Telecommunications Council. So that each article would be labeled with the proper taxonomy category, automatic classification methods were researched, to identify the most appropriate. Experiments showed that the best features to use in classification resulted from a new tailored stemming approach (i.e., a new Arabic light stemmer called P-Stemmer). When coupled with binary classification using SVM, the newly developed approach proved to be superior to state-of-the-art techniques. To address the second problem, i.e., summarization, preliminary work was done with English corpora. This was in the context of a new Problem Based Learning (PBL) course wherein students produced template summaries of big text collections. The techniques used in the course were extended to work with Arabic news. Due to the lack of high quality tools for Named Entity Recognition (NER) and topic identification for Arabic, two new tools were constructed: RenA for Arabic NER, and ALDA for Arabic topic extraction tool (using the Latent Dirichlet Algorithm). Controlled experiments with each of RenA and ALDA, involving Arabic speakers and a randomly selected corpus of 1000 Qatari news articles, showed the tools produced very good results (i.e., names, organizations, locations, and topics). Then the categorization, NER, topic identification, and additional information extraction techniques were combined to produce approximately 120,000 summaries for Qatari news articles, which are searchable, along with the articles, using LucidWorks Fusion, which builds upon Solr software. Evaluation of the summaries showed high ratings based on the 1000-article test corpus. Contributions of this research with Arabic news articles thus include a new: test corpus, taxonomy, light stemmer, classification approach, NER tool, topic identification tool, and template-based summarizer – all shown through experimentation to be highly effective. / Ph. D.

News consumption is primarily done through online news media outlets and social media. There has been a recent rise in both fake news generation, and consumption. Fake news refers to articles that deliberately contain false information to influence readers. Substantial dissemination of misinformation has been recognized to influence election results. This work focuses on hyperpartisanship in web-searched articles which refers to web searched articles which have polarized views and which represent a sensationalized view of the content. There are many such news websites which cater to propagating biased news for political and/or financial gain. This work uses Natural Language Processing (NLP) techniques on news articles to find out if a web-searched article can be termed as hyperpartisan or not. The methods were developed using a labeled dataset which was released as a part of the SemEval Task 4 - Hyperpartisan News Detection. The model was applied to queries related to U. S. midterm elections in 2018. We found that more than half the articles in web search queries showed hyperpartisanship attributes. / Master of Science / Over the recent years, the World Wide Web (WWW) has become a very important part of society. It has overgrown as a powerful medium not only to communicate with known contacts but also to gather, understand and propagate ideas with the whole world. However, in recent times there has been an increasing generation and consumption of misinformation and disinformation. These type of news, particularly fake and hyperpartisan news are particularly curated so as to hide the actual facts, and to present a biased, made-up view of the issue at hand. This activity can be harmful to the society as greater the spread and/or consumption of such news would be, more would be the negative decisions made by the readers. Thus, it poses a bigger threat to society as it affects the actions of people affected by the news. In this work, we look into a similar genre of misinformation that is hyperpartisan news. Hyperpartisan news follows a hyperpartisan orientation - the news exhibits biased opinions towards a entity (party, people, etc.) In this work, we explore to find how Natural Language Processing (NLP) methods could be used to automate the finding of hyperpartisanship in web searched articles, focusing on extraction of the linguistic features. We extend our work to test our findings in the web-searched articles related to midterm elections 2018.

In this work, we make a simple but important observation questions about images often contain premises -- objects and relationships implied by the question -- and that reasoning about premises can help Visual Question Answering (VQA) models respond more intelligently to irrelevant or previously unseen questions. When presented with a question that is irrelevant to an image, state-of-the-art VQA models will still answer based purely on learned language biases, resulting in nonsensical or even misleading answers. We note that a visual question is irrelevant to an image if at least one of its premises is false (i.e. not depicted in the image). We leverage this observation to construct a dataset for Question Relevance Prediction and Explanation (QRPE) by searching for false premises. We train novel irrelevant question detection models and show that models that reason about premises consistently outperform models that do not. We also find that forcing standard VQA models to reason about premises during training can lead to improvements on tasks requiring compositional reasoning. / Master of Science

Causal graphs are a useful notation to represent the interplay between the actors as well as the polarity and strength of the relationship that they share. They are used extensively in educational, professional, and industrial contexts to simulate different scenarios, validate behavioral aspects, visualize the connections between different processes, and explore the adversarial effects of changing certain nodes. However, as the size of the causal graphs increase, interpreting them also becomes increasingly tougher. In such cases, new analytical tools are required to enhance the user's comprehension of the graph, both in terms of correctness and speed. To this purpose, this thesis introduces 1) a system that allows for causal exploration of directed graphs, while enabling the user to see the effect of interventions on the target nodes, 2) the use of natural language generation techniques to create a coherent passage explaining the propagation effects, and 3) results of an expert user study validating the efficacy of the narratives in enhancing the user's understanding of the causal graphs. In overall, the system aims to enhance user experience and promote further causal exploration. / Master of Science / Narrative generation is the art of creating coherent snippets of text that cumulatively describe a succession of events, played across a period of time. These goals of narrative generation are also shared by causal graphs – models that encapsulate inferences between the nodes through the strength and polarity of the connecting edges. Causal graphs are an useful mechanism to visualize changes propagating amongst nodes in the system. However, as the graph starts addressing real-world actors and their interactions, it becomes increasingly difficult to understand causal inferences between distant nodes, especially if the graph is cyclic. Moreover, if the value of more than a single node is altered and the cumulative effect of the change is to be perceived on a set of target nodes, it becomes extremely difficult to the human eye. This thesis attempts to alleviate this problem by generating dynamic narratives detailing the effect of one or more interventions on one or more target nodes, incorporating time-series analysis, Wikification, and spike detection. Moreover, the narrative enhances the user's understanding of the change propagation occurring in the system. The efficacy of the narrative was further corroborated by the results of user studies, which concluded that the presence of the narrative aids the user's confidence level, correctness, and speed while exploring the causal network.

While many people enjoy hiking as a weekend activity, to many outdoor enthusiasts there is a hiking culture with which they feel affiliated. However, the way that these cultures interact with each other is still unclear. Exploring these different cultures and understanding how they relate to each other can help in engaging stakeholders of the trail. This is an important step toward finding ways to encourage environmentally friendly outdoor recreation practices and developing hiker-approved (and environmentally conscious) technologies to use on the trail. We explored these cultures by analyzing an extensive collection of tweets (over 1.5 million). We used topic modeling to identify the topics described by the communities of Triple Crown trails. We labeled training data for a classifier that identifies tweets relating to depreciative behaviors on the trail. Then, we compared the distribution of tweets across various depreciative trail behaviors to those of corresponding blog posts in order to see how tweets reflected cultures in comparison with blog posts. To harness metadata beyond the text of the tweets, we experimented with visualization techniques. We combined those efforts with ethnographic studies of hikers and conservancy organizations to produce this exploration of trail cultures. In this thesis, we show that through the use of natural language processing, we can identify cultural differences between trail communities. We identify the most significantly discussed forms of trail depreciation, which is helpful to conservation organizations so that they can more appropriately share which Leave No Trace practices hikers should place extra effort into practicing. / Master of Science / In a memoir of her hike on the Pacific Crest Trail, Wild, Cheryl Strayed said to a reporter in an amused tone, “I’m not a hobo, I’m a long-distance hiker”. While many people enjoy hiking as a weekend activity, to many outdoor enthusiasts there is a hiking culture with which they feel affiliated. There are cultures of trail conservation, and cultures of trail depreciation. There are cultures of long-distance hiking, and there are cultures of day hiking and weekend warrior hiking. There are also cultures across different hiking trails—where the hikers of one trail have different sets of values and behaviors than for another trail. However, the way that these cultures interact with each other is still unclear. Exploring these different cultures and understanding how they relate to each other can help in engaging stakeholders of the trail. This is an important step toward finding ways to encourage environmentally friendly outdoor recreation practices and developing hiker-approved (and environmentally conscious) technologies to use on the trail. We decided to explore these cultures by analyzing an extensive collection of tweets (over 1.5 million). We combined those expoorts with ethnographic style studies of conservancy organizations and avid hikers to produce this exploration of trail cultures.

Recent advances in large neural network-style models have demonstrated great performance in various applications, such as image generation, question answering, and audio classification. However, these deep and high-capacity models require a large amount of labeled data to function properly, rendering them inapplicable in many real-world scenarios. This dissertation focuses on the development and evaluation of advanced machine learning algorithms to solve the following research questions: (1) How to learn novel classes with limited labeled data, (2) How to adapt a large pre-trained model to the target domain if only unlabeled data is available, (3) How to boost the performance of the few-shot learning model with unlabeled data, and (4) How to utilize limited labeled data to learn new classes without the training data in the same domain. First, we study few-shot learning in text classification tasks. Meta-learning is becoming a popular approach for addressing few-shot text classification and has achieved state-of-the-art performance. However, the performance of existing approaches heavily depends on the interclass variance of the support set. To address this problem, we propose a TART network for few-shot text classification. The model enhances the generalization by transforming the class prototypes to per-class fixed reference points in task-adaptive metric spaces. In addition, we design a novel discriminative reference regularization to maximize divergence between transformed prototypes in task-adaptive metric spaces to improve performance further. In the second problem we focus on self-learning in cross-lingual transfer task. Our goal here is to develop a framework that can make the pretrained cross-lingual model continue learning the knowledge with large amount of unlabeled data. Existing self-learning methods in crosslingual transfer tasks suffer from the large number of incorrectly pseudo-labeled samples used in the training phase. We first design an uncertainty-aware cross-lingual transfer framework with pseudo-partial-labels. We also propose a novel pseudo-partial-label estimation method that considers prediction confidences and the limitation to the number of candidate classes. Next, to boost the performance of the few-shot learning model with unlabeled data, we propose a semi-supervised approach for few-shot semantic segmentation task. Existing solutions for few-shot semantic segmentation cannot easily be applied to utilize image-level weak annotations. We propose a class-prototype augmentation method to enrich the prototype representation by utilizing a few image-level annotations, achieving superior performance in one-/multi-way and weak annotation settings. We also design a robust strategy with softmasked average pooling to handle the noise in image-level annotations, which considers the prediction uncertainty and employs the task-specific threshold to mask the distraction. Finally, we study the cross-domain few-shot learning in the semantic segmentation task. Most existing few-shot segmentation methods consider a setting where base classes are drawn from the same domain as the new classes. Nevertheless, gathering enough training data for meta-learning is either unattainable or impractical in many applications. We extend few-shot semantic segmentation to a new task, called Cross-Domain Few-Shot Semantic Segmentation (CD-FSS), which aims to generalize the meta-knowledge from domains with sufficient training labels to low-resource domains. Then, we establish a new benchmark for the CD-FSS task and evaluate both representative few-shot segmentation methods and transfer learning based methods on the proposed benchmark. We then propose a novel Pyramid-AnchorTransformation based few-shot segmentation network (PATNet), in which domain-specific features are transformed into domain-agnostic ones for downstream segmentation modules to fast adapt to unseen domains. / Doctor of Philosophy / Nowadays, deep learning techniques play a crucial role in our everyday existence. In addition, they are crucial to the success of many e-commerce and local businesses for enhancing data analytics and decision-making. Notable applications include intelligent transportation, intelligent healthcare, the generation of natural language, and intrusion detection, among others. To achieve reasonable performance on a new task, these deep and high-capacity models require thousands of labeled examples, which increases the data collection effort and computation costs associated with training a model. Moreover, in many disciplines, it might be difficult or even impossible to obtain data due to concerns such as privacy and safety. This dissertation focuses on learning with limited labeled data in natural language processing and computer vision tasks. To recognize novel classes with a few examples in text classification tasks, we develop a deep learning-based model that can capture both cross- task transferable knowledge and task-specific features. We also build an uncertainty-aware self-learning framework and a semi-supervised few-shot learning method, which allow us to boost the pre-trained model with easily accessible unlabeled data. In addition, we propose a cross-domain few-shot semantic segmentation method to generalize the model to different domains with a few examples. By handling these unique challenges in learning with limited labeled data and developing suitable approaches, we hope to improve the efficiency and generalization of deep learning methods in the real world.