This thesis investigates the enhancement of legal contract Natural Language Inference (NLI) classification through supervised fine-tuning on general domain NLI, in the case of ContractNLI and Span NLI BERT (Koreeda and Manning, 2021), a multi-task document information extraction dataset and framework. Annotated datasets of a specific professional domain are scarce due to the high time and labour cost required to create them. Since NLI is a simple yet effective task in inducing and evaluating natural language understanding (NLU) abilities in language models, there is potential in leveraging abundant general domain NLI datasets to aid information extraction and classification for legal contracts. This work evaluates the impact of transfer learning from Adversarial NLI (Nie et al.,2020) from the general domain to ContractNLI, via sequential and mixed batch fine-tuning. The study also extends its investigation to the effects of the model’s evidence identification component on NLI, by fine-tuning on the Contract Understanding Atticus Dataset (Hendrycks et al., 2021). The results highlight the benefits of fine-tuning with general domain NLI data, particularly for hypotheses in the target task with balanced entailment and contradiction training examples. In addition, the study demonstrates the reciprocal relationship between evidence identification and NLI classification, where improvements in the former enhance the accuracy of the latter. With NLI being more commonly applied to information extraction settings in specialised domains, this work sheds light on the potential impacts of existing general domain NLI resources in stepping up classification performance in specific domains.

No description available.

Metaphoric Paraphrase generation is a relatively new and unexplored Natural Language Generation task. The aim of the task is to develop computational systems that paraphrase literal sentences into cogent metaphoric ones. Challenges in the field include representation of common sense knowledge and ensuring meaning retention when dealing with phrases that are dissimilar in their literal sense. This thesis will deal with the specific task of paraphrasing literal adjective phrases into metaphoric noun phrases, taking into consideration the preceding context of the adjective phrase. Two different systems were developed as part of this study. The systems are identical, apart from the fact that one is endowed with a knowledge representation based on Conceptual Metaphor Theory. The paraphrases generated by the systems, along with paraphrases written by a native speaker of English, were scored on the parameters of meaning retention and creativity by a crowd-sourced panel. Both systems were able to generate cogent metaphoric paraphrases, although fairly unreliably compared to the human. The system endowed with Conceptual Metaphor Theory knowledge got a lower average meaning retention score and a higher average creativity score than the system without Conceptual Metaphor Theory knowledge representation. In addition to that it was found that less similarity in sentence embeddings of literal sentences and metaphoric paraphrases of them correlates with a higher level of perceived meaning retention and a lower perceived creativity of the metaphoric paraphrase. It was also found that less difference in GPT-2 log probability between literal sentences and metaphoric paraphrases of them correlates with humans evaluating the paraphrases as less creative.

In this thesis, we explore the impact of M-BERT and different transfer sizes on the choice of different transfer languages in dependency parsing. In order to investigate our research questions, we conduct a series of experiments on the treebanks in Universal Dependencies with UUParser.     The main conclusions and contributions of this study are as follows:   First, we train a variety of languages in several different scripts with M-BERT being added into the parsing framework, which is one of the most state-of-the-art deep learning models based on the Transformer architecture. In general, we get advancing results with M-BERT compared with the randomly initialized embedding in UUParser.    Second, since it is a common way to choose a source language, which is 'close' to the target language in cross-lingual parsing, we try to explore what 'close' languages actually are, as there is not a definition for 'close'. In our study, we explore how strongly the parsing results are correlated with the different linguistic distances between the source and target languages. The relevant data is queried from URIEL Database. We find that the parsing performance is more dependent on inventory, syntactic and featural distance than on the geographic, genetic and phonological distance in zero-shot experiments. In the few-shot prediction, the parsing accuracy shows stronger correlation with inventory and syntactic distance than with others.     Third, we vary the training sizes in few-shot experiments with M-BERT being added to see how the parsing results are influenced. We find that it is very obvious that few-shot experiments outperform zero-shot experiments. With the source sizes being cut, all parsing scores decrease. However, we do not see a linear drop of the results.

No description available.

No description available.

Text normalization plays a crucial role in text-to-speech systems by ensuring that the input text is in an appropriate format and consists of standardized words prior to grapheme-to-phoneme conversion for text-to-speech. The aim of this study was to assess the performance of five text normalization systems based on different methods. These text normalization systems were evaluated on the English Google text normalization dataset. The evaluation was based on the similarity between the ground truth and normalized outputs from each text normalization system. Since multiple ground truth issues occurred during the evaluation, the original similarity scores needed to be manually re-scored. The re-scoring was employed on a sample data semi-randomly extracted from the evaluation dataset. According to the results, the accuracy of these text normalization systems  can be ranked as follows: the Duplex system, the Hybrid system, the VT system, the RS system, and the WFST system. For the two rule-based systems from ReadSpeaker, the VT system performed slightly better than the RS system, with a slight difference in the original similarity score. By analyzing the error patterns produced during the normalization process, the study provided valuable insights into the strengths and limitations of these systems. The findings of this study contribute to the refinement of internal rules, leading to improved accuracy and effectiveness of text normalization in text-to-speech applications.

This thesis demonstrates that hierarchical classifiers that organize a classifica-tion task across multiple, sequential stages substantially improve emotion clas-sification accuracy compared to flat classifiers that classify text in a single step, effectively recognizing underrepresented emotions and classifying similar emo-tions within Japanese social media texts, using the WRIME dataset. The study also investigates different training data formats—emotion intensity lists and sin-gle primary emotion labels—and reveals that while classifiers trained with lists generally achieve better overall performance, single-label classifiers excel inidentifying underrepresented emotions and those expressed with high intensity.We also explore the impact of sentence length and emotion intensity on classifierperformance, finding that hierarchical classifiers are more effective with textsthat exhibit higher emotional intensities and shorter sentences. These findingsdemonstrate the potential of hierarchical approaches to address the complexities of emotion classification tasks, contributing valuable insights into the develop-ment of more effective natural language processing systems. Future research directions include enhancing the computational efficiency of hierarchical classi-fiers and comparing the hierarchical classifiers implemented with various deep learning approaches.

In this thesis, the development of an automatic speech recognition (ASR) system for Northern Sotho, a low-resource language in South Africa, is investigated. Low-resource languages face challenges such as limited linguistic data and insufficient computational resources. In an attempt to alleviate these challenges, the multilingual Wav2Vec2-XLSR model is fine-tuned using Northern Sotho speech data with two main strategies to improve ASR performance: inclusion of background noise during training and semi-supervised learning with additional generated labels. An additional dataset compiled from news in Northern Sotho is used for evaluation of the models. The experiments demonstrate that moderate levels of background noise can enhance model robustness, though excessive noise degrades performance, particularly on clean data. Semi-supervised learning with generated labels proves beneficial, especially when working with smaller labelled datasets, though optimal results are always achieved with large, in-domain labelled datasets. The last finding is confirmed by the additional news dataset, which proved extremely challenging, with high error rates achieved by models trained on clean data and limited benefits of noise augmentation.

Research projects in Optimality Theory tend to take a synchronic view of a particular generalisation, and set their standards for rigour in typological terms (see for example Suzuki 1998 on dissimilation, Crosswhite 2001 on vowel reduction). The goal of this thesis is to use Stratal OT to take a diachronic view of multiple generalisations within the morpho-phonology of one language, namely Latin, with the principal empirical aim of producing an analysis that is demonstrably true to all the attested facts of the generalisations in question. To that end, I have written PyOT, a computer program implementing the OT calculus and a theory of phonological representations, which I use in this work to model the histories of Lachmann’s Law, rhotacism and the phonologically conditioned allomorphy of the -alis/aris- suffix as active generalisations within the phonological component of the grammar. Appendix A gives the results of the computer model applied to a dataset consisting of 185 attested Latin forms, which suffice to illustrate the exact conditions of the generalisations in question. I show that producing a complete analysis of the three generalisations I have chosen to model entails analysis of other generalisations that interact with them, including the treatment of the Indo-European voiced aspirates in Latin (which interacts with rhotacism), and reduplication in forming perfect stems (which interacts with Lachmann’s Law). Constraint rankings sufficient to model these interactions, and consistent with the general conditions of the interacting generalisations have been included in the model. The intention is for this work to illustrate both the utility of formal phonological theory in advancing hypotheses within historical-comparative linguistics, and the potential of PyOT as a tool for producing Optimality-Theoretic models of (eventually) a language’s entire phonology.