Paraphrasing and translation have previously been treated as unconnected natural language processing tasks. We show the two are intimately related. The major contributions of this thesis are as follows: We define a novel technique for automatically generating paraphrases using bilingual parallel corpora, which are more commonly used as training data statistical models of translation. We show that paraphrases can be used to improve the quality of statistical machine translation by addressing the problem of coverage and introducing a degree of generalisation into the models. We explore the topic of automatic evaluation of translation quality, and show that the current standard evaluation methodology cannot be guaranteed to correlate with human judgements of translation quality. Whereas previous data-driven approaches to paraphrasing were dependent upon either data sources which were uncommon such as multiple translation of the same source text, or language specific resources such as parsers, our approach is able to harness more widely parallel corpora and can be applied to any language which has a parallel corpus. Being a language independent and probabilistic approach allows our method to be easily integrated into statistical machine translation. Paraphrasing can be used to increase coverage by adding translations of previously unseen source words and phrases. Results show that augmenting a state-of-the-art SMT system with paraphrases leads to significantly improved coverage and translation quality. For a training corpus with 10,000 sentence pairs we increase the coverage of unique test set unigrams from 48% to 90%, with more than half of the newly covered items accurately translated, as opposed to none in current approaches.
| Identifer | oai:union.ndltd.org:bl.uk/oai:ethos.bl.uk:642332 |
| Date | January 2008 |
| Creators | Callison-Burch, C. |
| Publisher | University of Edinburgh |
| Source Sets | Ethos UK |
| Detected Language | English |
| Type | Electronic Thesis or Dissertation |
Page generated in 0.0051 seconds