La tâche d'alignement d'un texte dans une langue source avec sa traduction en langue cible est souvent nommée alignement de bi-textes. Elle a pour but de faire émerger les relations de traduction qui peuvent s'exprimer à différents niveaux de granularité entre les deux faces du bi-texte. De nombreuses applications de traitement automatique des langues naturelles s'appuient sur cette étape afin d'accéder à des connaissances linguistiques de plus haut niveau.Parmi ces applications, nous pouvons citer bien sûr la traduction automatique, mais également l'extraction de lexiques et de terminologies bilingues, la désambigüisation sémantique ou l'apprentissage des langues assisté par ordinateur.La complexité de la tâche d'alignement de bi-textes s'explique par les différences linguistiques entre les langues. Ces différences peuvent être d'ordre sémantique, syntaxique, ou morphologique.Dans le cadre des approches probabilistes, l'alignement de bi-textes est modélisé par un ensemble de variables aléatoires cachés. Afin de réduire la complexité du problème, le processus aléatoire sous-jacent fait l'hypothèse simplificatrice qu'un mot en langue source est lié à au plus un mot en langue cible, ce qui induit une relation de traduction asymétrique. Néanmoins, cette hypothèse est simpliste, puisque les alignements peuvent de manière générale impliquer des groupes de mots dans chacune des langues. Afin de rétablir cette symétrie, chaque langue est considérée tour à tour comme la langue source et les deux alignements asymétriques résultants sont combinés à l'aide d'une heuristique. Cette étape de symétrisation revêt une importance particulière dans l’approche standard en traduction automatique, puisqu'elle précède l'extraction des unités de traduction, à savoir les paires de segments.L'objectif de cette thèse est de proposer de nouvelles approches pour d'une part l'alignement debi-texte, et d'autre part l'extraction des unités de traduction. La spécificité de notre approche consiste à remplacer les heuristiques utilisées par des modèles d'apprentissage discriminant.Nous présentons un modèle "Maximum d'entropie'' (ou MaxEnt) pour l'alignement de mot, pour lequel chaque lien d'alignement est prédit de manière indépendante. L'interaction entre les liens d'alignement est alors prise en compte par l'empilement ("stacking'') d'un second modèle prenant en compte la structure à prédire sans pour autant augmenter la complexité globale. Cette formulation peut être vue comme une manière d'apprendre la combinaison de différentes méthodes d'alignement: le modèle considère ainsi l'union des alignements d'entrées pour en sélectionner les liens jugés fiables. Le modèle MaxEnt proposé permet d'améliorer les performances d'un système état de l'art de traduction automatique en considérant le jeu de données de la tâche NIST'09, Arabe vers Anglais. Ces améliorations sont mesurées en terme de taux d'erreur sur les alignements et aussi en terme de qualité de traduction via la métrique automatique BLEU.Nous proposons également un modèle permettant à la fois de sélectionner et d'évaluer les unités de traduction extraites d'un bi texte aligné. Ces deux étapes sont reformulées dans le cadre de l'apprentissage supervisé, afin de modéliser la décision de garder ou pas une paire de segments comme une unité fiable de traduction. Ce cadre permet l'utilisation de caractéristiques riches et nombreuses favorisant ainsi une décision robuste. Nous proposons une méthode simple et efficace pour annoter les paires de segments utiles pour la traduction. Le problème d'apprentissage automatique qui se pose alors est particulier, puisque nous disposons que d'exemples positifs. Nous proposons donc d'utiliser l'approche SVM à une classe afin de modéliser la sélection des unités de traduction.Grâce à cette approche, nous obtenons des améliorations significatives en terme de score BLEU pour un système entrainé avec un petit ensemble de données. / Bitext alignment is the task of aligning a text in a source language and its translation in the target language. Aligning amounts to finding the translational correspondences between textual units at different levels of granularity. Many practical natural language processing applications rely on bitext alignments to access the rich linguistic knowledge present in a bitext. While the most predominant application for bitexts is statistical machine translation, they are also used in multilingual (and monolingual) lexicography, word sense disambiguation, terminology extraction, computer-aided language learning andtranslation studies, to name a few.Bitext alignment is an arduous task because meaning is not expressed seemingly across languages. It varies along linguistic properties and cultural backgrounds of different languages, and also depends on the translation strategy that have been used to produce the bitext.Current practices in bitext alignment model the alignment as a hidden variable in the translation process. In order to reduce the complexity of the task, such approaches suppose that a word in the source sentence is aligned to one word at most in the target sentence.However, this over-simplistic assumption results in asymmetric, one-to-many alignments, whereas alignments are typically symmetric and many-to-many.To achieve symmetry, two one-to-many alignments in opposite translation directions are built and combined using a heuristic.In order to use these word alignments in phrase-based translation systems which use phrases instead of words, a heuristic is used to extract phrase pairs that are consistent with the word alignment.In this dissertation we address both the problems of word alignment and phrase pairs extraction.We improve the state of the art in several ways using discriminative learning techniques.We present a maximum entropy (MaxEnt) framework for word alignment.In this framework, links are predicted independently from one another using a MaxEnt classifier.The interaction between alignment decisions is approximated using stackingtechniques, which allows us to account for a part of the structural dependencies without increasing the complexity. This formulation can be seen as an alignment combination method,in which the union of several input alignments is used to guide the output alignment. Additionally, input alignments are used to compute a rich set of feature functions.Our MaxEnt aligner obtains state of the art results in terms of alignment quality as measured by thealignment error rate, and translation quality as measured by BLEU on large-scale Arabic-English NIST'09 systems.We also present a translation quality informed procedure for both extraction and evaluation of phrase pairs. We reformulate the problem in the supervised framework in which we decide for each phrase pair whether we keep it or not in the translation model. This offers a principled way to combine several features to make the procedure more robust to alignment difficulties. We use a simple and effective method, based on oracle decoding,to annotate phrase pairs that are useful for translation. Using machine learning techniques based on positive examples only,these annotations can be used to learn phrase alignment decisions. Using this approach we obtain improvements in BLEU scores for recall-oriented translation models, which are suitable for small training corpora.

La traduction automatique des documents est considérée comme l’une des tâches les plus difficiles en traitement automatique des langues et de la parole. Les particularités linguistiques de certaines langues, comme la langue arabe, rendent la tâche de traduction automatique plus difficile. Notre objectif dans cette thèse est d'améliorer les systèmes de traduction de l'arabe vers le français et vers l'anglais. Nous proposons donc une étude détaillée sur ces systèmes. Les principales recherches portent à la fois sur la construction de corpus parallèles, le prétraitement de l'arabe et sur l'adaptation des modèles de traduction et de langue.Tout d'abord, un corpus comparable journalistique a été exploré pour en extraire automatiquement un corpus parallèle. Ensuite, différentes approches d’adaptation du modèle de traduction sont exploitées, soit en utilisant le corpus parallèle extrait automatiquement soit en utilisant un corpus parallèle construit automatiquement.Nous démontrons que l'adaptation des données du système de traduction permet d'améliorer la traduction. Un texte en arabe doit être prétraité avant de le traduire et ceci à cause du caractère agglutinatif de la langue arabe. Nous présentons notre outil de segmentation de l'arabe, SAPA (Segmentor and Part-of-speech tagger for Arabic), indépendant de toute ressource externe et permettant de réduire les temps de calcul. Cet outil permet de prédire simultanément l’étiquette morpho-syntaxique ainsi que les proclitiques (conjonctions, prépositions, etc.) pour chaque mot, ensuite de séparer les proclitiques du lemme (ou mot de base). Nous décrivons également dans cette thèse notre outil de détection des entités nommées, NERAr (Named Entity Recognition for Arabic), et nous examions l'impact de l'intégration de la détection des entités nommées dans la tâche de prétraitement et la pré-traduction de ces entités nommées en utilisant des dictionnaires bilingues. Nous présentons par la suite plusieurs méthodes pour l'adaptation thématique des modèles de traduction et de langue expérimentées sur une application réelle contenant un corpus constitué d’un ensemble de phrases multicatégoriques.Ces expériences ouvrent des perspectives importantes de recherche comme par exemple la combinaison de plusieurs systèmes lors de la traduction pour l'adaptation thématique. Il serait également intéressant d'effectuer une adaptation temporelle des modèles de traduction et de langue. Finalement, les systèmes de traduction améliorés arabe-français et arabe-anglais sont intégrés dans une plateforme d'analyse multimédia et montrent une amélioration des performances par rapport aux systèmes de traduction de base. / Machine Translation is one of the most difficult tasks in natural language and speech processing. The linguistic peculiarities of some languages makes the machine translation task more difficult. In this thesis, we present a detailed study of machine translation systems from arabic to french and to english.Our principle researches carry on building parallel corpora, arabic preprocessing and adapting translation and language models. We propose a method for automatic extraction of parallel news corpora from a comparable corpora. Two approaches for translation model adaptation are explored using whether parallel corpora extracted automatically or parallel corpora constructed automatically. We demonstrate that adapting data used to build machine translation system improves translation.Arabic texts have to be preprocessed before machine translation and this because of the agglutinative character of arabic language. A prepocessing tool for arabic, SAPA (Segmentor and Part-of-speech tagger for Arabic), much faster than the state of the art tools and totally independant of any other external resource was developed. This tool predicts simultaneously morphosyntactic tags and proclitics (conjunctions, prepositions, etc.) for every word, then splits off words into lemma and proclitics.We describe also in this thesis, our named entity recognition tool for arabic, NERAr, and we focus on the impact of integrating named entity recognition in the preprocessing task. We used bilingual dictionaries to propose translations of the detected named entities. We present then many approaches to adapt thematically translation and language models using a corpora consists of a set of multicategoric sentences.These experiments open important research perspectives such as combining many systems when translating. It would be interesting also to focus on a temporal adaptation of translation and language models.Finally, improved machine translation systems from arabic to french and english are integrated in a multimedia platform analysis and shows improvements compared to basic machine translation systems.

[pt] Este estudo foi motivado pela utilização, ainda em caráter experimental, de um tradutor automático por uma empresa multinacional de localização de software. A fim de contribuir para essa iniciativa pioneira no país, propomos uma avaliação do programa, enfocando as implicações da utilização desta tecnologia no processo de localização de software. Empregamos a taxonomia proposta pelo FEMTI (Framework for the Evaluation of Machine Translation in ISLE), desenvolvida especialmente para a avaliação de tradução automática, com base nas normas ISO/IEC de qualidade de software. São considerados aspectos operacionais, como a integração do sistema de tradução automática às ferramentas de memória de tradução, bem como questões relacionadas à linguagem. O corpus utilizado para a avaliação foi um manual de usuário de um telefone celular. Além dos problemas lingüísticos recorrentes na maioria das ferramentas de tradução automática disponíveis atualmente, são analisados os desvios relacionados à tradução da interface com o usuário, mais especificamente aos menus do telefone celular. Esses desvios são discutidos dentro das categorias pertinentes da taxonomia do FEMTI e, sempre que possível, foram sugeridas soluções. Para complementar a análise lingüística, apresentamos outros três estudos realizados para o português. Nossos resultados indicam que o sistema pode ser bem-sucedido neste mercado em função principalmente da delimitação do domínio e da adoção dos procedimentos impostos pelo processo de localização. Esse sucesso depende da integração do tradutor automático às memórias de tradução e de investimentos relativamente pequenos na atualização dos recursos lingüísticos (regras gramaticais e dicionários) para refletir as características próprias do domínio e do tipo de texto. / [en] This study was motivated by the trial implementation of a machine translation engine by a multinational software localization company. In order to contribute to this innovative experiment in the Brazilian market, we evaluate the engine, focusing on the implications of its implementation in the software localization industry. We use the FEMTI (Framework for the Evaluation of Machine Translation in ISLE) taxonomy, which is based on the ISO/IEC guidelines for software evaluation. Operational aspects, such as the engine s integration with translation memory tools, are taken into consideration, as well as language issues. Our evaluation is based on the machine translated version of a mobile phone user guide. In addition to the language problems common to most machine translation engines currently available, we analyze issues related to the user interface, particularly to the phone menus. These problems are discussed as examples of each related FEMTI topic and we suggest solutions whenever possible. To add to our language evaluation, we present three other studies dedicated to Portuguese. Our results indicate the engine can be successful in this industry mainly in terms of domain restriction and localization workflow procedures. Its success depends on its integration to translation memory tools and requires relatively little investment in updating the language resources (rules and dictionaries) to reflect the language characteristics specific to domain and text type.

Cette thèse de doctorat aborde les problématiques de l'estimation de confiance pour la traduction automatique, et de la traduction automatique statistique de la parole spontanée à grand vocabulaire. J'y propose une formalisation du problème d'estimation de confiance, et aborde expérimentalement le problème sous le paradigme de la classification et régression multivariée. Je propose une évaluation des performances des différentes méthodes évoquées, présente les résultats obtenus lors d'une campagne d'évaluation internationale et propose une application à la post-édition par des experts de documents traduits automatiquement. J'aborde ensuite le problème de la traduction automatique de la parole. Après avoir passé en revue les spécificités du medium oral et les défis particuliers qu'il soulève, je propose des méthodes originales pour y répondre, utilisant notamment les réseaux de confusion phonétiques, les mesures de confiances et des techniques de segmentation de la parole. Je montre finalement que le prototype propose rivalise avec des systèmes état de l'art à la conception plus classique / In this thesis I shall deal with the issues of confidence estimation for machine translation and statistical machine translation of large vocabulary spontaneous speech translation. I shall first formalize the problem of confidence estimation. I present experiments under the paradigm of multivariate classification and regression. I review the performances yielded by different techniques, present the results obtained during the WMT2012 internation evaluation campaign and give the details of an application to post edition of automatically translated documents. I then deal with the issue of speech translation. After going into the details of what makes it a very specific and particularly challenging problem, I present original methods to partially solve it, by using phonetic confusion networks, confidence estimation techniques and speech segmentation. I show that the prototype I developped yields performances comparable to state-of-the-art of more standard design

Différentes composantes des systèmes de traduction automatique statistique sont considérées comme des problèmes d'optimisations. En effet, l'apprentissage du modèle de traduction, le décodage et l'optimisation des poids de la fonction log-linéaire sont trois importants problèmes d'optimisation. Savoir définir les bons algorithmes pour les résoudre est l'une des tâches les plus importantes afin de mettre en place un système de traduction performant. Plusieurs algorithmes d'optimisation sont proposés pour traiter les problèmes d'optimisation du décodeur. Ils sont combinés pour résoudre, d'une part, le problème de décodage qui produit une traduction dans la langue cible d'une phrase source, d'autre part, le problème d'optimisation des poids des scores combinés dans la fonction log-linéaire pour d'évaluation des hypothèses de traduction au cours du décodage. Le système de traduction statistique de référence est basé sur un algorithme de recherche en faisceau pour le décodage, et un algorithme de recherche linéaire pour l'optimisation des poids associés aux scores. Nous proposons un nouveau système de traduction avec un décodeur entièrement basé sur les algorithmes génétiques. Les algorithmes génétiques sont des algorithmes d'optimisation bio-inspirés qui simulent le processus de l'évolution naturelle des espèces. Ils permettent de manipuler un ensemble de solutions à travers plusieurs itérations pour converger vers des solutions optimales. Ce travail, nous permet d'étudier l'efficacité des algorithmes génétiques pour la traduction automatique statistique. L'originalité de notre proposition est de proposer deux algorithmes : un algorithme génétique, appelé GAMaT, comme décodeur pour un système de traduction statistique à base de segments, et un algorithme génétique, appelé GAWO, pour l'optimisation des poids de la fonction log-linéaire afin de l'utiliser comme fonction fitness pour GAMaT. Nous proposons également, une approche neuronale pour définir une nouvelle fonction fitness pour GAMaT. Cette approche consiste à utiliser un réseau de neurones pour l'apprentissage d'une fonction qui combine plusieurs scores, évaluant différents aspects d'une hypothèse de traduction, combinés auparavant dans la fonction log-linéaire, et qui prédit le score BLEU de cette hypothèse de traduction. Ce travail, nous a permis de proposer un nouveau système de traduction automatique statistique ayant un décodeur entièrement basé sur des algorithmes génétiques / Different components of statistical machine translation systems are considered as optimization problems. Indeed, the learning of the translation model, the decoding and the optimization of the weights of the log-linear function are three important optimization problems. Knowing how to define the right algorithms to solve them is one of the most important tasks in order to build an efficient translation system. Several optimization algorithms are proposed to deal with decoder optimization problems. They are combined to solve, on the one hand, the decoding problem that produces a translation in the target language for each source sentence, on the other hand, to solve the problem of optimizing the weights of the combined scores in the log-linear function to fix the translation evaluation function during the decoding. The reference system in statistical translation is based on a beam-search algorithm for the decoding, and a line search algorithm for optimizing the weights associated to the scores. We propose a new statistical translation system with a decoder entirely based on genetic algorithms. Genetic algorithms are bio-inspired optimization algorithms that simulate the natural process of evolution of species. They allow to handle a set of solutions through several iterations to converge towards optimal solutions. This work allows us to study the efficiency of the genetic algorithms for machine translation. The originality of our work is the proposition of two algorithms: a genetic algorithm, called GAMaT, as a decoder for a phrase-based machine translation system, and a second genetic algorithm, called GAWO, for optimizing the weights of the log-linear function in order to use it as a fitness function for GAMaT. We propose also, a neuronal approach to define a new fitness function for GAMaT. This approach consists in using a neural network to learn a function that combines several scores, which evaluate different aspects of a translation hypothesis, previously combined in the log-linear function, and that predicts the BLEU score of this translation hypothesis. This work allowed us to propose a new machine translation system with a decoder entirely based on genetic algorithms

Alors que Twitter évolue vers un outil omniprésent de diffusion de l'information, la compréhension des tweets en langues étrangères devient un problème important et difficile. En raison de la nature intrinsèquement à commutation de code, discrète et bruitée des tweets, la traduction automatique (MT) à l'état de l'art n'est pas une option viable (Farzindar & Inkpen, 2015). En effet, au moins pour le hindi et le japonais, nous observons que le pourcentage de tweets « compréhensibles » passe de 80% pour les locuteurs natifs à moins de 30% pour les lecteurs monolingues cible (anglais ou français) utilisant Google Translate. Notre hypothèse de départ est qu'il devrait être possible de créer des outils génériques, permettant aux étrangers de comprendre au moins 70% des « tweets locaux », en utilisant une interface polyvalente de « lecture active » (LA, AR en anglais) tout en déterminant simultanément le pourcentage de tweets compréhensibles en-dessous duquel un tel système serait jugé inutile par les utilisateurs prévus.Nous avons donc spécifié un « SUFT » (système d'aide à la compréhension des tweets étrangers) générique, et mis en œuvre SUFT-1, un système interactif à mise en page multiple basé sur la LA, et facilement configurable en ajoutant des dictionnaires, des modules morphologiques et des plugins de TA. Il est capable d'accéder à plusieurs dictionnaires pour chaque langue source et fournit une interface d'évaluation. Pour les évaluations, nous introduisons une mesure liée à la tâche induisant un coût négligeable, et une méthodologie visant à permettre une « évaluation continue sur des données ouvertes », par opposition aux mesures classiques basées sur des jeux de test liés à des ensembles d'apprentissage fermés. Nous proposons de combiner le taux de compréhensibilité et le temps de décision de compréhensibilité comme une mesure de qualité à deux volets, subjectif et objectif, et de vérifier expérimentalement qu'une présentation de type lecture active, basée sur un dictionnaire, peut effectivement aider à comprendre les tweets mieux que les systèmes de TA disponibles.En plus de rassembler diverses ressources lexicales, nous avons construit une grande ressource de "formes de mots" apparaissant dans les tweets indiens, avec leurs analyses morphologiques (à savoir 163221 formes de mots hindi dérivées de 68788 lemmes et 72312 formes de mots marathi dérivées de 6026 lemmes) pour créer un analyseur morphologique multilingue spécialisé pour les tweets, capable de gérer des tweets à commutation de code, de calculer des traits unifiés, et de présenter un tweet en lui attachant un graphe de LA à partir duquel des lecteurs étrangers peuvent extraire intuitivement une signification plausible, s'il y en a une. / As Twitter evolves into a ubiquitous information dissemination tool, understanding tweets in foreign languages becomes an important and difficult problem. Because of the inherent code-mixed, disfluent and noisy nature of tweets, state-of-the-art Machine Translation (MT) is not a viable option (Farzindar & Inkpen, 2015). Indeed, at least for Hindi and Japanese, we observe that the percentage of "understandable" tweets falls from 80% for natives to below 30% for target (English or French) monolingual readers using Google Translate. Our starting hypothesis is that it should be possible to build generic tools, which would enable foreigners to make sense of at least 70% of “native tweets”, using a versatile “active reading” (AR) interface, while simultaneously determining the percentage of understandable tweets under which such a system would be deemed useless by intended users.We have thus specified a generic "SUFT" (System for Helping Understand Tweets), and implemented SUFT-1, an interactive multi-layout system based on AR, and easily configurable by adding dictionaries, morphological modules, and MT plugins. It is capable of accessing multiple dictionaries for each source language and provides an evaluation interface. For evaluations, we introduce a task-related measure inducing a negligible cost, and a methodology aimed at enabling a « continuous evaluation on open data », as opposed to classical measures based on test sets related to closed learning sets. We propose to combine understandability ratio and understandability decision time as a two-pronged quality measure, one subjective and the other objective, and experimentally ascertain that a dictionary-based active reading presentation can indeed help understand tweets better than available MT systems.In addition to gathering various lexical resources, we constructed a large resource of "word-forms" appearing in Indian tweets with their morphological analyses (viz. 163221 Hindi word-forms from 68788 lemmas and 72312 Marathi word-forms from 6026 lemmas) for creating a multilingual morphological analyzer specialized to tweets, which can handle code-mixed tweets, compute unified features, and present a tweet with an attached AR graph from which foreign readers can intuitively extract a plausible meaning, if any.

A comunicação multilíngue é uma tarefa cada vez mais imperativa no cenário atual de grande disseminação de informações em diversas línguas. Nesse contexto, são de grande relevância os sistemas de tradução automática, que auxiliam tal comunicação, automatizando-a. Apesar de ser uma área de pesquisa bastante antiga, a Tradução Automática ainda apresenta muitos problemas. Um dos principais problemas é a ambigüidade lexical, ou seja, a necessidade de escolha de uma palavra, na língua alvo, para traduzir uma palavra da língua fonte quando há várias opções de tradução. Esse problema se mostra ainda mais complexo quando são identificadas apenas variações de sentido nas opções de tradução. Ele é denominado, nesse caso, \"ambigüidade lexical de sentido\". Várias abordagens têm sido propostas para a desambiguação lexical de sentido, mas elas são, em geral, monolíngues (para o inglês) e independentes de aplicação. Além disso, apresentam limitações no que diz respeito às fontes de conhecimento que podem ser exploradas. Em se tratando da língua portuguesa, em especial, não há pesquisas significativas voltadas para a resolução desse problema. O objetivo deste trabalho é a proposta e desenvolvimento de uma nova abordagem de desambiguação lexical de sentido, voltada especificamente para a tradução automática, que segue uma metodologia híbrida (baseada em conhecimento e em córpus) e utiliza um formalismo relacional para a representação de vários tipos de conhecimentos e de exemplos de desambiguação, por meio da técnica de Programação Lógica Indutiva. Experimentos diversos mostraram que a abordagem proposta supera abordagens alternativas para a desambiguação multilíngue e apresenta desempenho superior ou comparável ao do estado da arte em desambiguação monolíngue. Adicionalmente, tal abordagem se mostrou efetiva como mecanismo auxiliar para a escolha lexical na tradução automática estatística / Crosslingual communication has become a very imperative task in the current scenario with the increasing amount of information dissemination in several languages. In this context, machine translation systems, which can facilitate such communication by providing automatic translations, are of great importance. Although research in Machine Translation dates back to the 1950\'s, the area still has many problems. One of the main problems is that of lexical ambiguity, that is, the need for lexical choice when translating a source language word that has several translation options in the target language. This problem is even more complex when only sense variations are found in the translation options, a problem named \"sense ambiguity\". Several approaches have been proposed for word sense disambiguation, but they are in general monolingual (for English) and application-independent. Moreover, they have limitations regarding the types of knowledge sources that can be exploited. Particularly, there is no significant research aiming to word sense disambiguation involving Portuguese. The goal of this PhD work is the proposal and development of a novel approach for word sense disambiguation which is specifically designed for machine translation, follows a hybrid methodology (knowledge and corpus-based), and employs a relational formalism to represent various kinds of knowledge sources and disambiguation examples, by using Inductive Logic Programming. Several experiments have shown that the proposed approach overcomes alternative approaches in multilingual disambiguation and achieves higher or comparable results to the state of the art in monolingual disambiguation. Additionally, the approach has shown to effectively assist lexical choice in a statistical machine translation system

Machine Translation (MT) for language pairs with long distance dependencies and word reordering, such as German-English, is prone to producing output that is lexically or syntactically incoherent. Statistical MT (SMT) models used explicit or latent syntax to improve reordering, however failed at capturing other long distance dependencies. This thesis explores how explicit sentence-level syntactic information can improve translation for such complex linguistic phenomena. In particular, we work at the level of the syntactic-semantic interface with representations conveying the predicate-argument structures. These are essential to preserving semantics in translation and SMT systems have long struggled to model them. String-to-tree SMT systems use explicit target syntax to handle long-distance reordering, but make strong independence assumptions which lead to inconsistent lexical choices. To address this, we propose a Selectional Preferences feature which models the semantic affinities between target predicates and their argument fillers using the target dependency relations available in the decoder. We found that our feature is not effective in a string-to-tree system for German-English and that often the conditioning context is wrong because of mistranslated verbs. To improve verb translation, we proposed a Neural Verb Lexicon Model (NVLM) incorporating sentence-level syntactic context from the source which carries relevant semantic information for verb disambiguation. When used as an extra feature for re-ranking the output of a German-English string-to-tree system, the NVLM improved verb translation precision by up to 2.7% and recall by up to 7.4%. While the NVLM improved some aspects of translation, other syntactic and lexical inconsistencies are not being addressed by a linear combination of independent models. In contrast to SMT, neural machine translation (NMT) avoids strong independence assumptions thus generating more fluent translations and capturing some long-distance dependencies. Still, incorporating additional linguistic information can improve translation quality. We proposed a method for tightly coupling target words and syntax in the NMT decoder. To represent syntax explicitly, we used CCG supertags, which encode subcategorization information, capturing long distance dependencies and attachments. Our method improved translation quality on several difficult linguistic constructs, including prepositional phrases which are the most frequent type of predicate arguments. These improvements over a strong baseline NMT system were consistent across two language pairs: 0.9 BLEU for German-English and 1.2 BLEU for Romanian-English.

<p>In this thesis I have investigated verb frame divergences in a bilingual Head-driven Phrase Structure Grammar for machine translation. The purpose was threefold: (1) to describe and classify verb frame divergences (VFDs) between Swedish and English, (2) to practically implement a bilingual grammar that covered many of the identified VFDs and (3) to find out what cases of VFDs could be solved and implemented using a common semantic representation, or interlingua, for Swedish and English.</p><p>The implemented grammar, BiTSE, is a Head-driven Phrase Structure Grammar based on the LinGO Grammar Matrix, a language independent grammar base. BiTSE is a bilingual grammar containing both Swedish and English. The semantic representation used is Minimal Recursion Semantics (MRS). It is language independent, so generating from it gives all equivalent sentences in both Swedish and English. Both the core of the languages and a subset of the identified VFDs are successfully implemented in BiTSE. For other VFDs tentative solutions are discussed.</p><p>MRS have previously been proposed as suitable for semantic transfer machine translation. I have shown that VFDs can naturally be handled by an interlingual design in many cases, minimizing the need of transfer.</p><p>The main contributions of this thesis are: an inventory of English and Swedish verb frames and verb frame divergences; the bilingual grammar BiTSE and showing that it is possible in many cases to use MRS as an interlingua in machine translation.</p>

The purpose of language models is in general to capture and to model regularities of language, thereby capturing morphological, syntactical and distributional properties of word sequences in a given language. They play an important role in many successful applications of Natural Language Processing, such as Automatic Speech Recognition, Machine Translation and Information Extraction. The most successful approaches to date are based on n-gram assumption and the adjustment of statistics from the training data by applying smoothing and back-off techniques, notably Kneser-Ney technique, introduced twenty years ago. In this way, language models predict a word based on its n-1 previous words. In spite of their prevalence, conventional n-gram based language models still suffer from several limitations that could be intuitively overcome by consulting human expert knowledge. One critical limitation is that, ignoring all linguistic properties, they treat each word as one discrete symbol with no relation with the others. Another point is that, even with a huge amount of data, the data sparsity issue always has an important impact, so the optimal value of n in the n-gram assumption is often 4 or 5 which is insufficient in practice. This kind of model is constructed based on the count of n-grams in training data. Therefore, the pertinence of these models is conditioned only on the characteristics of the training text (its quantity, its representation of the content in terms of theme, date). Recently, one of the most successful attempts that tries to directly learn word similarities is to use distributed word representations in language modeling, where distributionally words, which have semantic and syntactic similarities, are expected to be represented as neighbors in a continuous space. These representations and the associated objective function (the likelihood of the training data) are jointly learned using a multi-layer neural network architecture. In this way, word similarities are learned automatically. This approach has shown significant and consistent improvements when applied to automatic speech recognition and statistical machine translation tasks. A major difficulty with the continuous space neural network based approach remains the computational burden, which does not scale well to the massive corpora that are nowadays available. For this reason, the first contribution of this dissertation is the definition of a neural architecture based on a tree representation of the output vocabulary, namely Structured OUtput Layer (SOUL), which makes them well suited for large scale frameworks. The SOUL model combines the neural network approach with the class-based approach. It achieves significant improvements on both state-of-the-art large scale automatic speech recognition and statistical machine translations tasks. The second contribution is to provide several insightful analyses on their performances, their pros and cons, their induced word space representation. Finally, the third contribution is the successful adoption of the continuous space neural network into a machine translation framework. New translation models are proposed and reported to achieve significant improvements over state-of-the-art baseline systems.