La traduction automatique statistique est un domaine très en demande et où les machines sont encore loin de produire des résultats de qualité humaine. La principale méthode utilisée est une traduction linéaire segment par segment d'une phrase, ce qui empêche de changer des parties de la phrase déjà traduites. La recherche pour ce mémoire se base sur l'approche utilisée dans Langlais, Patry et Gotti 2007, qui tente de corriger une traduction complétée en modifiant des segments suivant une fonction à optimiser. Dans un premier temps, l'exploration de nouveaux traits comme un modèle de langue inverse et un modèle de collocation amène une nouvelle dimension à la fonction à optimiser. Dans un second temps, l'utilisation de différentes métaheuristiques, comme les algorithmes gloutons et gloutons randomisés permet l'exploration plus en profondeur de l'espace de recherche et permet une plus grande amélioration de la fonction objectif. / Statistical Machine Translation is a field ingreat demand and where machines are still far from producing human-level results.The main method used is a segment by segment linear translation of a sentence, which prevents modification of already translated parts of the sentence. Research for this memoir is based on an approach used by Langlais, Patry and Gotti 2007, which tries to correct a completed translation by modifying segments following a function which needs to be optimized. As a first step, exploration of new traits such as an inverted language model and a collocation model brings a new dimension to the optimization function. As a second step, use of different metaheuristics, such as the greedy and randomized greedy algorithms, allows greater depth while exploring the search space and allows a greater improvement of the objective function.

This thesis investigates how the sub-structure of words can be accounted for in probabilistic models of language. Such models play an important role in natural language processing tasks such as translation or speech recognition, but often rely on the simplistic assumption that words are opaque symbols. This assumption does not fit morphologically complex language well, where words can have rich internal structure and sub-word elements are shared across distinct word forms. Our approach is to encode basic notions of morphology into the assumptions of three different types of language models, with the intention that leveraging shared sub-word structure can improve model performance and help overcome data sparsity that arises from morphological processes. In the context of n-gram language modelling, we formulate a new Bayesian model that relies on the decomposition of compound words to attain better smoothing, and we develop a new distributed language model that learns vector representations of morphemes and leverages them to link together morphologically related words. In both cases, we show that accounting for word sub-structure improves the models' intrinsic performance and provides benefits when applied to other tasks, including machine translation. We then shift the focus beyond the modelling of word sequences and consider models that automatically learn what the sub-word elements of a given language are, given an unannotated list of words. We formulate a novel model that can learn discontiguous morphemes in addition to the more conventional contiguous morphemes that most previous models are limited to. This approach is demonstrated on Semitic languages, and we find that modelling discontiguous sub-word structures leads to improvements in the task of segmenting words into their contiguous morphemes.

The purpose of language models is in general to capture and to model regularities of language, thereby capturing morphological, syntactical and distributional properties of word sequences in a given language. They play an important role in many successful applications of Natural Language Processing, such as Automatic Speech Recognition, Machine Translation and Information Extraction. The most successful approaches to date are based on n-gram assumption and the adjustment of statistics from the training data by applying smoothing and back-off techniques, notably Kneser-Ney technique, introduced twenty years ago. In this way, language models predict a word based on its n-1 previous words. In spite of their prevalence, conventional n-gram based language models still suffer from several limitations that could be intuitively overcome by consulting human expert knowledge. One critical limitation is that, ignoring all linguistic properties, they treat each word as one discrete symbol with no relation with the others. Another point is that, even with a huge amount of data, the data sparsity issue always has an important impact, so the optimal value of n in the n-gram assumption is often 4 or 5 which is insufficient in practice. This kind of model is constructed based on the count of n-grams in training data. Therefore, the pertinence of these models is conditioned only on the characteristics of the training text (its quantity, its representation of the content in terms of theme, date). Recently, one of the most successful attempts that tries to directly learn word similarities is to use distributed word representations in language modeling, where distributionally words, which have semantic and syntactic similarities, are expected to be represented as neighbors in a continuous space. These representations and the associated objective function (the likelihood of the training data) are jointly learned using a multi-layer neural network architecture. In this way, word similarities are learned automatically. This approach has shown significant and consistent improvements when applied to automatic speech recognition and statistical machine translation tasks. A major difficulty with the continuous space neural network based approach remains the computational burden, which does not scale well to the massive corpora that are nowadays available. For this reason, the first contribution of this dissertation is the definition of a neural architecture based on a tree representation of the output vocabulary, namely Structured OUtput Layer (SOUL), which makes them well suited for large scale frameworks. The SOUL model combines the neural network approach with the class-based approach. It achieves significant improvements on both state-of-the-art large scale automatic speech recognition and statistical machine translations tasks. The second contribution is to provide several insightful analyses on their performances, their pros and cons, their induced word space representation. Finally, the third contribution is the successful adoption of the continuous space neural network into a machine translation framework. New translation models are proposed and reported to achieve significant improvements over state-of-the-art baseline systems.

Speech recognition in Turkish Language is a challenging problem in several perspectives. Most of the challenges are related to the morphological structure of the language. Since Turkish is an agglutinative language, it is possible to generate many words from a single stem by using suffixes. This characteristic of the language increases the out-of-vocabulary (OOV) words, which degrade the performance of a speech recognizer dramatically. Also, Turkish language allows words to be ordered in a free manner, which makes it difficult to generate robust language models. In this thesis, the existing models and approaches which address the problem of Turkish LVCSR (Large Vocabulary Continuous Speech Recognition) are explored. Different recognition units (words, morphs, stem and endings) are used in generating the n-gram language models. 3-gram and 4-gram language models are generated with respect to the recognition unit. Since the solution domain of speech recognition is involved with machine learning, the performance of the recognizer depends on the sufficiency of the audio data used in acoustic model training. However, it is difficult to obtain rich audio corpora for the Turkish language. In this thesis, existing approaches are used to solve the problem of Turkish LVCSR by using a limited audio corpus. We also proposed several data selection approaches in order to improve the robustness of the acoustic model.

Avec la croissance explosive de la numérisation du patrimoine culturel , de nombreuses patrimoine culturel institutions ont été la conversion des objets physiques du patrimoine culturel dans la représentation numérique ou représentation descriptive . Toutefois , la conversion a donné lieu à plusieurs questions telles que : 1 ) les documents sont de nature descriptive , 2 ) l'ambiguïté et de la brièveté des documents , 3 ) le vocabulaire spécifique est utilisé dans les documents , et 4 ), il existe également des variations dans les termes utilisés dans le document . En outre, l'utilisation de mots-clés inexactes également entraîné problème de requête court . La plupart du temps , les problèmes sont causés par la faute agrégée en annotant les documents alors que le problème de requête court est causé par l'utilisateur naïf qui a peu de connaissances et d'expérience dans le domaine du patrimoine culturel . Dans cette recherche, l'objectif principal est de modéliser le système d' accès à l'information pour surmonter partiellement les questions soulevées par le processus de documentation et le fond des utilisateurs du patrimoine culturel numérique . Par conséquent , trois types d'outils d'accès aux informations sont introduites et établies à savoir l'information système de recherche , la recherche de contexte , et jeu mobile sur le patrimoine culturel qui permettent à l' utilisateur d'accéder , d'apprendre et d'explorer les informations sur le patrimoine culturel . Fondamentalement , l'idée principale d'information système de recherche et contexte de recherche est d'intégrer la relation de lien entre les termes dans le modèle de la langue par l'extension de Dirichlet lissage pour résoudre les problèmes qui se posent à la fois le processus de documentation et de fond des utilisateurs . En outre, un modèle de préférence est présenté sur la base de la théorie de la charge d'un condensateur de quantifier le contexte cognitif basé sur le temps et les intégrer dans la longue Dirichlet lissage . En outre, un jeu mobile est introduite par l'intégration des éléments des jeux de monopole et chasse au trésor pour atténuer les problèmes découlant de l'arrière-plan des utilisateurs en particulier leur comportement décontracté . Les premier et deuxième approches ont été testées sur le patrimoine culturel dans CLEF ( chic) ​​collection qui se compose de questions et de courts documents . Les résultats montrent que l'approche est efficace et donne une meilleure précision lors de la récupération . Enfin , une enquête a été menée pour étudier la troisième approche , et le résultat donne à penser que le jeu est en mesure d'aider les participants à explorer et apprendre les informations sur le patrimoine culturel . En outre, les participants ont également estimé qu'une recherche d'information outil qui est intégré avec le jeu peut fournir plus d'informations à l'utilisateur d'une manière plus pratique tout en jouant le jeu et en visitant les sites du patrimoine dans le match. En conclusion , les résultats montrent que les solutions proposées sont en mesure de résoudre les problèmes posés par le processus de documentation et le fond des utilisateurs du patrimoine culturel numérique . / With the explosive growth of digitization in cultural heritage, many cultural heritage institu- tions have been converting physical objects of cultural heritage into digital representation or descriptive representation. However, the conversion resulted in several issues such as: 1) the documents are descriptive in nature, 2) ambiguity and brevity of the documents, 3) dedicated vocabulary is used in the documents, and 4) there is also variation in the terms used in the doc- ument. Besides, the usage of inaccurate keywords also resulted in short query problem. Most of the time, the issues are caused by the aggregated fault in annotating the documents while the short query problem is caused by naive user who has little prior knowledge and experience in cultural heritage domain. In this research, the main aim is to model information access system to overcome partially the issues arising from the documentation process and the background of the users of digital cultural heritage. Therefore, three types of information access tool are introduced and established namely information retrieval system, context search, and mobile game on cultural heritage that allow the user to access, learn, and explore the information on cultural heritage. Basically, the main idea for information retrieval system and context search is to incorporate the link relationship between terms into the Language Model by extending of Dirichlet Smoothing to solve the problems arising from both the documentation process and background of the users. In addition, a Preference Model is introduced based on the Theory of Charging a Capacitor to quantify the cognitive context based on time and integrate into the extended Dirichlet Smoothing. Besides, a mobile game is introduced by integrating the ele- ments of the games of monopoly and treasure hunt to mitigate the problems arising from the background of the users especially their casual behavior. The first and second approaches were tested on the Cultural Heritage in CLEF (CHiC) collection that consists of short queries and documents. The results show that the approach is effective and yields better accuracy during the retrieval. Finally, a survey was carried out to investigate the third approach, and the result suggests that the game is able to help the participants to explore and learn the information on cultural heritage. In addition, the participants also felt that an information seeking tool that is integrated with the game can provide more information to the user in a more convenient manner while playing the game and visiting the heritage sites in the game. In conclusion, the results show that the proposed solutions are able to solve the problems arising from the documentation process and the background of the users of digital cultural heritage.

Les trois piliers d’un système de reconnaissance automatique de la parole sont le lexique,le modèle de langage et le modèle acoustique. Le lexique fournit l’ensemble des mots qu’il est possible de transcrire, associés à leur prononciation. Le modèle acoustique donne une indication sur la manière dont sont réalisés les unités acoustiques et le modèle de langage apporte la connaissance de la manière dont les mots s’enchaînent.Dans les systèmes de reconnaissance automatique de la parole markoviens, les modèles acoustiques et linguistiques sont de nature statistique. Leur estimation nécessite de gros volumes de données sélectionnées, normalisées et annotées.A l’heure actuelle, les données disponibles sur le Web constituent de loin le plus gros corpus textuel disponible pour les langues française et anglaise. Ces données peuvent potentiellement servir à la construction du lexique et à l’estimation et l’adaptation du modèle de langage. Le travail présenté ici consiste à proposer de nouvelles approches permettant de tirer parti de cette ressource.Ce document est organisé en deux parties. La première traite de l’utilisation des données présentes sur le Web pour mettre à jour dynamiquement le lexique du moteur de reconnaissance automatique de la parole. L’approche proposée consiste à augmenter dynamiquement et localement le lexique du moteur de reconnaissance automatique de la parole lorsque des mots inconnus apparaissent dans le flux de parole. Les nouveaux mots sont extraits du Web grâce à la formulation automatique de requêtes soumises à un moteur de recherche. La phonétisation de ces mots est obtenue grâce à un phonétiseur automatique.La seconde partie présente une nouvelle manière de considérer l’information que représente le Web et des éléments de la théorie des possibilités sont utilisés pour la modéliser. Un modèle de langage possibiliste est alors proposé. Il fournit une estimation de la possibilité d’une séquence de mots à partir de connaissances relatives à ’existence de séquences de mots sur le Web. Un modèle probabiliste Web reposant sur le compte de documents fourni par un moteur de recherche Web est également présenté. Plusieurs approches permettant de combiner ces modèles avec des modèles probabilistes classiques estimés sur corpus sont proposées. Les résultats montrent que combiner les modèles probabilistes et possibilistes donne de meilleurs résultats que es modèles probabilistes classiques. De plus, les modèles estimés à partir des données Web donnent de meilleurs résultats que ceux estimés sur corpus. / The three pillars of an automatic speech recognition system are the lexicon, the languagemodel and the acoustic model. The lexicon provides all the words that can betranscribed, associated with their pronunciation. The acoustic model provides an indicationof how the phone units are pronounced, and the language model brings theknowledge of how words are linked. In modern automatic speech recognition systems,the acoustic and language models are statistical. Their estimation requires large volumesof data selected, standardized and annotated.At present, the Web is by far the largest textual corpus available for English andFrench languages. The data it holds can potentially be used to build the vocabularyand the estimation and adaptation of language model. The work presented here is topropose new approaches to take advantage of this resource in the context of languagemodeling.The document is organized into two parts. The first deals with the use of the Webdata to dynamically update the lexicon of the automatic speech recognition system.The proposed approach consists on increasing dynamically and locally the lexicon onlywhen unknown words appear in the speech. New words are extracted from the Webthrough the formulation of queries submitted toWeb search engines. The phonetizationof the words is obtained by an automatic grapheme-to-phoneme transcriber.The second part of the document presents a new way of handling the informationcontained on the Web by relying on possibility theory concepts. A Web-based possibilisticlanguage model is proposed. It provides an estition of the possibility of a wordsequence from knowledge of the existence of its sub-sequences on the Web. A probabilisticWeb-based language model is also proposed. It relies on Web document countsto estimate n-gram probabilities. Several approaches for combining these models withclassical models are proposed. The results show that combining probabilistic and possibilisticmodels gives better results than classical probabilistic models alone. In addition,the models estimated from Web data perform better than those estimated on corpus.

We live in a time where software is used everywhere. It is used even for creating other software by helping developers with writing or generating new code. To do this properly, metrics to measure software quality are being used to evaluate the final code. However, they are sometimes too costly to compute, or simply don't have the expected effect. Therefore, new and better ways of software evaluation are needed. In this research, we are investigating the usage of the statistical approaches used commonly in the natural language processing (NLP) area. In order to introduce and evaluate new metrics, a Java N-gram language model is created from a large Java language code corpus. Naturalness, a method-level metric, is introduced and calculated for chosen projects. The correlation with well-known software complexity metrics are calculated and discussed. The results, however, show that the metric, in the form that we have defined it, is not suitable for software complexity evaluation since it is highly correlated with a well-known metric (token count), which is much easier to compute. Different definition of the metric is suggested, which could be a target of future study and research.

This thesis deals with creating of unique shopping lists management application and we used the newest startup techniques and principles during its development. All our hypotheses were tested by early adopters and the new courses of development were based on their feedback. The result of this thesis is a mobile application for Android operating system which is placed on Google Play market and two its components which will extend the application on the market. The main component is inteligent sorting of items on the shopping list by the supermarket model, which is created from last purchases in this supermarket. The second one is web application enabling us send new shopping lists to the mobile device.

Most current automatic speaker recognition system extract speaker-depend features by looking at short-term spectral information. This approach ignores long-term information. I explored approach that use the fundamental frequency and energy trajectories for each speaker. This approach models prosody dynamics on single fonemes or syllables. It is known from literature that prosodic systems do not work as well the acoustic one but it improve the system when fusing. I verified this assumption by fusing my results with state of the art acoustic system from BUT. Data from standard evaluation campaigns organized by National Institute of Standarts and Technology are used for all experiments.

Statistické jazykové modely jsou důležitou součástí mnoha úspěšných aplikací, mezi něž patří například automatické rozpoznávání řeči a strojový překlad (příkladem je známá aplikace Google Translate). Tradiční techniky pro odhad těchto modelů jsou založeny na tzv. N-gramech. Navzdory známým nedostatkům těchto technik a obrovskému úsilí výzkumných skupin napříč mnoha oblastmi (rozpoznávání řeči, automatický překlad, neuroscience, umělá inteligence, zpracování přirozeného jazyka, komprese dat, psychologie atd.), N-gramy v podstatě zůstaly nejúspěšnější technikou. Cílem této práce je prezentace několika architektur jazykových modelůzaložených na neuronových sítích. Ačkoliv jsou tyto modely výpočetně náročnější než N-gramové modely, s technikami vyvinutými v této práci je možné jejich efektivní použití v reálných aplikacích. Dosažené snížení počtu chyb při rozpoznávání řeči oproti nejlepším N-gramovým modelům dosahuje 20%. Model založený na rekurentní neurovové síti dosahuje nejlepších publikovaných výsledků na velmi známé datové sadě (Penn Treebank).