Thesis (PhD (Electrical and Electronic Engineering))--University of Stellenbosch, 2010. / ENGLISH ABSTRACT: We propose a new methodology, based on proper scoring rules, for the evaluation of the goodness of pattern recognizers with probabilistic outputs. The recognizers of interest take an input, known to belong to one of a discrete set of classes, and output a calibrated likelihood for each class. This is a generalization of the traditional use of proper scoring rules to evaluate the goodness of probability distributions. A recognizer with outputs in well-calibrated probability distribution form can be applied to make cost-effective Bayes decisions over a range of applications, having di fferent cost functions. A recognizer with likelihood output can additionally be employed for a wide range of prior distributions for the to-be-recognized classes. We use automatic speaker recognition and automatic spoken language recognition as prototypes of this type of pattern recognizer. The traditional evaluation methods in these fields, as represented by the series of NIST Speaker and Language Recognition Evaluations, evaluate hard decisions made by the recognizers. This makes these recognizers cost-and-prior-dependent. The proposed methodology generalizes that of the NIST evaluations, allowing for the evaluation of recognizers which are intended to be usefully applied over a wide range of applications, having variable priors and costs. The proposal includes a family of evaluation criteria, where each member of the family is formed by a proper scoring rule. We emphasize two members of this family: (i) A non-strict scoring rule, directly representing error-rate at a given prior. (ii) The strict logarithmic scoring rule which represents information content, or which equivalently represents summarized error-rate, or expected cost, over a wide range of applications. We further show how to form a family of secondary evaluation criteria, which by contrasting with the primary criteria, form an analysis of the goodness of calibration of the recognizers likelihoods. Finally, we show how to use the logarithmic scoring rule as an objective function for the discriminative training of fusion and calibration of speaker and language recognizers. / AFRIKAANSE OPSOMMING: Ons wys hoe om die onsekerheid in die uittree van outomatiese sprekerherkenning- en taalherkenningstelsels voor te stel, te meet, te kalibreer en te optimeer. Dit maak die bestaande tegnologie akkurater, doeltre ender en meer algemeen toepasbaar.

Increasingly, government and educational institutions are turning to study abroad programs as one of the primary means of providing students with important cross-cultural and linguistic skills. However, because of constraints on time and financial resources most students participate in short term programs lasting approximately two months. Additionally, research regarding students’ linguistic and cultural growth in study abroad shows that some students gain far more from their participation than others. The purpose of this study was to examine the cultural and linguistic learning that takes place in short-term programs and discover some of the factors that predict students’ growth while abroad. This study examined the relationship between important factors such as students’ level of cultural sensitivity, students’ relationship with their host family, motivation to learn the language, the amount of time students’ spent interacting with native speakers, as well as students’ oral language skills during a seven-week study abroad program. Surveys were used to measure each of the variables except for students’ oral language skills. To measure changes in oral skills, the researcher created an innovative instrument in which native speakers rated clips of student speech in the target language from before and after their time abroad. As found in previous research, students varied greatly in the amount of progress made in oral language skills and cultural sensitivity while abroad. However, students generally demonstrated small, but significant improvements in their oral language skills, despite the brief nature of their program. Further, the instrument created to measure growth in oral language skills showed high reliability. Interestingly, students’ level of cultural sensitivity prior to going abroad predicted changes in oral language skills. These results provide support for students’ participation in short-term study abroad programs since students generally experience noticeable improvements in language skills. They also suggest that students who are more culturally sensitive may have an advantage in language learning during study abroad programs. These results could be helpful for administrators in determining who may benefit most from such programs and may suggest that helping students gain cultural sensitivity could also aid students’ language learning. / text

L’objectif de la présente étude est de cerner les difficultés rencontrées en lecture et en orthographe par les enfants issus de l’immigration de 2ème et 3ème génération qui on été comparés à leurs pairs autochtones inscrits dans les même classes. L’échantillon comprend 357 élèves scolarisés dans des écoles communales en discrimination positive de la région de Bruxelles Capitale. Un suivi longitudinal de la 3ème à la 5ème année primaire a également été mené. Les résultats nous ont permis d’évaluer les performances en langue orale et en langue écrite de ces élèves, et de mettre en évidence d’une part les mécanismes de lecture et d’écriture et d’autre part les facteurs susceptibles d’expliquer les retards sur le plan psycholinguistique. Deux programmes d’entraînements, au niveau phonologique et syntaxique ont été élaborés en vue de mesurer leur effet sur les différences de compétences. / The objective of this study is to determine the difficulties encountered in reading and spelling by children from emigrant families (2nd and 3rd generation) compared with their pars autochthons registered in the same classes. The sample includes 357 pupils in positive discrimination elementary schools from the area of Brussels City. A longitudinal follow-up from 3rd to 5th primary grade was also carried out. The results enabled us to evaluate the performances in spoken and written language of these pupils and to highlight the mechanisms of reading and writing and the factors which explain psycholinguistics delays. Two training programs, one phonologic, the other syntactic were elaborate in order to measure their effect on the differential of skills.

<p>This study investigates the use of formulaic language in spoken French produced by native and non-native speakers. It aims at describing the development of formulaic sequences in learners ranging from beginners to very advanced users. It draws on data from the InterFra corpus, which includes both formal and semi-formal learners. Four measures are used to characterize this development: extent of formulaic language used, category distribution, type / token ratio and frequency of types. </p><p>It has been shown that a user’s knowledge of formulaic sequences impacts heavily on language proficiency and idiomaticity. Because these sequences follow neither grammatical nor lexical rules, they constitute the last threshold for advanced L2 learners. In second language acquisition, the term formulaic sequence not only applies to strict idiomatic constructions, but it is also used to refer to sequences that appear to be acquired in a holistic manner during the first phases of acquisition. A categorization is therefore proposed that can account for native and non-native usage of formulaic sequences (prefabs). Five categories of prefabs are included: Lexical, Grammatical, Discourse, Situational and Idiosyncratic. </p><p>The extent of a learner’s use of formulaic language increases as the learner progresses, the largest amount found in the production of native speakers and very advanced learners. The learner’s distribution of categories moves towards native speaker distribution, albeit slowly. Situational and Idiosyncratic prefabs are found to characterize the early phases of acquisition, while Lexical prefabs are mastered later and are a major difficulty for L2 learners. Only very advanced learners who have spent considerable time in France produce the same proportion of Lexical prefabs as native speakers. Discourse prefabs constitute the most important category for all groups, including natives and non-natives. It can therefore be postulated that the main function of formulaic sequences in spoken French is that of discourse structuring and speech management. The development and use of formulaic language is explained within a framework of Frequency Effects. Coupled with other factors, frequency can account for why Lexical prefabs are hard to acquire and why formulaic sequences take a long time to master.</p> / The thesis is published and can be purchased by Peter Lang http://www.peterlang.com/index.cfm?vID=11369&vLang=E&vHR=1&vUR=1&vUUR=38

This thesis focuses on the problem of scalable optimization of dialogue behaviour in speech-based conversational systems using reinforcement learning. Most previous investigations in dialogue strategy learning have proposed flat reinforcement learning methods, which are more suitable for small-scale spoken dialogue systems. This research formulates the problem in terms of Semi-Markov Decision Processes (SMDPs), and proposes two hierarchical reinforcement learning methods to optimize sub-dialogues rather than full dialogues. The first method uses a hierarchy of SMDPs, where every SMDP ignores irrelevant state variables and actions in order to optimize a sub-dialogue. The second method extends the first one by constraining every SMDP in the hierarchy with prior expert knowledge. The latter method proposes a learning algorithm called 'HAM+HSMQ-Learning', which combines two existing algorithms in the literature of hierarchical reinforcement learning. Whilst the first method generates fully-learnt behaviour, the second one generates semi-learnt behaviour. In addition, this research proposes a heuristic dialogue simulation environment for automatic dialogue strategy learning. Experiments were performed on simulated and real environments based on a travel planning spoken dialogue system. Experimental results provided evidence to support the following claims: First, both methods scale well at the cost of near-optimal solutions, resulting in slightly longer dialogues than the optimal solutions. Second, dialogue strategies learnt with coherent user behaviour and conservative recognition error rates can outperform a reasonable hand-coded strategy. Third, semi-learnt dialogue behaviours are a better alternative (because of their higher overall performance) than hand-coded or fully-learnt dialogue behaviours. Last, hierarchical reinforcement learning dialogue agents are feasible and promising for the (semi) automatic design of adaptive behaviours in larger-scale spoken dialogue systems. This research makes the following contributions to spoken dialogue systems which learn their dialogue behaviour. First, the Semi-Markov Decision Process (SMDP) model was proposed to learn spoken dialogue strategies in a scalable way. Second, the concept of 'partially specified dialogue strategies' was proposed for integrating simultaneously hand-coded and learnt spoken dialogue behaviours into a single learning framework. Third, an evaluation with real users of hierarchical reinforcement learning dialogue agents was essential to validate their effectiveness in a realistic environment.

Le dialogue siège au cœur de l’activité de langage. Il doit donc être au cœur de l’enseignement des langues vivantes. Telle est la position centrale de la thèse, appliquée à l’enseignement du russe. Après le rappel des travaux sur le dialogue dans les traditions russe et française, la thèse analyse un important corpus enregistré de dialogues russes spontanés, et montre concrètement que son exploitation pédagogique passe nécessairement par une typologie des dialogues, à la fois linguistique et pragmatique. La thèse examine enfin, sous cet angle, une série de manuels existants et formule une série de propositions concrètes visant à construire une véritable didactique du dialogue. / Dialogue is at the very center of linguistic activity. It must therefore be at the center of the teaching of modern languages. This is the basic position of the present thesis, as applied to the teaching of Russian. After a survey of the existing works on dialogue in the Russian and French traditions, the thesis analyzes a large recorded corpus of spontaneous Russian dialogues and demonstrates concretely that in order to make use of it pedagogically it is necessary first to have a typology, both linguistic and pragmatic, of the dialogues. Finally the thesis examines, from this point of view, several existing handbooks, and formulates a certain number of concrete proposals whose aim is to achieve a real didactics of dialogue.

在人類溝通及語言使用行為中，口語和場景是構成人類跨感官理解歷程的兩項重要成分。究竟兩類資訊如何共同改變理解歷程仍待檢驗。本論文旨在探問四項研究問題：一，過去文獻對理解期間的視覺注意力運作提出兩類觀點。階層取徑主張口語優先並決定視覺表徵的處理，互動取徑則認為口語和視覺表徵可獨立影響視覺注意力。二，口語可促進指涉物體的凝視行為，然口語指涉效果是否受作業目標影響的本質仍不清楚。三，以複雜場景作為視覺情境，探討視覺複雜性和語義一致性表徵如何影響理解歷程。四，檢驗視覺刺激的預覽時間如何改變口語和場景表徵因素對理解歷程的影響。 　　本論文透過一系列視覺情境典範實驗探討以上研究問題。在每ㄧ項嘗試次中，參與者在聆聽中文語句期間同時觀看包含包含兩項物體的圖片：一為鑲嵌在一致（例如：原野）、不一致（例如：天空）和空白背景的口語指涉目標物體（例如：老虎），另一項則為口語未指涉且與背景一致的非目標物體（例如：禿鷹）。其次，四項實驗直交地操弄「作業目標」（「口語理解作業」或「場景理解作業」）和「預覽時間」（「一秒預覽」或「無預覽」）因素。 　　實驗結果發現：一，無論作業目標為何，所有實驗皆出現穩定的口語指涉效果。二，場景的視覺複雜性和語義一致性表徵不僅可獨立引導物體凝視行為，也可和口語共同決定理解期間的視覺注意力運作。三，作業目標對口語指涉效果及場景一致性效果產生差異化的調節作用。四，預覽時間有效促進口語理解作業的口語指涉效果，場景理解作業則不受影響。 　　整體而言，本論文的實驗證據支持互動取徑觀點。換言之，在跨感官理解的過程中，人類認知運作可透過協調語言、視覺和記憶等次系統，快速整合口語和場景所提供的物理和語義表徵，並依據當下情境動態地改變人類對外在世界的感官經驗。 / In human communication and language use, both speech and scene constitute the　cross-modal comprehension process. However, how these two elements combine to　affect human comprehension process has not yet been fully resolved. Four　research　questions will be examined. First, two approaches can account for the comprehension process: the hierarchical approach asserts speech plays the main part whereas the visual feature has only a supporting role, while the interactive approach states that both speech and visual feature combine to determine the comprehension process. Second, despite the speech can cause the spoken reference effect on having more fixations on its visual referent, the nature of this effect is still unclear. Third, most past studies adopted simple object array as visual context, little is known about the impact of real world scenes on the comprehension process. Fourth, whether the preview time could alter the influence of speech and scene on comprehension will be tested. A series of visual world paradigm experiments were conducted. Factors of task demand (speech comprehension vs. scene comprehension) and preview time (1-second vs. none) were orthogonally manipulated in four experiments. In each trial, participants listened to a spoken sentence in Chinese while viewing a picture with two critical objects: one is the mentioned target object (e.g., tiger), which was embedded in either a consistent, inconsistent or blank background; the other is an unmentioned non-target object (e.g., eagle) that was always consistent with its background. Several findings were found. First, the reliable spoken reference effect were shown regardless of the task demand was given. Second, the visual complexity and scene consistency not only can individually guide fixations on objects, but can work together with the speech to determine the visual attention during comprehension. Third, task demand could differently modulate the spoken reference and scene consistency effect, respectively. Fourth, preview time significantly enhances the spoken reference effect in the speech comprehension task, whereas no impact was observed in the scene comprehension task. These evidence supported the view of interactive approach. In conclusion, human’s different cognitive systems, including language, vision and memory, can interact with each other and cause the moment to moment experience of how we understand the complex world around us.

International students are often under considerable pressure from language barriers, culture shock, social isolation and lack of social support in American universities. Those stressors often discourage international students as ESL learners from practicing English with native students on campus. Based on Krashen & Terrell’s subconscious acquisition and conscious learning hypothesis, Vygotsky’s Zone of Proximal Development , Cummins’ Basic Interpersonal Communication System (BICS) and Cognitive Academic Language Proficiency (CALP), Horwitz’s language anxiety and Oxford’s indirect learning strategies , this study explores how international students and American students, respectively, perceive English Corner and whether English Corner could be an applicable out-of-class learning environment for international students to practice English and socialize with American students on American campuses. English Corner refers to regular meetings that English learners in Mainland China voluntarily organize in public places to practice spoken English. A survey was conducted on language learning strategies, socialization, acculturation, autonomy and English Corner among international students and native students at the University of North Texas. The questionnaires were adapted from Oxford and Nyikos’ study as to what variables affect choice of language learning strategies, Iheanacho’s study as to how international students use the Morris Library at the University of Delaware and their perception of library services and programs, and Battle’s study as to how information literacy instruction affects library anxiety among international students. The findings of this study may help American universities realize the importance of English Corner as one optimal intervention program for international students and American students. The support for English Corner may help international students improve their English learning, alleviate their language anxiety and create more opportunities for international students and native students to socialize with each other.

Brown (1973) a proposé la « longueur moyenne des énoncés » (LME) comme indice standard du développement langagier. La LME se calcule selon le nombre moyen de morphèmes dans 100 énoncés de parole spontanée. L’hypothèse sous-jacente à cet indice est que la complexité syntaxique croît avec le nombre de morphèmes dans les énoncés. Selon Brown, l’indice permet d’estimer le développement d’une « compétence grammaticale » jusqu’à environ quatre morphèmes. Certains auteurs ont toutefois critiqué le manque de fiabilité de la LME et la limite de quatre morphèmes. Des rapports démontrent des variations de la LME avec l’âge, ce qui suggère que des facteurs comme la croissance des capacités respiratoires peuvent influencer l’indice de Brown. La présente étude fait état de ces problèmes et examine comment la LME et certaines mesures de diversité lexicale varient selon le développement des capacités respiratoires. On a calculé la LME et la diversité lexicale dans la parole spontanée de 50 locuteurs mâles âgés de 5 à 27 ans. On a également mesuré, au moyen d’un pneumotachographe, la capacité vitale (CV) des locuteurs. Les résultats démontrent que la LME et des mesures de diversité lexicale corrèlent fortement avec la croissance de la CV. Ainsi, la croissance des fonctions respiratoires contraint le développement morphosyntaxique et lexical. Notre discussion fait valoir la nécessité de réévaluer l’indice de la LME et la conception « linguistique » du développement langagier comme une compétence mentale qui émerge séparément de la croissance des structures de performance. / Brown (1973) proposed the « mean length of utterance » (MLU) as a standard index of language development. MLU is calculated by counting the mean number of morphemes in 100 utterances of spontaneous speech. The underlying hypothesis of this index is that syntactic complexity rises with the number of morphemes in utterances. According to Brown, MLU indexes the development of a « grammatical competence » up to about four morphemes. However, authors have criticized the lack of reliability of the MLU and the limit of four morphemes. Reported variations in MLU with age also suggest that factors such as the growth of respiratory capacities can influence Brown’s index. The present study reviews those problems and examines how the MLU and certain measures of lexical diversity vary according to the development of respiratory capacities. We calculated the MLU and the lexical diversity in the spontaneous speech of 50 male speakers aged 5 to 27 years. We also measured, with a pneumotachograph, the vital capacity (VC) of the speakers. The results show that MLU and measures of lexical diversity strongly correlate with the growth of VC. Thus, the growth of respiratory functions constrains morphosyntactic and lexical development. Our discussion focuses on the necessity to revaluate the MLU index and a « linguistic » conception of language development as a mental competence emerging separately from the growth of performance structures.

La généralisation de l’usage des systèmes de dialogue homme-machine accroît la nécessité du développement rapide des différents composants de ces systèmes. Les systèmes de dialogue peuvent être conçus pour différents domaines d’application et dans des langues différentes. La nécessité d’une production rapide pour de nouvelles langues reste un problème ouvert et crucial auquel il est nécessaire d’apporter des solutions efficaces.Nos travaux s’intéressent particulièrement au module de compréhension de la parole et proposent des approches pour la portabilité rapide peu coûteuse de ce module.Les méthodes statistiques ont montré de bonnes performances pour concevoir les modules de compréhension de la parole pour l’étiquetage sémantique de tours de dialogue.Cependant ces méthodes nécessitent de larges corpus pour être apprises. La collecte de ces corpus est aussi coûteuse en temps et en expertise humaine.Dans cette thèse, nous proposons plusieurs approches pour porter un système de compréhension d’une langue vers une autre en utilisant les techniques de la traduction automatique. Les premiers travaux consistent à appliquer la traduction automatique à plusieurs niveaux du processus de portabilité du système de compréhension afin de réduire le coût lié à production de nouvelles données d’apprentissage. Les résultats expérimentaux montrent que l’utilisation de la traduction automatique permet d’obtenir des systèmes performant avec un minimum de contribution humaine.Cette thèse traite donc à la fois de la traduction automatique et de la compréhension de la parole. Nous avons effectué une comparaison approfondie entre les méthodes utilisées pour chacune des tâches et nous avons proposé un décodage conjoint basé sur une méthode discriminante qui à la fois traduit une phrase et lui attribue ses étiquettes sémantiques. Ce décodage est obtenu par une approche à base de graphe qui permet de composer un graphe de traduction avec un graphe de compréhension. Cette représentation peut être généralisée pour permettre des transmissions d’informations riches entre les composants du système de dialogue / The generalisation of human-machine dialogue system increases the need for a rapid development of the various components of these systems. Dialogue systems can be designed for different applications and in different languages. The need for a fast production of systems for new languages ​​is still an open and crucial issue which requires effective solutions. Our work is particularly interested in speech understanding module and propose approaches for language portability of this module. The statistical methods showed good performance to design modules for speech understanding. However, these methods require large corpora to be trained. The collection of these corpora is expensive in time and human expertise. In this thesis, we propose several approaches to port an understanding system from one language to another using machine translation techniques. The experimental results show that the use of machine translation allows to produce efficient systems with minimal human effort. This thesis addresses both machine translation and speech understanding domain. We conducted a comparison between the methods used for each task and we have proposed a joint decoding between translation and understanding based on a discriminant method. This decoding is achieved by a graph-based approach which allows to compose a translation graph with an understanding graph. This representation can be generalized to allow a rich transmission of information between the components of the dialogue system