Master of Science / Department of Psychology / Richard J. Harris / Memory processes have undergone extensive investigation using various experimental methods. While working memory studies have profoundly influenced the development of new cognitive theories, including the Dual-Processing Theory (Mayer & Moreno, 1998), limited research has investigated the effects of subtitled messages on multimedia comprehension. Previous eye movement research has investigated the ability of observers to attend to the multiple sensory inputs associated with multimedia viewing (D’Ydewalle, Van Rensbergen & Pollet, 1987; D’Ydewalle & De Bruycker, 2007); however, eye movement research only scratches the surface of cognitive abilities associated with multimedia learning. In order to satisfy the need for more subtitling research two studies were performed to investigate the effects of subtitled movies on the comprehension of movie content. Both investigations involved the presentation of 10-minute movie clips from A Few Good Men and See No Evil, Hear No Evil. Participants completed three types of multiple-choice recognition questions for each movie, including: pictorial-only questions, verbal-only questions and combined-information questions. Experiment 1 was designed to investigate the difference between levels of comprehension, when verbal information was presented only in the participants’ native language (i.e. English soundtrack and/or subtitles). Results of Experiment 1 indicate that participants performed significantly better on verbal-only and combined-information questions when their native language was present in the subtitles as opposed to the soundtrack. These findings confirm previous findings that reading verbal information in subtitles is more efficient than listening to the soundtrack. Comparison of performance on the pictorial-only questions across presentation formats in Experiment 1 showed participants in the English soundtrack with no subtitles condition performed significantly better than all other conditions. Although Experiment 1 provides a basic understanding of how native language soundtracks and subtitles influence comprehension of movies, subtitled media are primarily used when viewing a movie with verbal information from a foreign language. Experiment 2 built on the results of Experiment 1 by incorporating an unfamiliar language (i.e. French). The question sets used in Experiment 1 were also used in Experiment 2; however, two French vocabulary tests were also used in Experiment 2 to measure incidental foreign language acquisition. Consistent with the results of Experiment 1, participants performed significantly better on verbal-only and combined-information questions when their native language was in the subtitles. This finding extended the conclusion that native language verbal information presented visually (i.e. subtitles) yields better performance on questions requiring verbal cues than native language verbal information presented orally (i.e. soundtrack) to foreign language material. Comparison of performance on the pictorial-only questions across presentation formats in Experiment 2 showed no significant differences. Comparison across the two experiments reflected a distraction effect associated with the presence of a foreign language. Performance on the French vocabulary tests was very poor across all conditions and yielded no significant differences, suggesting that the tasks may have been too difficult.

No description available.

L'objectif de ce travail est d'étudier la compréhension et la mémorisation de plusieurs aspects des films sous-titrés, en relation avec le niveau de maîtrise des langues des participants. L'acquisition de vocabulaire d'une autre langue a également été évaluée, selon la combinaison des langues à l'écran (dialogues et sous-titres). Trois séries d'expériences sont présentées afin d'examiner le rôle des sous-titres dans chacun de ces mécanismes. Le traitement de trois types d'information des films a été pris en compte (éléments visuels, dialogues et situation). Les résultats indiquent un rôle facilitateur des sous-titres lorsqu'ils sont nécessaires (faible maîtrise de la langue du film), ou perturbateur lorsqu'ils sont superflus. Ensuite, l'effet des sous-titres sur la mémorisation d'un film a été estimé, selon la version et le niveau de saillance des éléments testés. Les résultats indiquent un effet du type de sous-titrage sur la compréhension et la restitution des informations, en fonction de leur caractère central ou périphérique par rapport à l'intrigue. Enfin, les possibilités d'acquisition de vocabulaire sont envisagées, selon le type de sous-titres, le nombre de visionnages d'un film et une consigne dirigeant l'attention vers les dialogues. La version inversée (dialogues en L1, soustitres en L2) mène à la meilleure restitution du vocabulaire, principalement après le second visionnage de la séquence. La consigne a également un effet positif sur la restitution des mots. Les résultats sont interprétés dans le cadre des théories de l’attention et de la charge cognitive, ainsi qu'à l'aide de divers modèles de l'apprentissage multimédia. / The aim of this work is to study how different aspects of subtitled films are understood and memorised, in relation to the viewers' knowledge of the on-screen languages. Vocabulary acquisition is also evaluated, depending on the order in which the languages are presented on the screen (dialogues and subtitles). Three series of experiments are presented in order toexamine the role of subtitles in each of these three mechanisms. The processing of three dimensions of a film (visual, dialogue and situation) is evaluated, depending on the languages on the screen and the participants' knowledge of them. Results indicate a global facilitating effect of subtitles when they are necessary, as opposed to a global negative effect when they are not needed. Next, the effect of subtitles on film memorisation is examined, depending on the type of subtitling and the degree of salience of the information tested. An effect of the type of subtitles is found on comprehension and data retrieval, especially for elements that are considered most central to the plot. Finally, vocabulary acquisition possibilities are measured, depending on the type of subtitles, on the number of viewings of the film and on viewing instructions directing attention to the dialogues. The reversed version (dialogues in L1, subtitles in L2) lead to the best restitution level, particularly after the second viewing of the film. The viewing instructions also have an effect on word restitution. The results are interpreted in the light of attention and cognitive load theories, within the framework of various multimedia learning models.

nÃo hà / Nas Ãltimas dÃcadas, o ensino-aprendizagem de lÃngua estrangeira (LE) tem se beneficiado dos estudos sobre o uso de material audiovisual legendado, embora haja uma carÃncia no que se refere a essas pesquisas com participantes em nÃvel inicial de aprendizagem, se comparadas aos estudos com participantes em nÃvel intermediÃrio e avanÃado (MATIELO; DâELY; BARRETA, 2015). Partindo dessa necessidade, nossa pesquisa (Parecer: 1.322.500 â CEP/UFC) investiga a efetividade do uso de filmes legendados combinados à traduÃÃo de legendas intralinguais como forma de ampliar o repertÃrio vocabular de estudantes de lÃngua inglesa (LI) do Ensino MÃdio. Para isso, aplicam-se os pressupostos da teoria funcionalista da TraduÃÃo encontrados em Reiss (2004), em Vermeer (2004), principalmente em Nord (2000; 2001; 2005a; 2005b; 2006a; 2006b; 2009; 2010a; 2010b; 2010c; 2012) e seu projeto de traduÃÃo que parte de uma anÃlise textual dos elementos intratextuais e extratextuais, comuns aos diversos tipos de texto, como forma de promover um entendimento abrangente do texto-fonte (TF). Faz-se, aqui, tambÃm, uma discussÃo sobre as caracterÃsticas da linguagem fÃlmica (MARTIN 2005; ZABALBEASCOA 2008; AUMONT 2006; 2009; GAUDREAULT; JOST 2009) e suas implicaÃÃes no campo da traduÃÃo audiovisual (TAV), com Ãnfase na legendagem (ORERO 2004; DÃAZ CINTAS 2004; 2005; 2008; DÃAZ CINTAS; ANDERMAN 2009; GAMBIER 2006; 2009; DÃAZ CINTAS; REMAEL 2014), percebendo as contribuiÃÃes que os materiais audiovisuais legendados podem trazer ao ensino-aprendizagem de uma LE. Por meio da aplicaÃÃo de uma sequÃncia didÃtica (SD) em sala de aula do Ensino MÃdio, verificou-se que houve uma aquisiÃÃo vocabular em LI que ultrapassou 50% da carÃncia lexical demonstrada pelos participantes no prÃ-teste, o que representa um ganho considerÃvel para os estudantes dessa LE, tornando essa ferramenta didÃtica, SD, da forma como foi aplicada, profÃcua. Podendo, assim, ser utilizada em outros contextos de sala de aula de LE como mais uma opÃÃo de ensino. / In recent decades, foreign languages (FL) teaching and learning has benefited from studies on the use of subtitled audiovisual material, although there is a shortage of work concerning research with participants in initial level of learning, compared to studies involving participants at intermediate and advanced level (MATIELO; DâELY; BARRETA, 2015). Based on this need, our research (Parecer: 1.322.500 â CEP/UFC) examines the effectiveness of the use of subtitled films combined to the translation of intralingual subtitles as a way to broaden the vocabulary repertoire of English language (EL) students from high school. To this end, we apply the assumptions of the functionalist theory of Translation found in Reiss (2004), in Vermeer (2004), and mainly in Nord (2000; 2001; 2005a; 2005b; 2006a; 2006b; 2009; 2010a; 2010b; 2010c; 2012) and her translation project which starts from the textual analysis of intratextual and extratextual elements, shared by various types of text, as a form of promoting a comprehensive understanding of the source text (SC). We also discuss the characteristics of the filmic language (MARTIN 2005; ZABALBEASCOA 2008; AUMONT 2006; 2009; GAUDREAULT; JOST 2009) and its implications for the field of audiovisual translation (AVT), focusing on subtitling (ORERO 2004; DÃAZ CINTAS 2004; 2005; 2008; DÃAZ CINTAS; ANDERMAN 2009; GAMBIER 2006; 2009; DÃAZ CINTAS; REMAEL 2014), as well as considering the contributions that the subtitled audiovisual material may bring to the FL teaching and learning. Through the application of a didactic sequence (DS) with a high school class, it was found that there was a vocabulary acquisition in the EL which surpassed 50% of the lexical deficit that was identified in the participantsâ pretests, which represents a considerable gain for this FL students, making this pedagogical tool, DS, the way it was applied, productive. Thus, it may be used in other FL class contexts as another teaching option.

V dnešnej dobe systémy rozpoznávania reči s veľkým slovníkom dosahujú pomerne vysoké presnosti. Za ich výsledkami však často stoja desiatky ba až stovky hodín manuálne oanotovaných trénovacích dát. Takéto dáta sú často bežne nedostupné alebo pre požadovaný jazyk vôbec neexistujú. Možným riešením je použitie bežne dostupných no menej kvalitných audiovizuálnych dát. Táto práca sa zaoberá technikou zpracovania práve takýchto dát a ich použitím pre trénovanie akustických modelov. Ďalej táto práca pojednáva o možnom využití týchto dát pre kontinuálne vylepšovanie modelov, kedže tieto dáta sú prakticky nevyčerpateľné. Pre tieto účely bol v rámci práce navrhnutý nový prístup pre výber dát.