Analysen syftar till att studera hur Ikeakatalogens modelläsare konstrueras i förhållande till familjen som social praktik idag genom att studera hur diskursen har förändrats över tid. Nedan följer en diakron multimodal textanalys av fem svenska Ikeakataloger mellan 1973 och 2013. Analysen tar i första hand avstamp i ett sociolingvistiskt perspektiv med Hallidays systemisk-funktionella lingvistik, SFL, som grundverktyg men då analysunderlaget innehåller en stor del bilder finns även ett sociosemiotiskt perspektiv. Dessa inslag vilar på Kress och van Leeuwens applikation av SFL vad gäller tolkning av andra semiotiska artefakter än verbalt språk. Analysen visar att säljdiskursen har genomgått en betydande förändring under de senaste 40 åren. Sedan millennieskiftet marknadsförs inte längre inredning med argumenten kvalitet eller design. Diskursen 2013 beskriver istället livsstilskoncept och erbjuder modelläsaren möjlighet att stärka sitt personliga varumärke. Vilken familjemedlems behov som ska uppmärksammas eller tillfredställas skiftar över tid. Tilltalet har gått från ett textuellt och semiotiskt pluralt ni-tilltal, till ett nästan helt uteslutande textuellt singulärt du-tilltal. Även semiotiken visar på en utveckling mot individualisering. I de senare katalogerna realiseras parrelationen i första hand via inanimata objekt som kläder och skor och i mycket liten utsträckning via personer. Barnet däremot ges en allt mer framträdande roll över tid och i 2013 års katalog utgör barnets behov det främsta säljargumentet efter individens valfrihet. I samma katalog är det uteslutande mannen som realiseras som vårdgivare. Kvinnan som mamma har idag lägre status än tidigare. Däremot realiseras kvinnan utan barn med allt högre status. Detta och analysen av hur deltagarnas och produkternas placering har förändrats över tid tycks implicera att den prioriterade modelläsaren är kvinna, har barn och tillräckligt god ekonomi för att kunna lägga pengar på inredning. Hon måste dock i egenskap av mamma, förhålla sig till att övriga familjemedlemmar och kvinnor utan barn realiseras med högre status.

Doctor of Philosophy(PhD) / There is increasing awareness among linguists that discourse analysis inevitably involves analyses of meanings arising from the combination of multiple modes of communication. The evolving multimodal pedagogic environment for teaching English as a foreign language (henceforth EFL), among other communicative contexts, calls for a social, semiotic, and linguistic explanation. Situated within the theoretical landscape of social semiotics and in the pedagogic context of EFL education, the present study aims to elucidate how linguistic and visual semiotic resources are co-deployed to construe interpersonal meaning in multimodal textbooks. The data drawn upon are eighteen EFL textbooks for primary and secondary schooling, published by People’s Education Press between 2002 and 2006. The research design consists of three complementary sub-studies. First, it investigates the ways in which the semantic regions of ENGAGEMENT and GRADUATION can be modelled in multimodal texts, with special reference to the interplay of voices in textbook discourse. The second sub-study analyzes how verbal and visual semiotic resources are co-deployed to construe the ‘emotion and attitude’ goal highlighted in curriculum standards, with a particular focus on verbiage-image relations. Third, it extends the linguistic concept ‘modality’ to multimodal discourse, exploring coding orientation in texts for different educational contexts and between different constituent genres. The main findings of this thesis are as follows: (1) A range of multimodal resources (i.e. labelling, dialogue balloon, jointly-constructed text, illustration and highlighting) are identified as enabling editor voice to negotiate meanings with reader voice and character voice. It is found that the way in which an ENGAGEMENT value can be scaled is strongly associated with the intrinsic property of the given multimodal resource. The interaction between multiple voices is closely related to contact, social distance, and point of view. (2) It is shown that images play an essential role in realizing attitudinal meanings. Together with verbal APPRAISAL resources, visual semiotic features work to position the readers in ways that align them to set pedagogic goals, guiding them in completing jointly-constructed texts. Moreover, an attitudinal shift from an emotional release to a more institutionalized type of evaluation can be identified as students advance through the school years. (3) It is argued that what counts as real in multimodal texts is socially defined and specific to a given communicative context. The nature of pedagogic discourse should be taken into account when visual displays are produced for pedagogic materials. The implications of this study include both theoretical and pedagogic aspects。Theoretically it adapts and extends APPRAISAL analysis to multimodal discourse, exploring the intersemiotic complementarity and co-instantiation in construing global evaluative stance. This semiotic exploration, in return, suggests ways in which discourse analysis may help textbook users better understand and interpret the multimodal features. With the affordances as well as limitations of semiotic resources made explicit, we may have one step further towards a comprehensive and critical understanding of multimodal construal of interpersonal meaning in pedagogic materials.

In human-human dialogues, face-to-face meetings are often preferred over phone conversations.One explanation is that non-verbal modalities such as gesture provide additionalinformation, making communication more efficient and accurate. If so, computerprocessing of natural language could improve by attending to non-verbal modalitiesas well. We consider the problem of sentence segmentation, using hand-annotatedgesture features to improve recognition. We find that gesture features correlate wellwith sentence boundaries, but that these features improve the overall performance of alanguage-only system only marginally. This finding is in line with previous research onthis topic. We provide a regression analysis, revealing that for sentence boundarydetection, the gestural features are largely redundant with the language model andpause features. This suggests that gestural features can still be useful when speech recognition is inaccurate.

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / The presence and relevance of images in discursive genres of contemporary communication has led to concerns about the formation of visually literate readers. Among the multimodal genres, we foreground the textbook for English learning (LDI), which is typically rich in terms of images. However, the activities suggested in such materials rarely point to the visual language exploration (GRAY, 2010). Thus, in this study, the reading sessions of the Top Notch English textbook series are analyzed, together with the methodological instructions of the Teacher‟s guide, aiming to verify to what extent the reading of the images of the discursive genres is explored and the nature of this reading. A context of genre consumption analysis was carried out in the Brazilian Military School System (SCMB). Within the theoretical framework of Critical Genre Analysis (MOTTA-ROTH, 2008) and using the Grammar of Visual Design (KRESS, van LEEUWEN, 2006) as lexical-grammatical reference, all the ten reading sessions of intermediate level Top Notch textbook were analyzed. These reading sessions are composed of semiotic complexes assembled with verbal written language and images. The analysis showed that 32.8% of the reading activities are offered to be approached multimodally through LDI and Teacher‟s guide instructions and also that 67% of the images of the reading sessions are activated for visual reading, what points to the role of the image as a meaning carrier, not as an illustration. There was a predominance of representational meanings (71%) explored in the multimodal reading activities, majorly placed in the pre reading stage, as a way to prepare for the verbal reading. However, the reading activities of corpus do not promote multimodal critical literacy, once they do not stimulate on the student readers the development of critical consciousness, one which unveils the power relations that occur via the genre and also the relationship between the text and the context where it takes place. In the multiliteracies perspective, we hope to contribute for the development of a broader reading towards the comprehension of represented meanings, not only in written verbal language, but also in the images of multimodal texts. / A presença e relevância de imagens em gêneros discursivos da comunicação contemporânea têm gerado uma preocupação crescente com a formação de leitores visualmente letrados. Dentre os gêneros multimodais destaca-se o livro didático para o ensino da língua inglesa (LDI), tipicamente rico em imagens. Entretanto, as atividades propostas nesses materiais raramente remetem à exploração da linguagem visual (GRAY, 2010). Em vista disso, neste trabalho são analisadas as seções de leitura da série de livros didáticos Top Notch, conjuntamente com as instruções metodológicas do Manual do professor com o objetivo de verificar em que medida a leitura de imagens dos gêneros discursivos apresentados é explorada e qual a natureza dessa leitura. Foi realizada uma análise do contexto de consumo do LDI o Sistema Colégio Militar do Brasil (SCMB). Dentro do enquadramento teórico da Análise Crítica de Gênero (MOTTA-ROTH, 2008) e tendo como base analítica léxico-gramatical a Gramática do Design Visual (KRESS, van LEEUWEN, 2006), foram analisadas todas as dez seções de leitura que constituem o Top Notch intermediário, as quais formam complexos semióticos, compostos de linguagem verbal escrita e de imagens. A análise revelou que 32,8% das atividades de leitura são oferecidas para serem abordadas multimodalmente por meio de instruções no LDI e no Manual do professor e que 67% das imagens das seções de leitura são acionadas para a leitura visual, o que aponta para o papel da imagem como portadora de significado e não como ilustração. Houve uma predominância dos significados representacionais (71%) explorados nas atividades de leitura multimodal, majoritariamente na etapa de pré-leitura, como forma de preparação para a leitura verbal. Entretanto, as atividades de leitura do corpus não promovem o letramento crítico multimodal por não estimularem nos alunos leitores o desenvolvimento de uma consciência crítica, que desvele as relações de poder que ocorrem por meio do gênero e a relação entre o texto e o contexto no qual ocorre. Na perspectiva dos multiletramentos, espera-se contribuir para o desenvolvimento da leitura mais ampla no sentido da compreensão dos significados representados não só na linguagem verbal escrita, mas também nas imagens em textos multimodais.

Läroböcker har en stark ställning och fyller viktiga funktioner i många ämnen i skolan. I och med decentraliseringen av skolan har ansvaret för läromedelsgranskning förflyttats från staten till de enskilda lärarna. Ansvaret ställer högre krav på lärarnas förmåga att kritiskt granska och välja läroböcker av god kvalitet. För att lärare ska kunna granska läroböcker är det bra att ha verktyg för att tränga in i och analysera läroböcker utifrån olika perspektiv.    Syftet med studien är att genom en multimodal analys undersöka olika modaliteters funktion och samspel i sex läroböcker i historieämnet avsedda för årskurs 4-6. De modaliteter som undersöks är brödtext, bild, bildtext och rubriker. Studien utgår ifrån ett designteoretiskt multimodalt perspektiv där lärande ses som en process och människan skapar mening genom att integreras i olika modaliteter och delta i olika kommunikativa sammanhang. Metoden som används för att besvara studiens syfte och forskningsfrågor är en analysmodell som utformades efter en etablerad modell för ämnesdidaktiskt arbete med multimodala texter.    Resultatet visar att det finns ett samspel mellan modaliteterna i läroböckerna men med vissa skillnader. Det som skiljer läroböckerna åt är mängden modaliteter och hur de fördelas i uppslagen. Även läroböckernas layout varierar, med skillnader i typsnitt, färg och placering av modaliteter. Skillnaden i layouten bidrar till att läroböckerna ger olika intryck och påverkar hur lätta böckerna är att orientera sig i. En annan skillnad som framgår är att det förekommer olika bildtyper i läroböckerna såsom fotografier, tidslinjer, släktträd, kartor och tecknade bilder. Bildtyperna fyller olika funktion och bidrar på olika sätt till förståelse.     En slutsats som kan dras är att ingen av de analyserade läroböckerna kan stå för sig själva, eftersom eleverna kan behöva vägledning i att orientera sig bland de olika modaliteterna i läroböckerna. Lärare kan behöva det ha i åtanke vid val och användning av läroböcker i undervisningen.

Submitted by Fernando Souza (fernandoafsou@gmail.com) on 2017-08-09T11:20:43Z No. of bitstreams: 1 arquivototal.pdf: 4153663 bytes, checksum: 1498bea749a2cf07373bbe25692a39fd (MD5) / Made available in DSpace on 2017-08-09T11:20:44Z (GMT). No. of bitstreams: 1 arquivototal.pdf: 4153663 bytes, checksum: 1498bea749a2cf07373bbe25692a39fd (MD5) Previous issue date: 2016-07-28 / The discursive multimodality is a common linguistic phenomenon to any genre, and it is the union of more than one semiotic resource in a enunciation event. It is something that requires an understanding of semiotic materiality of texts paying attention to the understanding of all resources used in its constitution. It is something that requires a comprehension of semiotic materiality of texts paying attention to the understanding of all resources used in its constitution. Thus, this research focuses on such epistemological approach and it has as principal goal to investigate the implementation of a multimodal model as didactic proposal to the approach of the multimodal genres in Portuguese Language Teaching. Thus, this research focuses on this epistemological approach and aims to investigate the implementation of a multimodal model as didactic proposal to address the multimodal genres in Portuguese Language Teaching classes. The central object of the investigation is a field research developed in a class of ninth year of the Primary School in a public school in collaboration with the titular teacher of the cited class. The generation of data corresponded to three specific moments: the sequences of lessons, which consists in the planning and realization of the didactic project with the multimodal genres; the oral and written texts produced by the collaborator teacher about her participation in the research; and the textual productions of the students of the investigated class.The central object of this presente research is a research developed in a class of ninth year in a public school, that was done with the collaboration of the teacher of that class. The research data is based on three specific moments: the sequences of lessons, which is the planning and realization of the educational project with multimodal genres; oral and written texts produced by the titular teacher about her participation in the research; and textual productions of the students of the investigated group. Based on multimodal theoretical model, a sequence of three classes were developed and applied, in order to develop visual literacy. The main results point to the identification of multimodal model as an essential educational tool in the visual literacy process to develop a educational attainment. This fact leads to another: the perception of a critical and more aware use of language materialized by the interaction of verbal and image in the student productions. In addition, evaluations of own pedagogical, made by the titular teacher, demostrated a positive assessment performed experience while also unveil epistemological difficulties during the research. Therefore, we can state the positive contribution of the multimodal teaching project to approach a practice of reading and production of imagistic texts in a socially conscious way. This fact corraborate to the thesis that participation in an educational project through multimodal model enables contact with aspects of the production of multimodal genres and with its own meta-language developing Visual Literacy and critical training learner. / A multimodalidade discursiva é um fenômeno linguístico comum a todo gênero textual e que consiste na união de mais de um recurso semiótico em uma composição enunciativa. É algo que requer uma compreensão da materialidade semiótica dos textos atentando para o entendimento de todos os recursos usados em sua constituição. Assim, a presente pesquisa centra-se nessa abordagem epistemológica e tem como principal objetivo investigar a implementação de um modelo multimodal como proposta didática para a abordagem dos gêneros multimodais no Ensino de Língua Portuguesa. O estudo encontra embasamento teórico nos postulados sobre Multimodalidade e Letramento Visual, especialmente, nos trabalhos de Almeida (2009; 2011), Kress e van Leeuwen (1996) e Stokes (2002) e nas concepções acerca da Linguística Aplicada e dos gêneros textuais com os estudos de Moita Lopes (2006; 2009; 2013), Bronckart (2006), Dionísio (2005) e Balloco (2005). O objeto central da investigação é uma pesquisa de campo desenvolvida em uma turma de nono ano do Ensino Fundamental de uma escola pública em colaboração com a professora titular da referida turma. A geração dos dados correspondeu a três momentos específicos: as sequências de aulas, que consiste no planejamento e realização do projeto didático com os gêneros multimodais; os textos orais e escritos produzidos pela professora colaboradora sobre sua participação na pesquisa; e as produções textuais dos alunos da turma investigada. Foram elaboradas e aplicadas, à luz do modelo multimodal, três sequências de aulas, tendo em vista o desenvolvimento do letramento visual. À luz da análise desenvolvida, os principais resultados apontam para a identificação do modelo multimodal como uma ferramenta didática essencial no processo de letramento visual e na formação crítica do aluno. Essas evidências conduzem a outro fato constatado na análise - à percepção de uma linguagem crítica e mais consciente em relação ao seu uso materializada pela interação do verbal e da imagem, nas produções dos alunos. Além disso, as avaliações do próprio fazer pedagógico, feitas pela professora colaboradora, revelam uma avaliação positiva da experiência realizada, embora desvele também dificuldades de ordem epistemológica no decorrer da pesquisa. Portanto, como desdobramentos de todas as análises, ficou constatado a contribuição positiva do projeto didático para a abordagem de uma prática de leitura e produção de textos imagéticos de forma socialmente consciente. Esse fato serviu para confirmar a tese de que a participação em um projeto de ensino por meio do modelo multimodal possibilita o contato com aspectos da produção dos gêneros multimodais e com uma metalinguagem própria desse gênero contribuindo para o desenvolvimento do Letramento Visual e para a formação crítica do aprendiz.

Dissertação (mestrado)—Universidade de Brasília, Faculdade de Ciência da Informação, Programa de Pós-Graduação em Ciência da Informação, 2011. / Submitted by Albânia Cézar de Melo (albania@bce.unb.br) on 2012-01-30T14:13:38Z No. of bitstreams: 1 2011_EmilioEvaristoSousa.pdf: 8552042 bytes, checksum: 6de9bded55da13c77b126515e1f631fa (MD5) / Approved for entry into archive by Patrícia Nunes da Silva(patricia@bce.unb.br) on 2012-02-02T18:39:13Z (GMT) No. of bitstreams: 1 2011_EmilioEvaristoSousa.pdf: 8552042 bytes, checksum: 6de9bded55da13c77b126515e1f631fa (MD5) / Made available in DSpace on 2012-02-02T18:39:13Z (GMT). No. of bitstreams: 1 2011_EmilioEvaristoSousa.pdf: 8552042 bytes, checksum: 6de9bded55da13c77b126515e1f631fa (MD5) / Recuperar informação disponibilizada em vídeos longos pode ser uma experiência demorada. A recuperação da informação em vídeos apresenta alguns problemas, pois é realizada, principalmente, através das descrições textuais feitas pelos usuários. O objetivo desta pesquisa é promover, com maior índice de precisão (Cleverdon,1962), a recuperação da informação disponibilizada em vídeos por meio da indexação multimodal e uso de ontologia. Foi desenvolvida uma ontologia de domínio, associada com uma indexação multimodal para avaliação da precisão em um sistema de recuperação da informação audiovisual. Observou-se nos experimentos um aumento significativo da precisão nas configurações que utilizaram a ontologia para recuperar informação audiovisual. A utilização de uma ontologia de domínio da Doutrina Espírita alinhada a indexação multimodal, em um sistema de recuperação de trechos de vídeos de palestras sobre esse tópico, permitiu melhorar a eficiência do mesmo em termos de precisão. ______________________________________________________________________________ ABSTRACT / Retrieving information available on long videos can be a time consuming experience. Information retrieval on video presents some problems because users mainly do it through textual descriptions. The objective of this research is to promote, with more precision (Cleverdon,1962), information retrieval available on video through the use of multimodal indexing and use of ontology. A domain ontology was developed, associated with an multimodal index for evaluation of accuracy in an audiovisual information retrieval system. A significant increase in accuracy in the experiments was observed in settings that used the ontology to retrieve audiovisual information. The use of a domain ontology of the Spiritist Doctrine aligned with multimodal indexing in a retrieval system for sections of videos of lectures on this topic, allowed for increased efficiency in terms of the same precision.

The search for multiple solutions is applicable to many fields (Engineering [54][67], Science [75][80][79][84][86], Economics [13][59], and others [51]). Multiple solutions allow for human judgement to select the best solution from a group of solutions that best match the search criteria. Finding multiple solutions to an optimisation problem has shown to be difficult to solve. Evolutionary computation (EC) and more recently Particle Swarm Optimisation (PSO) algorithms have been used in this field to locate and maintain multiple solutions with fair success. This thesis develops and empirically analyses a new method to find multiple solutions within a convoluted search space. The method is a hybrid of the NichePSO [14] and the sequential niche technique (SNT)[8]. The original SNT was developed using a Genetic Algorithm (GA). It included restrictions such as knowing or approximating the number of solutions that exist. A further pitfall of the SNT is that it introduces false optima after modifying the search space, thereby reducing the accuracy of the solutions. However, this can be resolved with a local search in the unmodified search space. Other sequential niching algorithms require that the search be repeated sequentially until all solutions are found without considering what was learned in previous iterations, resulting in a blind and wasteful search. The NichePSO has shown to be more accurate than GA based algorithms [14][15]. It does not require knowledge of the number of solutions in the search space prior to the search process. However, the NichePSO does not scale well for problems with many optima [16]. The method developed in this thesis, referred to as the derating NichePSO, combines SNT with the NichePSO. The main objective of the derating NichePSO is to eliminate the inaccuracy of SNT and to improve the scalability of the NichePSO. The derating NichePSO is compared to the NichePSO, deterministic crowding [23] and the original SNT using various multimodal functions. The performance of the derating NichePSO is analysed and it is shown that the derating NichePSO is more accurate than SNT and more scalable than the NichePSO. / Dissertation (MSc)--University of Pretoria, 2007. / Computer Science / MSc / Unrestricted

This study explored the critical features of temporal synchrony for the facilitation of prenatal perceptual learning with respect to unimodal stimulation using an animal model, the bobwhite quail. The following related hypotheses were examined: (1) the availability of temporal synchrony is a critical feature to facilitate prenatal perceptual learning, (2) a single temporally synchronous note is sufficient to facilitate prenatal perceptual learning, with respect to unimodal stimulation, and (3) in situations where embryos are exposed to a single temporally synchronous note, facilitated perceptual learning, with respect to unimodal stimulation, will be optimal when the temporally synchronous note occurs at the onset of the stimulation bout. To assess these hypotheses, two experiments were conducted in which quail embryos were exposed to various audio-visual configurations of a bobwhite maternal call and tested at 24 hr after hatching for evidence of facilitated prenatal perceptual learning with respect to unimodal stimulation. Experiment 1 explored if intermodal equivalence was sufficient to facilitate prenatal perceptual learning with respect to unimodal stimulation. A Bimodal Sequential Temporal Equivalence (BSTE) condition was created that provided embryos with sequential auditory and visual stimulation in which the same amodal properties (rate, duration, rhythm) were made available across modalities. Experiment 2 assessed: (a) whether a limited number of temporally synchronous notes are sufficient for facilitated prenatal perceptual learning with respect to unimodal stimulation, and (b) whether there is a relationship between timing of occurrence of a temporally synchronous note and the facilitation of prenatal perceptual learning. Results revealed that prenatal exposure to BSTE was not sufficient to facilitate perceptual learning. In contrast, a maternal call that contained a single temporally synchronous note was sufficient to facilitate embryos’ prenatal perceptual learning with respect to unimodal stimulation. Furthermore, the most salient prenatal condition was that which contained the synchronous note at the onset of the call burst. Embryos’ prenatal perceptual learning of the call was four times faster in this condition than when exposed to a unimodal call. Taken together, bobwhite quail embryos’ remarkable sensitivity to temporal synchrony suggests that this amodal property plays a key role in attention and learning during prenatal development.

During the past few decades, there has been increasing attention to multimodal adaptive language learning interface design. The purpose of this study was to examine users’ experiences with a chatbot language learning interface through the lens of cognitive emotions and emotions in learning. A particular focus of this study was on users’ interactions with a chatbot in a public setting and in a private environment. Focusing on the event of users’ interaction with a chatbot interface, seventy-five interactions were videotaped in this study, in which fifteen users were asked to interact with the chatbot “Lucy” for their language learning. The video-stimulated post interaction interviews with participants provided complementary data for understanding their experiences with the language learning system. Analysis of twenty-five interactions selected from a total of seventy-five revealed five main factors of chatbot language tutor interface design and their relative significance in the process of users’ meaning making and knowledge construction. Findings showed that users’ sensory, emotional, cultural, linguistic and relational engagement influenced their responses to the chatbot interface, which in turn, shaped their learning processes. Building on a theoretical framework of cognitive emotions and emotions in learning, this study documented users’ language learning processes with the chatbot language learning interface by investigating users’ experiences. The findings and techniques resulting from this study will help designers and researchers achieve a better understanding of users’ experiences with technology and the role of emotions in the processes of learning when using technology and assist them to improve the design of language learning environments. / Education, Faculty of / Curriculum and Pedagogy (EDCP), Department of / Graduate