In this dissertation, I address the relationship between the content and design of governmental websites and the ideological interests of the overseeing political administrations. Three case studies contrast the contemporary political climates in the Bolivarian Republic of Venezuela, the Republic of Colombia, and the United Stastes with the form and function of their respective military recruitment homepages. Through a semiotic, and specifically multimodal lens, I aim to determine to what degree there is evidence of the governing poltical party’s ideological perspective on the websites. To accomplish this task, each case study is introduced by a brief contemporary history in order to provide a summary of key political events within state. Each case study then includes a detailed analysis of 3-4 governmental homepages. Within those analyses, a comprehensive multimodal analysis is conducted for the most evocative content on each homepage. Finally, conclusions are reached for each case study, paying special attention to the content found along the reading path.

In this thesis, the four types of math problems that occur in the digital educational math game Zcooly Affären 2 have been analyzed. Focus has been aimed at how the problems mediate instructions and mathematical concepts to players, and how the mathematical content is supported by the context within the game. The result shows both positive and negative aspects of the math problems, based on the theories for math education used in this thesis. Apart from the analysis of the game, this thesis also presents earlier research regarding what possibilities digital games possess as tools for learning. Researchers have partly different views on how digital games should be designed to enhance learning, and whether or not games are efficient to use for learning purposes.

Denna studie har undersökt hur en grupp elever använder ett virtuellt rum som klassrum. Effekten blev att de lärde sig fakta med hela kroppen. Det virtuella klassrummet kan bli ett nytt pedagogiskt redskap för skolungdomar där de kan vara med och bygga upp eller påverka innehållet. Studien är en etnografisk undersökning där jag har studerat multimodalitetsteorier av bl.a. Gunther Kress i vilken man tar hänsyn till alla teckenvärldar som ljud, bild, gester och kroppens rörelse i rummet. Undersökningen har behandlat frågorna; Vad produceras när barn och ungdomar använder virtuell verklighet i klassrummet? Vilka diskurser framträder när VR används som resurs för lärande? Hur kan designen i en virtuell klassrumsmiljö komma till uttryck? Hur kan virtuella läromedel distribueras? Jag har undersökt hur diskurser kan förändras efter platsens förutsättningar och hur designen av en VR-undervisning kan beskrivas. Studien innefattar en multimodal produktion under en bildlektion i ett virtuellt klassrum, undersökningen inkluderar även distribution av virtuell konst i lärmiljö. Jag har vidare funderat på vilka förutsättningar som krävs för att virtuell verklighet skall bli en standard i framtidens klassrum.

Students of chemistry encounter difficulties due to its abstract nature and the need to understand and communicate its concepts on macro, submicro, and symbolic levels using a range of representations and representational modes. Research suggests that when students are required to use multiple representations they have difficulties in understanding individual representations and in negotiating meaning through their use. This study sought to address these issues through the application of digital technologies. The main areas of research that provided a theoretical framework for this study were multiple representations in chemistry education and writing-to-learn in science. Other research in these areas has suggested that a better understanding of multiple representations might enhance students’ chemical literacy; however, limited research has investigated the impact of using digital technologies to create multimodal texts on students’ learning in chemistry, particularly the development of students’ skills in generating and integrating multiple representations. Until recently, much of the writing-to-learn research has focused on written composition. The knowledge-transforming model was proposed by Bereiter and Scardamalia (1987) to explain the influence of written composition on knowledge construction. However, having been developed prior to the time when students had ready access to digital technologies and a consequent capacity to create multimedia and digital texts, this model does not account for the production of such multimodal texts. This study examined the effect of learning experiences that utilised digital technologies to support students in using multiple representations and through writing-to-learn activities to create multimodal texts on learning outcomes in chemistry. The study was conducted in a metropolitan public co-educational high school in Queensland, Australia. Two Year 11 chemistry classes participated in the study, which was conducted in the first term of a 2-year course in which students learn chemistry as a separate discipline. The study consisted of a pilot study and an intervention study with two phases. The pilot study was used to trial the learning activities and data collection instruments and to gain an insight into instructional approaches that might be appropriate for the study. Phase 1 of the intervention study employed a pretest–posttest design. In this phase, students learned about chemical bonding and structure and their effects on the properties and behaviours of different materials. They also learned about the multiple representations used to understand and communicate about chemical bonding and structure. Within a modified crossover design, Phase 2 of the study employed mixed methods to compare the effects on learning outcomes when they created two different scientific texts: a digital poster and a laboratory report. Both text types required students to integrate multiple representations to report on their learning during laboratory investigations. These text types were chosen because they are commonly used by scientists to communicate their experimental findings. In Phase 1, students engaged in computer-based inquiries using both molecular modelling and simulation software to investigate phenomena such as intra- and inter-molecular bonding and their effects on properties, the differences between various types of bonds, the multiple representations used to describe and investigate bonding and structure, and to present their understanding to others. In Phase 2, students used a range of scaffolding resources to design and carry out two inquiries about the chemistry of biomaterials. In the first inquiry, students made and compared the properties of two different bioplastic films; in the second, students compared the relative fermentation rates of a range of carbohydrates. In both inquiries, students were required to report their findings and explain them on the submicro level using appropriate representations. Scaffolds included Science Writing Heuristics, which explicitly required students to consider which multiple representations would support their claims and explanations of data; digital resources for selecting, modifying, or creating representations; and genre templates. Pretest–posttest comparisons for both phases showed that the instructional approaches and resources used were effective for enhancing students’ learning outcomes. In all comparisons, the posttest performances were significantly higher. In the first phase, several of the identified alternative or missing conceptions about chemical bonding were effectively addressed, and in both phases, students’ conceptual understanding and their representational competencies were enhanced. The pretest–posttest comparisons for Phase 2 suggested that creating a diversified text – a digital poster – for explaining experimental results is at least as effective for enhancing understanding and representational competencies as creating a more traditional laboratory report. Other data were analysed to gain an insight into how or why the instructional strategies and resources used might have been effective. The student interviews revealed a number of advantages of using digital technologies, including promotion of higher order thinking, enhanced motivation and interest, the capacity of digital technologies to support and enhance visualisation, and the production of multiple representations in multiple modes. Students suggested that the digital resources allowed them to make links between macroscopic, molecular, and symbolic levels and to include a range of representations in their explanations. The evaluation questionnaire revealed similar trends. Analysis of the students’ texts suggested that the approaches used in Phase 2 were effective in supporting students’ content and rhetorical problem solving and the interactions between the two. Students utilised a range of representations, particularly structural diagrams, when making explanations of their macroscopic data on the submicro level. This study has implications for the instructional approaches used by chemistry teachers because it showed that integrating digital technologies into learning environments is effective when introducing students to the multiple representations used in chemistry and in the development of students’ chemical literacies. It also contributes to writing-to-learn research by focusing on multimodal communication and the benefits of creating multimodal texts for presenting, organising, and explaining data, and for representing knowledge. Significant findings of the study relate to the importance of digital technologies in generating multimodal texts and representations for instruction, scaffolding, and in student-centred inquiry-based learning. Further research might focus on the use of such resources for addressing other commonly identified alternative conceptions, the creation of other multimodal text types, the use of other digital technologies or authoring tools, or on the development of teachers’ technological pedagogical content knowledge, which is required for effective classroom implementation of these resources and strategies.

Students of chemistry encounter difficulties due to its abstract nature and the need to understand and communicate its concepts on macro, submicro, and symbolic levels using a range of representations and representational modes. Research suggests that when students are required to use multiple representations they have difficulties in understanding individual representations and in negotiating meaning through their use. This study sought to address these issues through the application of digital technologies. The main areas of research that provided a theoretical framework for this study were multiple representations in chemistry education and writing-to-learn in science. Other research in these areas has suggested that a better understanding of multiple representations might enhance students’ chemical literacy; however, limited research has investigated the impact of using digital technologies to create multimodal texts on students’ learning in chemistry, particularly the development of students’ skills in generating and integrating multiple representations. Until recently, much of the writing-to-learn research has focused on written composition. The knowledge-transforming model was proposed by Bereiter and Scardamalia (1987) to explain the influence of written composition on knowledge construction. However, having been developed prior to the time when students had ready access to digital technologies and a consequent capacity to create multimedia and digital texts, this model does not account for the production of such multimodal texts. This study examined the effect of learning experiences that utilised digital technologies to support students in using multiple representations and through writing-to-learn activities to create multimodal texts on learning outcomes in chemistry. The study was conducted in a metropolitan public co-educational high school in Queensland, Australia. Two Year 11 chemistry classes participated in the study, which was conducted in the first term of a 2-year course in which students learn chemistry as a separate discipline. The study consisted of a pilot study and an intervention study with two phases. The pilot study was used to trial the learning activities and data collection instruments and to gain an insight into instructional approaches that might be appropriate for the study. Phase 1 of the intervention study employed a pretest–posttest design. In this phase, students learned about chemical bonding and structure and their effects on the properties and behaviours of different materials. They also learned about the multiple representations used to understand and communicate about chemical bonding and structure. Within a modified crossover design, Phase 2 of the study employed mixed methods to compare the effects on learning outcomes when they created two different scientific texts: a digital poster and a laboratory report. Both text types required students to integrate multiple representations to report on their learning during laboratory investigations. These text types were chosen because they are commonly used by scientists to communicate their experimental findings. In Phase 1, students engaged in computer-based inquiries using both molecular modelling and simulation software to investigate phenomena such as intra- and inter-molecular bonding and their effects on properties, the differences between various types of bonds, the multiple representations used to describe and investigate bonding and structure, and to present their understanding to others. In Phase 2, students used a range of scaffolding resources to design and carry out two inquiries about the chemistry of biomaterials. In the first inquiry, students made and compared the properties of two different bioplastic films; in the second, students compared the relative fermentation rates of a range of carbohydrates. In both inquiries, students were required to report their findings and explain them on the submicro level using appropriate representations. Scaffolds included Science Writing Heuristics, which explicitly required students to consider which multiple representations would support their claims and explanations of data; digital resources for selecting, modifying, or creating representations; and genre templates. Pretest–posttest comparisons for both phases showed that the instructional approaches and resources used were effective for enhancing students’ learning outcomes. In all comparisons, the posttest performances were significantly higher. In the first phase, several of the identified alternative or missing conceptions about chemical bonding were effectively addressed, and in both phases, students’ conceptual understanding and their representational competencies were enhanced. The pretest–posttest comparisons for Phase 2 suggested that creating a diversified text – a digital poster – for explaining experimental results is at least as effective for enhancing understanding and representational competencies as creating a more traditional laboratory report. Other data were analysed to gain an insight into how or why the instructional strategies and resources used might have been effective. The student interviews revealed a number of advantages of using digital technologies, including promotion of higher order thinking, enhanced motivation and interest, the capacity of digital technologies to support and enhance visualisation, and the production of multiple representations in multiple modes. Students suggested that the digital resources allowed them to make links between macroscopic, molecular, and symbolic levels and to include a range of representations in their explanations. The evaluation questionnaire revealed similar trends. Analysis of the students’ texts suggested that the approaches used in Phase 2 were effective in supporting students’ content and rhetorical problem solving and the interactions between the two. Students utilised a range of representations, particularly structural diagrams, when making explanations of their macroscopic data on the submicro level. This study has implications for the instructional approaches used by chemistry teachers because it showed that integrating digital technologies into learning environments is effective when introducing students to the multiple representations used in chemistry and in the development of students’ chemical literacies. It also contributes to writing-to-learn research by focusing on multimodal communication and the benefits of creating multimodal texts for presenting, organising, and explaining data, and for representing knowledge. Significant findings of the study relate to the importance of digital technologies in generating multimodal texts and representations for instruction, scaffolding, and in student-centred inquiry-based learning. Further research might focus on the use of such resources for addressing other commonly identified alternative conceptions, the creation of other multimodal text types, the use of other digital technologies or authoring tools, or on the development of teachers’ technological pedagogical content knowledge, which is required for effective classroom implementation of these resources and strategies.

Thesis (MA)--Stellenbosch University, 2014. / ENGLISH ABSTRACT: This study investigates multimodal assessment in the South African higher education context. The communication landscape of students is becoming increasingly multimodal, resulting in a shift away from higher education institutions’ preferred mode (that is, the written mode). This is partly as a result of the digital era in which we live, as the verbal, visual and audio modes co-exist to make meaning, thereby creating new forms of text (Iedema 2003: 33). Although there is a common acceptance that the communication landscape has changed, higher education institutions still seem to consider the written text and written communication as the most dominant form of meaning-making (Lea 2004: 743). Thus, there is a disparity between the types of literacies with which students arrive at university, and the types of literacies that they are expected to use in university. I argue that this disparity is problematic for education, and maintain that pedagogies be transformed in order to resolve this issue. In this way, students will be able to “benefit from learning in ways that allow them to participate fully in public, community, and economic life” (Cazden et al. 1996: 60). Data for this research includes assignments that were produced by second-year students of Applied English Language Studies, a subject offered by the Department of General Linguistics at Stellenbosch University. These assignments include a multimodal component as well as a formal, written component. Analysis of their assignments revealed that students show great dexterity in their creations of multimodal texts. Apart from their design skills, it was revealed that students have knowledge of a wide variety of social discourses, which is currently mostly ignored in the education context. Thus, I propose that this knowledge, along with the digital and visual design skills with which students arrive at university, be valorised and utilised as an entry point for the teaching of linguistic literacy. This proposal is partly supported by schema theory, a cognitive theory of learning, which entails that existing knowledge is used as a platform on which to build new knowledge. / AFRIKAANSE OPSOMMING: Hierdie studie ondersoek multimodale assessering in die Suid-Afrikaanse hoër onderwys konteks. Die kommunikasie landskap van studente word al hoe meer multimodaal wat ʼn skuif weg van die voorgekeurde modaliteit (die geskrewe) in hoër onderwys teweegbring. Dit is gedeeltelik a.g.v. die digitale era waarin ons leef waarin die verbale, visuele en klank modaliteite saam gebruik word om betekenis te skep; dus word nuwe vorme van teks geskep (Iedema 2003: 33). Alhoewel daar algemeen aanvaar word dat die kommunikasie landskap verander het, beskou hoër onderwys instansies nog steeds die geskrewe teks en geskrewe kommunikasie as die dominante vorm van betekenisskepping (Lea 2004: 743). Daar is dus ʼn gaping tussen die tipes geletterheid waarmee studente by die universiteit opdaag en watter daar van hulle verwag word om te gebruik in die universiteit. Ek voer aan dat hierdie gaping problematies is vir opvoedkunde en stel voor dat pedagogie verander moet word om dit aan te spreek. Op hierdie manier kan studente voordeel trek op maniere wat hul toelaat om ten volle deel te neem aan publieke, gemeenskaplike en ekonomiese lewe (Cazden et al. 1996: 60). Data vir hierdie navorsing sluit opdragte in wat deur tweede jaar Applied English Language Studies (ʼn vak wat deur die Departement Algemene Taalwetenskap by Stellenbosch Universiteit aangebied word) studente uitgevoer is. Die opdragte sluit ʼn multimodale element sowel as ʼn formele geskrewe element in. Analise van die opdragte wys dat studente vaardigheide het in die produksie van multimodale tekste. Behalwe die produksie vaardighede wys die analise ook dat hierdie studente kennis het van ʼn wye reeks sosiale diskoerse wat op die oomblik meestal geïgnoreer word in die opvoedkundige konteks. Ek voer dus aan dat hierdie kennis sowel as die digitale- en visuele produksie vaardigheide waarmee studente by die universiteit opdaag, gevalideer en gebruik word as ingangspoort vir die aanleer van talige geletterheid. Deels word die voorstel deur skema teorie ondersteun, ʼn teorie wat in kognitiewe benaderinge tot leer ontwikkel het en wat voorstel dat bestaande kennis gebruik kan word as ʼn platform om nuwe kennis te bou.

Made available in DSpace on 2015-05-14T12:43:25Z (GMT). No. of bitstreams: 1 parte1.pdf: 1313582 bytes, checksum: 7db4b861f9f9efa8a9e9d6296921f6e5 (MD5) Previous issue date: 2010-01-28 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES / This work consists of an investigation on images belonging to the German coursebook Themen aktuell 1 and the English New English File Elementary from the perspective of the Visual Grammar by the theoreticians Günther Kress e Theo van Leeuwen (2006). Based on Michael Halliday s Systemic Functional Linguistics (1994), the Visual Grammar aims at systematising the image-reading process from a theoretical as well as practical point of view. Through its applicable feature, this work aims at increasing visibility of particular sociocultural elements produced by the authors of foreign language coursebooks. In order to achieve this goal, it will take into consideration the constitutive elements of images as well as the property of images in construing meanings. / Este trabalho consiste em uma leitura contrastiva de imagens pertencentes ao livro didático de alemão Themen aktuell 1 e ao de inglês New English File Elementary, sob a perspectiva da Gramática Visual dos teóricos Günther Kress e Theo van Leeuwen (2006). Baseada na Gramática Sistêmico-Funcional (Halliday, 1994), a Gramática Visual busca sistematizar o processo de leitura de imagens de um ponto de vista teórico-prático. A partir de sua aplicabilidade, é objetivo deste trabalho trazer à visibilidade elementos socioculturais próprios das culturas produzidas pelos autores de livros didáticos de línguas estrangeiras através dos elementos constitutivos da imagem bem como da imagem enquanto elemento na construção de sentidos.

This study explores uses of metaphor and metonymy related to the concept of anger in the American television series Teenage Mutant Ninja Turtles from 2012. The theoretical framework of the study is Conceptual Metaphor Theory. The data consists of metaphors and metonymies in the verbal and the pictorial mode. The pictorial data are visual expressions of the type called pictorial runes. In both modes, the underlying conceptual metaphors are identified, and the results are then compared between modes. The main finding is that the verbal mode contains a greater variety of metaphorical expressions. Explanations suggested for differences found between modes are: different technical possibilities of the two modes; universality in the pictorial mode and language specific metaphors as well as universal ones in the verbal mode; a connection of pictorial and verbal data respectively to different genres within the series; and the different narrator roles between the two modes.

Cette thèse vise à étudier l’interaction pédagogique par visioconférence entre sept futurs professeurs de français langue étrangère en formation professionnelle à l’Université de Lyon et douze apprenants de français langue étrangère inscrits dans une école de commerce à Dublin City University. A partir de données écologiques, nous avons construit un corpus audiovisuel de vingt-huit séances et de quinze heures et demie d’interactions. Par une démarche inductive, en nous appuyant sur des méthodes développées dans le cadre de l’analyse conversationnelle, nous avons identifié 295 séquences d’explication lexicale. Nous avons analysé celles-ci par une approche multimodale qui adopte une vue holistique de la communication et prend en compte l’utilisation de l’ensemble des ressources sémiotiques sans privilégier par défaut le langage oral. Nos analyses multimodales montrent de quelles façons les enseignants utilisent les ressources à leur disposition pour se focaliser sur la forme des items lexicaux, pour se focaliser sur leur sens et pour impliquer les apprenants dans les séquences d’explication. Nos données permettent également de mener des comparaisons interindividuelles grâce au fait que plusieurs enseignants, ayant reçu les mêmes consignes et se trouvant dans des situations similaires, expliquent les mêmes items lexicaux. / This thesis aims to study videoconferenced pedagogical interaction between seven future teachers of French who are undergoing professional development at the University of Lyon and twelve learners of French who are enrolled in a business school at Dublin City University. From naturally occurring interaction, we have constructed an audiovisual corpus comprised of twenty-eight sessions and fifteen and a half hours of interaction. Through an inductive approach, and relying on methods developed in the field of conversation analysis, we have identified 295 lexical explanation sequences. We have analyzed them using a multimodal approach which adopts a holistic view of communication and takes into consideration the entire palette of semiotic resources used without automatically prioritizing spoken language. Our multimodal analyses show the ways in which the teacher trainees use the resources at their disposal to focus on the form and meaning of lexical items as well as how they involve their learners in the explanation sequences. Our corpus also allows us to conduct an interindividual comparison due to the fact that several teacher trainees, having received the same instructions and being put in similar situations, explain the same lexical items.

In the light of recent changes on the audiovisual scene in Poland, audiences can choose among different AVT modalities. Although voice-over still prevails on Polish TV, subtitles have become more and more popular as an alternative form of film translation on television. Due to rapid technological advances, commercial requirements and differences in Polish viewers’ preferences, it is thus crucial to understand how audiences at different levels of English proficiency (low, medium, high) retrieve meaning, especially complex ironic meaning relayed through different methods of film translation, such as subtitles and voice-over and the extent to which verbal and non-verbal semiotic channels contribute to irony comprehension. Wilson and Sperber’s (1981, 1992; 1995) echoic theory of irony has been selected as the theoretical framework, given its ability to account for multimodal irony in audiovisual texts as well as the significant importance of non-verbal semiotic resources in the generation and interpretation of irony. The study employs triangulation, incorporating descriptive, experimental and interactionist components. The descriptive component involves multimodal transcription (Baldry and Thibault, 2006) of selected fragments in which irony plays a pivotal narrative role. This procedure aims to determine what non-verbal modes contribute to the multimodal construal of irony and how it is relayed in the subtitled and voiced-over translations. In the experimental component, viewers’ eye movements are recorded using eye-tracking technology while watching subtitled and voiced-over fragments of Sherlock Holmes (2009) and Sherlock Holmes: A Game of Shadows (2011). In the interactionist components, a questionnaire is used in order to elucidate how and/or whether they retrieve ironic meaning as intended by the filmmakers in the selected excerpts. The most obvious finding to emerge from the descriptive data analysis is that multimodal irony is not relayed by the film dialogue alone but, rather, in unison with non-verbal semiotic resources. The instances of multimodal irony in the two Sherlock Holmes films were found to perform narrative and comedic functions by combining the visual, kinesic and acoustic modes of film language. The analysis and comparison of SL dialogues and TL translations revealed two broad categories of irony relay, namely: preservation and modification. The majority of the instances of multimodal irony were modified in the subtitled version, while preservation is only sporadically opted for. In its voiced-over counterpart, the intended meaning is preserved and modified in equal proportions. The experimental component showed major differences in gaze patterns among the participants with different language skills in the subtitled clips. For instance, on average, LLPs spent more time reading the subtitles than HLPs or MLPs. Similar visual behavior, on the other hand, was observed among all viewers in the voiced-over clips in which the on-screen character’s face attracted the greatest amount of visual attention. The interactionist strand showed that the viewers retrieved the intended meaning to various extents depending on their English language proficiency. This data undergirds an assessment of the effectiveness of subtitles and voice-over in the translation and reception of multimodal irony on screen.