Orientador: Edwiges Maria Morato / Tese (doutorado) - Universidade Estadual de Campinas, Instituto de Estudos da Linguagem / Made available in DSpace on 2018-08-19T02:43:41Z (GMT). No. of bitstreams: 1 Vezali_Patrik_D.pdf: 3445724 bytes, checksum: f17d31a7560b2656695234de1528b2ea (MD5) Previous issue date: 2011 / Resumo: Este trabalho é o resultado de anos de pesquisa sobre a questão da significação em geral e da relação entre linguagem, corpo e cognição em específico. Desde a graduação até este doutoramento transitamos entre várias áreas diferentes, por exemplo a Semiótica e a Filosofia da Linguagem; mesmo assim, nosso questionamento principal sempre esteve aliado ao entendimento da constituição sócio-cognitiva de nossa mente e, também, da mútua constituição entre os vários sistemas semiológicos para nossa compreensão/expressão no mundo, mas nunca perdendo nosso ponto de partida: a Linguística. Nesta tese de doutorado, partimos da observação da gestualidade em contextos de produção afásica. Os gestos seriam complementares e/ou compensatórios em casos em que a linguagem encontra-se alterada de alguma maneira? Para analisar essa questão, inscrevemos nossa reflexão com base em dados audiovisuais de interações entre afásicos e não afásicos. A referenciação dêitica será tomada como foco de análise por se constituir, segundo nossas hipóteses iniciais, como o lugar de excelência em que a relação fala e gesto pode manifestar-se plenamente já que os elementos dêiticos precisam da situação de uso para sua significação; isso aciona todo um complexo jogo de mútua constitutividade entre diferentes semiologias. Estabeleceremos nossas bases teóricas sobre as atividades de referenciação dêitica verbais e não verbais, analisando a questão da multimodalidade; bem como aprofundando nossa reflexão sobre o tema desta pesquisa ao analisar trabalhos basilares sobre a relação entre fala e gesto (Kendon, 2004; McNeill, 1992). A observação dos dados e constatação de certos fenômenos de conjugação entre fala e gesto, levaram-nos à elaboração de sistema de transcrição que possibilitasse a alternativa de transcrever o gesto alinhado à fala sem precisar de descrições e, também, para que conseguíssemos visualizar, no dado transcrito, os fenômenos de interesse de nossa análise. Como revelaram nossos dados, não são apenas as entidades gramaticais e lexicais que são acionadas para referenciar, elas são dependentes das condições de emprego e de uso da linguagem - os sentidos associados aos contextos de uso dos dêiticos. Os dados de interações entre afásicos e não afásicos salientam a participação do verbal e do não verbal na construção da referência, seja pela postura no mundo, seja pelo recurso a elementos do contexto, seja pela gestualidade - meios que dão visibilidade às ações referenciais e interativas / Abstract: This work is the result of years of research on the question of meaning in general and the relationship between language, body and cognition in particular. Since the College through PhD I moved between several different areas, for example Semiotics and the Philosophy of Language; but the main questions has always been allied to the understanding of a sociocognitive mind, and also the mutual constitution between various semiological systems to our understanding & expression in the world but never losing our starting point: Linguistics. In this PhD, we start the questioning about the gestures in the contexts of aphasic production. Would be the gestures complementary and/or compensatory in cases where the language is changed in any way? We have put our thinking based on data of audiovisual interactions between aphasics and non-aphasics to analyze this question. The deictic referencing is taken as the focus of analysis because they constitute, according to our initial hypothesis, as the place of excellence where this relationship may manifest itself fully as the deictic elements require the use situation for its significance, this triggers a whole multimodal complex set of relationships. We will establish our theoretical basis on the activities of reference deictic verbal and nonverbal, analyzing the issue of multimodality, as well as deepen our reflection on the theme of this research to analyze jobs for our basic conceptualization of the relationship between speech and gesture (Kendon, 2004; McNeill, 1992). The observation data and observation of certain phenomena of conjugation between speech and gesture, they took us to the development of system to allow the alternative of transcript the gesture aligned to the speech without the need to describe and capable to view, in the transcript data, the phenomena of interest to our analysis. As our data show, not just the grammatical and lexical entities that are triggered to refer, they are dependent on the conditions of apply and use of language - the meanings associated with the use of deictic contexts. The data show in the interactions between the aphasics and non-aphasics underlines the involvement of verbal and nonverbal in the construction of reference, either by position in the world, or by appealing to elements of context, whether by gesture - environments that give means to the referential actions and interactions / Doutorado / Linguistica / Doutor em Linguística

The aim of this study was to examine how the Swedish magazine King represent masculinity in its personal portrait. We have used image analysis with semiotic multimodality about the representation of masculinities in the Swedish magazine King. We have, in this study, been using six personal portrait in our analysis. We have been studying the first and the last picture in the person portraits and also the first pictures interacting with the headline.   To help answer our issue, we asked ourselves the following questions: How does the image and title interact on the front page of the personal portrait? What is the dominant representation of masculinity in King? What variations of masculinities can be found in King?   The result we have found is that King choose to portray the men shown in the personal portraits like men in a position of power. This certain power has been manifested in different ways, such as in violence, comics and other elements. The dominant masculinity we found in our personal portrait was the hegemonic were only one man was not included in the hegemonic masculinity.

This essay aims to compare two EFL (English as a Foreign Language) textbooks by using a multimodal discourse analysis in order to find out how EFL textbooks have changed in design and visually. In this essay the textual content is treated as one of several pillars making up design, this essay is interested in the visual changes. This analysis is done using two EFL textbooks with twenty years in between them, both are used in the same school by different teachers. A study like this is going to be published later this year but that study includes three subjects (English being one of them) and starts with textbooks from the 1930s up until now. In this essay, two chapters of each textbook will be looked at  in a closer analysis to represent each textbook; every page is analyzed without first reading the text. The conclusion of this essay is that the written communication still seem to be the most credible of the different communicative methods of making meaning, however, it is no longer the only credible way of making meaning.

La biométrie est une alternative qui se base sur l'identification des personnes à partir de leurs caractéristiques physiques (empreinte digitale, forme de la main, empreinte palmaire) et/ou comportementales (voix, signature dynamique). La biométrie tend à réaliser deux buts importants dans notre vie courante. Le premier but est de réaliser la sécurité en éliminant le doute sur l'identité d'une personne et le second but est de faciliter l'identification des individus. En effet, cette méthode d'identification est de plus en plus préférée par rapport aux méthodes traditionnelles impliquant les mots de passe et les badges. Les travaux de recherche de cette thèse s'inscrivent dans le cadre de la reconnaissance de personnes à l'aide de la biométrie de la main. L'objectif principal est de concevoir un système biométrique multimodal basé sur la fusion de la forme de la main et de l'empreinte palmaire.La première partie de cette thèse propose un nouveau système uni-modal de vérification de la forme de la main. En effet, ce système est basé d'une part, sur la détection du meilleur ensemble des points-clés localisés sur le contour de la main pour adopter la description SIFT (Scale Invariant Feature Transform). D'autre part, un raffinement de correspondance, basé région et apparence de la main est proposé, afin de raffiner autant que possible les points-clés faussement matchés.Tandis que la deuxième partie consiste à proposer un nouveau système d'identification palmaire. En effet, la méthode de représentation parcimonieuse est adoptée afin de décrire le trait biométrique de l'empreinte palmaire. Elle est basée sur l'extraction de descripteurs SIFT de chacun des points-clés détectés. Notre troisième partie concerne la proposition de différentes méthodes de fusion multi-types de la multi modalité, comprenant la fusion multi-représentation, la fusion multi-biométrique et la fusion multi-instance. En effet, la fusion multi-représentation est basée sur la combinaison de descripteurs SIFT et les caractéristiques géométriques de la main au niveau des scores, pour la vérification de la forme de la main. La fusion multi-biométrique est basée sur la combinaison des deux modalités biométriques à savoir la forme de la main et l'empreinte palmaire, au niveau des caractéristiques et de la décision. Par contre, la fusion multi-instance est basée sur la combinaison des empreintes palmaires droite et gauche, au niveau du rang.Ces différentes méthodes de fusion ont prouvé leur efficacité en obtenant de meilleurs taux de reconnaissance, qui sont compétitifs par rapport à d'autres approches multimodales de la biométrie de la main. / Biometry is a technology which is based on the personal identification using their physical features (fingerprint, hand geometry, palmprint) and/or behavioral features (voice, dynamic signature). Biometry aims to achieve two important goals in our current life. The first one is to ensure security by eliminating doubt regarding the identity of a person and the second one is to facilitate the identification of individuals. Indeed, this method of identification is increasingly preferred over traditional methods including passwords and badges. The research works of this thesis talk about the personal recognition using hand biometrics. The main objective is to design a multimodal biometric system based on the fusion of hand shape and palmprint modalities.Our first part is to propose a new unimodal biometric system for hand shape verification. In fact, this system is based firstly, on the detection of the best set of keypoints located on the contour of the hand for further SIFT (Scale Invariant Feature Transform) description. On the other hand, a matching refinement based hand region and appearance is proposed in order to refine as much as possible false matched keypoints.Our second part consists in the proposition of a new palmprint identification system. In fact, the sparse representation method is adopted in order to describe the palmprint biometric trait. It is based on the extraction on SIFT descriptors for each detected keypoint.Our third part concerns the proposition of multi-type fusion methods for multimodality, including the multi-representation fusion, the multi-biometric fusion and the multi-instance fusion. Indeed, the multi-representation fusion method is based on the combination of SIFT descriptors and geometrical features of the hand, at score level. The multi-biometric fusion method is based on the fusion of hand shape and palmprint modalities, at feature and decision levels. On the other hand, the multi-instance fusion method is based on the combination of left and right palmprints, at rank level.These different methods of fusion have proven their effectiveness by achieving encouraging recognition rates that are competitive to other popular multimodal hand biometric approaches.

Grounded in the a Pedagogy of Multiliteracies (New London Group, 1996), this dissertation reports on the implementation of a fourth semester French curriculum informed by Cope and Kalantzis's (2000, 2009, 2015) framework of learning by design, with a focus on visual and critical media literacy development to enable intermediate French students to consider multimodal texts from the perspective of consumers as well as producers and to understand the meaning potential that exists between and within the semiotic resources afforded in media production (Nelson and Kern, 2012). Drawing upon "the value of postmethod [and] postlinguistic teaching… which are not looking at language learning in the traditional sense… [but] rather at learners’ acquisition of… the ability to reflect on textualization and contextualization, considering language as one important dimension of semiosis among others" (Nelson and Kern, 2012, p. 61), this dissertation project examined how the frameworks of visual and critical media literacy within the process of design enhanced students' voice and agency in the foreign language classroom. The first inquiry aims to explore if and how a curriculum centered around visual and critical media literacies creates the conditions to 1) foster students' awareness of media ethics in the consumption and production of everyday media, and 2) engage students in a process of reflection upon the meanings created by semiotics resources used in mediated-texts, and their impact on shaping their vision of the world. Findings demonstrated that the implementation of visual and critical media literacy frameworks gave students the opportunity to reflect on their use of media and the ethical implications, and to foster students' greater understanding and interest in self-reflection and considerations of others. The second inquiry aims to demonstrate, through the production of digital storytelling, how instructors might address diversity in foreign language classrooms by 1) allowing students to connect universal themes (e.g. technology, friendship, immigration) with their personal stories, and 2) by giving them the opportunities to display their uniqueness by using their own voices and positioning themselves as participative agents for social change. Findings demonstrated that digital storytelling fosters classroom diversity by allowing the exploration of individual differences and enhancing the understanding of the distinctiveness of every individual. The third inquiry, a case study explores how Digital StoryTelling (DST) 1) contributes to students' understanding of the way semiotic resource choice and orchestration construct layers of meaning and satisfy the purposes of the message conveyed to the audience, and 2) supports students' agency through the process of design. Findings showed the potential of using multimodality projects as they allow students' emerging literacies to take center stage in the foreign language classroom and increase students’ agency and ‘semiotic agility’ (Prior, 2010; Thorne, 2013).

[pt] O texto multimodal é uma realidade no mundo atual, desde a mídia de grande circulação à Internet. O uso de múltiplas modalidades em sala de aula é freqüente, através de recursos como filmes, músicas, e imagens, entre muitos outros. O objetivo deste trabalho é observar o uso de textos multimodais na sala de aula de língua inglesa, assim como na escolha do professor no uso deste ou aquele texto multimodal, de modo que possamos ter uma visão, mesmo que parcial, do papel do texto multimodal no ensino de inglês nesse ambiente, em termos de (1) a visão do professor sobre os objetos de ensino e (2) o texto multimodal em relação aos outros recursos pedagógicos. O trabalho tem duas dimensões que se complementam: a visão teórica sobre a imagem e a visão que tange a pedagogia para o ensino de língua, por uma abordagem multimodal. A perspectiva teórica multimodal do estudo se baseou na teoria da gramática do design visual proposta por Kress e van Leeuwen (1996 e 2001) e em estudos de comunicação visual (Santaella, 2004; Sturken & Cartwright, 2001). A perspectiva multimodal no ensino é pautada pelos trabalhos de Oliveira (2003), Unsworth (2001) e Royce (2002). O estudo discute: (1) o uso das imagens no material didático e a visão das professoras desse uso, e (2) a proposta de levar aprendizes e docentes a se capacitarem a ler textos multimodais, de modo a adquirir a competência comunicativa multimodal. / [en] Multimodal texts have a strong presence in contemporary society, in the mass media and on Internet, for instance. The use of multiple modalities in the language classroom is common, as can be seen in films, songs, images, and many other resources. The purpose of this study is to observe teaching practices with multimodal texts in English language classrooms in order to gain a view of the role of verbal and visual texts in this kind of setting. The focus is on (1) the perception that the teachers have of their choices about what material to use, and (2) their vision of its use multimodal text in relation to teaching resources in general. The theoretical perspective of the study is based on the theory of the grammar of visual design as modeled by Kress and van Leeuwen (1996, 2001). The pedagogical perspective of the study is guided by the work of Unsworth (2001) and Royce (2002), authors that argue for teaching based on a multimodal approach. The study discusses: (1) the use of images in pedadogical material and teacher’s visions of its use, and (2) the notion of making learners and teachers to be able to read multimodal texts, so as to gain multimodal communicative competence.

Philosophiae Doctor - PhD / This study explores Mobility, Identity and Localization of Language in Multilingual Contexts of Urban Lusaka. By examining data from different sites of language practices of Lusaka urbanites, that include, casual and formal conversations, Zambian popular music, computer mediated discourses and advertisements; the study shows how interlocutors creatively draw on their extended communicative repertoire to make meaning, transform social structures/roles and stylize modern identities. Accordingly, the study consolidates the recent sociolinguistic theoretical position that views language as social practice and privileges speakers as social actors in shaping and recreating language. In this regard, the study foregrounds language as localized social practice and argues against the idea that language is homogenous and a bounded system. In doing so, the study adds to recent sociolinguistic theorizing calling for a paradigm shift to language studies. Therefore, the main research question that the study addresses, relates to how Lusaka urban dwellers achieve their mediated agency, voice and actorhood through linguistic choices during interactions in various social contexts of modern Lusaka. In turn, the question relates to how urbanites use language as localized social practice to maintain, transform and reproduce social structures/roles and identities in modern Lusaka. Owing to the type of data the study collected, a multifaceted methodological and analytical approach was employed for both data collection and analysis. Informed by a descriptive research design, the study used focus group discussions and individual key-informant interviews to collect data from casual and formal conversations. Data from Zambian popular music were purposively sampled from Youtube.com and music CDs. In addition, group/individual interviews with musicians were conducted in order to supplement data collected from music CDs and video sources. Data from online discourses were collected from the Facebook platform and from two Zambian based online news blogs, while data from print advertisements were collected through the capturing of images on billboards around Lusaka city as well as advertisements from newspapers and internet sites. Television and radio advertisements were recorded from the Zambia National Broadcasting Corporation TV and radio channels. All the data collected from these sources were generally analyzed using Conversational Analysis, Facework Negotiation Theory, Multimodal Discourse Analysis and its cognate analytical tools such as Resemiotisation, Semiotic Remediation, Intertextuality, Multivocality and Dialogism. The study shows that message consumption is not a function of isolated semiotic resource but a combination of semiotic material drawn from semiotics that people are familiar with. The study thus argues that social meaning is steeped into social and cultural experiences of the speakers and that any study of language practices in such contexts should take into account the multifaceted nature of human communication. Furthermore, the study demonstrates that given the advancements in communication technology and mobility of semiotic resources across modes which have largely contributed to a reconceptualization of the nature of human language, any study of language in social contexts ought to account for other meaning making semiosis in both methodological and approaches to data collection and analysis, respectively. The study further shows how interactants in late modern settings of Lusaka stylize their multiple identities by dissolving the traditional linguistic boundaries through use of the extended linguistic repertoire. In this vein, the study demonstrates that social identity is a dynamic aspect of social life which is actively negotiated and performed through speakers' linguistic choices. In this respect, the study finds that speakers simultaneously stylize translocal hybrid identities which include urban versus rural, modern versus traditional, African versus Christian (Western fused) as well as gendered ones, through their use of different linguistic choices. Furthermore, the study finds that language borders and domains of language use are permeable. In this regard, the study demonstrates how Lusaka urban speakers use localized language forms to colonize the formal spaces thereby challenging the dominant ideologies about language as a fixed, impermeable and a bounded system. In the process of colonizing formal spaces using localized language forms, the study shows how speakers perform acts of humour, role play, face saving, identity and meaning enhancement. In turn these localized repertoires are drawn upon as resources to accomplish different tasks which would not be accomplished if only a 'single' language were to be used. In this regard, the study views language as a resource that transcends the role of meaning making. In addition, the study shows how, through the use of localized repertoires in formal spaces, speakers transform traditions and modernity into a hybrid space which identifies them as having multiple identities. This demonstrates that speakers in such modern settings use language as a resource to accomplish several things at once. It also highlights speakers’ agency in recreating language as well as transforming their social spaces. The findings of the study entail contributions to recent arguments on language that view it not as an autonomous system but rather as embedded in people’s social interactions. It demonstrates that languages have no clear-cut borders.The study also contributes to methodological and analytical approaches to the study of language in recent times. In addition, the study adds new knowledge to our understanding of identity as a performative act which is actively negotiated for as people interact in different social contexts. This implies that identity is not a fixed thing as traditionally conceived. Ultimately, the study calls for a rethinking of our conception of language and identity considering modernity practices.

Magister Artium - MA / This study investigates the use of language and signage in and around Parliament of Botswana by drawing from Linguistic Landscapes (LL) found in the area. The main aim of this study is to explore the symbolic and functional values of signs and languages found in and around Parliament of Botswana. Its main objective is to analyse the visual language in display in and around Parliament looking at the visibility and clarity of language and information/ messages on the signage and to interpret the distribution and frequency of each one of the languages in signage. Secondly, considering that embassies, international companies, organizations and one of the oldest and busiest malls are within the vicinity of Parliament, the study explored whether their presence is captured in the LL.Following Gorter (2006), Shohamy and Gorter (2009) and Shohamy, Ben Rafael and Barni(2010), the data used was drawn from the different types of signs in and around parliament and its neighborhood and included signs with municipal and infrastructural discourses as well as commercial discourses. Using descriptive qualitative research and a thematic approach for data analysis, the study captures the distribution, function, composition and size, clarity of intended information or messages of language on multilingual and monophonic signs.The study gives an insight on the dominance of English against other Botswana languages in LL.In terms of language vitality, the linguistic landscape gives the incorrect picture as if English and Setswana are the only languages spoken in Botswana. Interestingly, Chinese is occasionally found in the linguistic landscape. The presence of Chinese in the shops near Parliament, I argue, is a ploy to attract people to ostensibly “cheap” products. The linguistic landscape shows the apparent contradiction in which even monuments of heroes and pictures of past “Dikgosi”(Chiefs) are constructed in English. The study further reveals even where Setswana is used it is always on bilingual signs and the quality of translation is sometimes poor. The study also suggests that most of the signage was constructed in English and then translated to Setswana. It is not surprising then that the information on the Setswana LL is not always as complete as one on English. In some cases, it says something quite unrelated to the intended message. The different signs and discourses analysed clearly show that people are aware of the signs that surround them and the marketing strategies employed. The signage revealed multiplicity and fusion of discourses with types ranging from low budget to the modern visual signs by individuals to established companies. Both the consumers and the authors are aware of the link between the LL and economics. For this reason, signage is placed where it is expected to achieve maximum visibility and for maximum consumer impact. It is not uncommon to find someone placing their LL on someone else’s, and “ambush” marketing is common place. The diversity of signs used in the study have illustrated and given an insight into the contrasting marketing strategies adopted not only by Batswana, but also by foreign investors, all vying for a piece of space in the Botswana landscape.

Philosophiae Doctor - PhD / This study explores how HIV and AIDS discourse is differently constituted and evaluated across different modes by different communities of speakers in Malawi. Particularly, the study explores how different languages and other social semiotics are used as resources across the different modes. Among other things, it further investigates the implications of the unequal social distribution of modes of communication and semiotic resources in Malawi (eastern region in particular) for the fight against HIV and AIDS. The study employed the Multimodal Discourse Analysis (MDA) approach, the notions of Resemiotisation (Kress and van Leeuwen 2006, Norris 2004, Martin and Rose 2004) and semiotic remediation (Prior and Hengst 2010) in analysing the data. This approach is necessary as the study focuses on HIV and AIDS communication which is essentially multimodal in nature. The study used both quantitative and qualitative methodologies involving questionnaires, interviews, focus group discussions, document analysis and observation of television programs, traditional dances and other modes such as music videos. The study found that different practices have been semiotically remediated and reformulated for health palatability. As a result, taboos have been de-tabooed and technical terms have been ‗untechnicalised‘ so that even ordinary people are able to use health technical terms. The study also shows how cultural practices (such as having ‗live‘ sexual contact with the widow) have been semiotically remediated with the usage of condoms or herbs for cleansing rituals. The study further finds that literacy is not a major challenge for the consumption of HIV and AIDS messages. However the study also shows that wrongly presented messages such as textual overcrowding, usage of proverbs and depiction of western culture in HIV and AIDS messages obscure consumption. In addition the study reveals that proverbs can hardly iv be understood by all consumers and in turn led to division between mostly the older generation and rural who understand and the younger and urban people who have difficulty comprehending the proverbs. Lastly the study finds that some modes of communication did not prove effective, for instance, SMS, television and radio as these do not benefit all consumers as they are socioeconomically determined.

Magister Artium - MA / This study explores the linguistic landscape of Metrorail in the Western Cape, South Africa. The Western Cape is a diverse, multicultural society with a history of colonialism and imperialism. For this reason, the language/s on signage was explored to reveal differences/similarities between the various groups and cultures within society.This kind of investigation entailed consideration of the signage displayed on trains,stations and other railway infrastructure. Thus, data was collected over a three-month period during 2010 which coincided with the FIFA Soccer World Cup, hosted by South Africa. A combined quantitative and qualitative approach for the analysis of data was supplemented with a multimodal, multi-semiotic approach. In addition, interviews were conducted of a cross-section of commuters as a way to give meaning to the analysis of the quantitative and qualitative data. The analysis explored the extent to which multilingualism and multiculturalism are reflected in the linguistic landscape of Metrorail.The focus of the study was on the degree of visibility of the official and non-official languages on signage, as faced by Metrorail commuters. The findings of the study reveal that the interplay between power relations, prestige, symbolic value, identity and vitality in the linguistic landscape of Metrorail results in a somewhat limited display of multilingualism. The findings also reflect the changed language attitudes and perceptions, the maintenance of power relations, the expression of identity, and the desire to be perceived in a certain way, in a broader South African context. Furthermore,the data reveals that the actual linguistic reality does not accurately reflect the aims of the Western Cape language policy in terms of promoting multilingualism. Moreover, it reveals that English is the preferred language of wider communication and it is also the dominant language on the official and non-official signage in the public space. Although the indigenous African languages, along with Afrikaans, are generally neglected in the public space, these languages are widely spoken by Metrorail commuters. The linguistic landscape of Metrorail therefore does not accurately reflect the linguistic reality of the various speech communities in the Western Cape. The linguistic landscape of Metrorail serves to index the broader social developments of the transformed sociolinguistic South African identity.