Submitted by Fernando Souza (fernandoafsou@gmail.com) on 2017-08-09T11:20:43Z No. of bitstreams: 1 arquivototal.pdf: 4153663 bytes, checksum: 1498bea749a2cf07373bbe25692a39fd (MD5) / Made available in DSpace on 2017-08-09T11:20:44Z (GMT). No. of bitstreams: 1 arquivototal.pdf: 4153663 bytes, checksum: 1498bea749a2cf07373bbe25692a39fd (MD5) Previous issue date: 2016-07-28 / The discursive multimodality is a common linguistic phenomenon to any genre, and it is the union of more than one semiotic resource in a enunciation event. It is something that requires an understanding of semiotic materiality of texts paying attention to the understanding of all resources used in its constitution. It is something that requires a comprehension of semiotic materiality of texts paying attention to the understanding of all resources used in its constitution. Thus, this research focuses on such epistemological approach and it has as principal goal to investigate the implementation of a multimodal model as didactic proposal to the approach of the multimodal genres in Portuguese Language Teaching. Thus, this research focuses on this epistemological approach and aims to investigate the implementation of a multimodal model as didactic proposal to address the multimodal genres in Portuguese Language Teaching classes. The central object of the investigation is a field research developed in a class of ninth year of the Primary School in a public school in collaboration with the titular teacher of the cited class. The generation of data corresponded to three specific moments: the sequences of lessons, which consists in the planning and realization of the didactic project with the multimodal genres; the oral and written texts produced by the collaborator teacher about her participation in the research; and the textual productions of the students of the investigated class.The central object of this presente research is a research developed in a class of ninth year in a public school, that was done with the collaboration of the teacher of that class. The research data is based on three specific moments: the sequences of lessons, which is the planning and realization of the educational project with multimodal genres; oral and written texts produced by the titular teacher about her participation in the research; and textual productions of the students of the investigated group. Based on multimodal theoretical model, a sequence of three classes were developed and applied, in order to develop visual literacy. The main results point to the identification of multimodal model as an essential educational tool in the visual literacy process to develop a educational attainment. This fact leads to another: the perception of a critical and more aware use of language materialized by the interaction of verbal and image in the student productions. In addition, evaluations of own pedagogical, made by the titular teacher, demostrated a positive assessment performed experience while also unveil epistemological difficulties during the research. Therefore, we can state the positive contribution of the multimodal teaching project to approach a practice of reading and production of imagistic texts in a socially conscious way. This fact corraborate to the thesis that participation in an educational project through multimodal model enables contact with aspects of the production of multimodal genres and with its own meta-language developing Visual Literacy and critical training learner. / A multimodalidade discursiva é um fenômeno linguístico comum a todo gênero textual e que consiste na união de mais de um recurso semiótico em uma composição enunciativa. É algo que requer uma compreensão da materialidade semiótica dos textos atentando para o entendimento de todos os recursos usados em sua constituição. Assim, a presente pesquisa centra-se nessa abordagem epistemológica e tem como principal objetivo investigar a implementação de um modelo multimodal como proposta didática para a abordagem dos gêneros multimodais no Ensino de Língua Portuguesa. O estudo encontra embasamento teórico nos postulados sobre Multimodalidade e Letramento Visual, especialmente, nos trabalhos de Almeida (2009; 2011), Kress e van Leeuwen (1996) e Stokes (2002) e nas concepções acerca da Linguística Aplicada e dos gêneros textuais com os estudos de Moita Lopes (2006; 2009; 2013), Bronckart (2006), Dionísio (2005) e Balloco (2005). O objeto central da investigação é uma pesquisa de campo desenvolvida em uma turma de nono ano do Ensino Fundamental de uma escola pública em colaboração com a professora titular da referida turma. A geração dos dados correspondeu a três momentos específicos: as sequências de aulas, que consiste no planejamento e realização do projeto didático com os gêneros multimodais; os textos orais e escritos produzidos pela professora colaboradora sobre sua participação na pesquisa; e as produções textuais dos alunos da turma investigada. Foram elaboradas e aplicadas, à luz do modelo multimodal, três sequências de aulas, tendo em vista o desenvolvimento do letramento visual. À luz da análise desenvolvida, os principais resultados apontam para a identificação do modelo multimodal como uma ferramenta didática essencial no processo de letramento visual e na formação crítica do aluno. Essas evidências conduzem a outro fato constatado na análise - à percepção de uma linguagem crítica e mais consciente em relação ao seu uso materializada pela interação do verbal e da imagem, nas produções dos alunos. Além disso, as avaliações do próprio fazer pedagógico, feitas pela professora colaboradora, revelam uma avaliação positiva da experiência realizada, embora desvele também dificuldades de ordem epistemológica no decorrer da pesquisa. Portanto, como desdobramentos de todas as análises, ficou constatado a contribuição positiva do projeto didático para a abordagem de uma prática de leitura e produção de textos imagéticos de forma socialmente consciente. Esse fato serviu para confirmar a tese de que a participação em um projeto de ensino por meio do modelo multimodal possibilita o contato com aspectos da produção dos gêneros multimodais e com uma metalinguagem própria desse gênero contribuindo para o desenvolvimento do Letramento Visual e para a formação crítica do aprendiz.

The main question of this study was: How and in what way can a multimodal semiotic visual analysis model be developed and used for contributing to the analysis and understanding of the manner in which the Men’s Health (South Africa) magazine – as a case study – represents and constructs masculinities in South Africa? The following three subsidiary research questions were formulated to address this topic: • What is the literature revealing with reference to the media as producers of meaning in relation to masculinity and visual texts? • How and in which way can a semiotic visual analysis multimodal model be developed with the purpose of contributing to the analysis of visual texts? • What is the outcome of the visual analysis multimodal model with reference to the case study about the representation and construction of masculinities in visual texts in MH? The first aim of this research was to establish an overview of masculinities and to explore the visual representation of masculinity with reference to mediation, reality, and ideology in the media. With reference to the media as producers of meaning in relation to masculinity and visual texts, a semiotic visual analysis and social semiotics were used to unpack culture as a site of the production of meanings. The media is one of the main sources from which men receive their entertainment and information about the world. In this sense, the media makes sense of the world. Mass media plays a key role in discourse and constructing the relationships between reality and ideology. During this construction, the media reflects on existing opinions and attitudes in society. A quantitative content analysis and a qualitative semiotic multimodal visual analysis were conducted on 27 visual texts purposively selected from MH to include editions from July 2010 to June 2011. This population covered 12 front covers, 12 editorials and three flip covers. The developed visual multimodal model was tested qualitatively on nine visual texts since these texts included the front covers, flip covers and editorials of the three editions with flip covers. v A second major aim of the study was to establish the way in which a semiotic visual analysis multimodal model needed to be developed and used for analysing visual texts, as well as for analysing the visual texts according to the multimodal model in order to understand how the multimodality and social semiotic resources were applied in MH to represent and construct masculinities. The rationale for the development and design of this model was based on the premise that a basic understanding of semiotics and visual language was needed. Without such an understanding, the vast amounts of visual messages that confront the reader would remain incomprehensible. Consequently, a productive dialogue in relation to visual communication cannot take place. The multimodal model developed in this thesis highlights visual text layout, in conjunction with language-in-use, that does not occur in isolation and that is deeply reliant on other forms of making meaning. The heptagon multimodal model consists of concept maps of the six functions of the designed hexagon model. This multimodality approach includes analysing simultaneously occurring semiotics and their various roles in conjunction with detailed, all-inclusive discourses. In the quantitative content analysis and the qualitative multimodal semiotic analysis, the six components of the developed heptagon model (visual grammar, positioning, typography, colour, modality, and iconography) are illustrated. The quantitative research supported the main research design, i.e. the qualitative multimodal semiotic analysis. It is envisaged that the development and construction of a multimodal semiotic model will make a contribution to the scholarly field of semiotic analysis. By discussing the fluidity of the variations of masculinities and male identities, by giving a brief overview of the role of the media in constructing masculinities, and by focusing on the discourses that took place in MH, the researcher creates an awareness of the inherited patriarchal masculinities by recommending envisioned masculinities to be inclusive as a component of the solution. This approach is illustrated by the use and findings of the multimodal semiotic visual analysis. / Communication Science / D. Litt. et. Phil

The main question of this study was: How and in what way can a multimodal semiotic visual analysis model be developed and used for contributing to the analysis and understanding of the manner in which the Men’s Health (South Africa) magazine – as a case study – represents and constructs masculinities in South Africa? The following three subsidiary research questions were formulated to address this topic: • What is the literature revealing with reference to the media as producers of meaning in relation to masculinity and visual texts? • How and in which way can a semiotic visual analysis multimodal model be developed with the purpose of contributing to the analysis of visual texts? • What is the outcome of the visual analysis multimodal model with reference to the case study about the representation and construction of masculinities in visual texts in MH? The first aim of this research was to establish an overview of masculinities and to explore the visual representation of masculinity with reference to mediation, reality, and ideology in the media. With reference to the media as producers of meaning in relation to masculinity and visual texts, a semiotic visual analysis and social semiotics were used to unpack culture as a site of the production of meanings. The media is one of the main sources from which men receive their entertainment and information about the world. In this sense, the media makes sense of the world. Mass media plays a key role in discourse and constructing the relationships between reality and ideology. During this construction, the media reflects on existing opinions and attitudes in society. A quantitative content analysis and a qualitative semiotic multimodal visual analysis were conducted on 27 visual texts purposively selected from MH to include editions from July 2010 to June 2011. This population covered 12 front covers, 12 editorials and three flip covers. The developed visual multimodal model was tested qualitatively on nine visual texts since these texts included the front covers, flip covers and editorials of the three editions with flip covers. v A second major aim of the study was to establish the way in which a semiotic visual analysis multimodal model needed to be developed and used for analysing visual texts, as well as for analysing the visual texts according to the multimodal model in order to understand how the multimodality and social semiotic resources were applied in MH to represent and construct masculinities. The rationale for the development and design of this model was based on the premise that a basic understanding of semiotics and visual language was needed. Without such an understanding, the vast amounts of visual messages that confront the reader would remain incomprehensible. Consequently, a productive dialogue in relation to visual communication cannot take place. The multimodal model developed in this thesis highlights visual text layout, in conjunction with language-in-use, that does not occur in isolation and that is deeply reliant on other forms of making meaning. The heptagon multimodal model consists of concept maps of the six functions of the designed hexagon model. This multimodality approach includes analysing simultaneously occurring semiotics and their various roles in conjunction with detailed, all-inclusive discourses. In the quantitative content analysis and the qualitative multimodal semiotic analysis, the six components of the developed heptagon model (visual grammar, positioning, typography, colour, modality, and iconography) are illustrated. The quantitative research supported the main research design, i.e. the qualitative multimodal semiotic analysis. It is envisaged that the development and construction of a multimodal semiotic model will make a contribution to the scholarly field of semiotic analysis. By discussing the fluidity of the variations of masculinities and male identities, by giving a brief overview of the role of the media in constructing masculinities, and by focusing on the discourses that took place in MH, the researcher creates an awareness of the inherited patriarchal masculinities by recommending envisioned masculinities to be inclusive as a component of the solution. This approach is illustrated by the use and findings of the multimodal semiotic visual analysis. / Communication Science / D. Litt. et. Phil

Indiana University-Purdue University Indianapolis (IUPUI) / With the increasing availability of Electronic Health Records (EHRs) and advances in deep learning techniques, developing deep predictive models that use EHR data to solve healthcare problems has gained momentum in recent years. The majority of clinical predictive models benefit from structured data in EHR (e.g., lab measurements and medications). Still, learning clinical outcomes from all possible information sources is one of the main challenges when building predictive models. This work focuses mainly on two sources of information that have been underused by researchers; unstructured data (e.g., clinical notes) and a patient network. We propose a novel hybrid deep learning model, DeepNote-GNN, that integrates clinical notes information and patient network topological structure to improve 30-day hospital readmission prediction. DeepNote-GNN is a robust deep learning framework consisting of two modules: DeepNote and patient network. DeepNote extracts deep representations of clinical notes using a feature aggregation unit on top of a state-of-the-art Natural Language Processing (NLP) technique - BERT. By exploiting these deep representations, a patient network is built, and Graph Neural Network (GNN) is used to train the network for hospital readmission predictions. Performance evaluation on the MIMIC-III dataset demonstrates that DeepNote-GNN achieves superior results compared to the state-of-the-art baselines on the 30-day hospital readmission task. We extensively analyze the DeepNote-GNN model to illustrate the effectiveness and contribution of each component of it. The model analysis shows that patient network has a significant contribution to the overall performance, and DeepNote-GNN is robust and can consistently perform well on the 30-day readmission prediction task. To evaluate the generalization of DeepNote and patient network modules on new prediction tasks, we create a multimodal model and train it on structured and unstructured data of MIMIC-III dataset to predict patient mortality and Length of Stay (LOS). Our proposed multimodal model consists of four components: DeepNote, patient network, DeepTemporal, and score aggregation. While DeepNote keeps its functionality and extracts representations of clinical notes, we build a DeepTemporal module using a fully connected layer stacked on top of a one-layer Gated Recurrent Unit (GRU) to extract the deep representations of temporal signals. Independent to DeepTemporal, we extract feature vectors of temporal signals and use them to build a patient network. Finally, the DeepNote, DeepTemporal, and patient network scores are linearly aggregated to fit the multimodal model on downstream prediction tasks. Our results are very competitive to the baseline model. The multimodal model analysis reveals that unstructured text data better help to estimate predictions than temporal signals. Moreover, there is no limitation in applying a patient network on structured data. In comparison to other modules, the patient network makes a more significant contribution to prediction tasks. We believe that our efforts in this work have opened up a new study area that can be used to enhance the performance of clinical predictive models.