Förväntat kunskapsbidrag: Min förhoppning är att denna studie kan bidra med kunskaper kring hur lärare i förskoleklass kan använda multimodal kommunikation optimalt i undervisning för att främja alla elevers lärande och utveckling. Syfte och frågeställningar: Studien vill synliggöra olika professioners uppfattningar om användningen av multimodal kommunikation i förskoleklassens undervisning för att främja alla elevers lärande och utveckling.  ● När och hur använder förskoleklasslärare multimodal kommunikation? ● Vilka uppfattningar har de olika professionerna om betydelsen av multimodalkommunikation för alla elevers lärande och utveckling i förskoleklassen? ● Vilka framgångsfaktorer och utmaningar beskrivs av professionerna när multimodalkommunikation ska användas i undervisning? Teori: För att tolka resultatet användes olika begrepp från designorienterat och multimodalt perspektiv på lärande. Perspektivet har varit relevant i denna studie eftersom den syftar till att undersöka hur lärare i förskoleklass använder olika multimodala resurser för att skapa lärmiljöer och processer som främjar alla elevers lärande och utveckling. Metod: Empirin samlades in genom semistrukturerade intervjuer med förskollärare, specialpedagoger och skollogopeder. Respondenterna kommer från tre olika skolor men förskollärarna arbetar alla på samma skola. Empirin bearbetades genom en fenomenografisk analysmodell. Resultat: Förskollärarna använder multimodal kommunikation i förskoleklassens undervisning för att främja språk, kunskap, lustfyllt lärande, sociala utveckling och grupplärande. Dock används verktygen begränsat och det finns osäkerhet kring hur de kan främja lärande, delaktighet och inkludering av elever i språklig sårbarhet. Professionerna har olika syn på hur verktygen ska anpassas, vilket skapar spänning kring användningen som generell insats eller som stödstrukturer för elever i språklig sårbarhet. Resultatet visar också att stöd från elevhälsan och positiv sammanhållning är viktiga för att stärka förskollärarnas kunskaper och motivation. Specialpedagogiska implikationer: En kommunikativt tillgänglig lärmiljö i förskoleklassen handlar om att skapa en lärmiljö som kan stödja lärande och delaktighet för elever i språklig sårbarhet. För att skapa en inkluderande och likvärdig lärmiljö för alla elever i förskoleklass är det viktigt att öka kunskap och medvetenheten hos förskoleklasslärare om hur man kan använda multimodal kommunikation optimalt i undervisningen. Det är även viktigt att ha fokus på elever i behov av stöd och att öka samarbetet mellan professioner för att tydliggöra skolkulturen kring förväntningar på dessa elever. Detta är också betydelsefullt för att stärka kvaliteten på undervisningen i förskoleklassen.

[pt] Grandes quantidades de vídeo são carregadas em plataformas de hospedagem de vídeo a cada minuto. Esse volume de dados apresenta um desafio no controle do tipo de conteúdo enviado para esses serviços de hospedagem de vídeo, pois essas plataformas são responsáveis por qualquer mídia sensível enviada por seus usuários. Nesta dissertação, definimos conteúdo sensível como sexo, violencia fisica extrema, gore ou cenas potencialmente pertubadoras ao espectador. Apresentamos um conjunto de dados de vídeo sensível para classificação binária de vídeo (se há conteúdo sensível no vídeo ou não), contendo 127 mil vídeos anotados, cada um com seus embeddings visuais e de áudio extraídos. Também treinamos e avaliamos quatro modelos baseline para a tarefa de detecção de conteúdo sensível em vídeo. O modelo com melhor desempenho obteve 99 por cento de F2-Score ponderado no nosso subconjunto de testes e 88,83 por cento no conjunto de dados Pornography-2k. / [en] Massive amounts of video are uploaded on video-hosting platforms every minute. This volume of data presents a challenge in controlling the type of content uploaded to these video hosting services, for those platforms are responsible for any sensitive media uploaded by their users. There has been an abundance of research on methods for developing automatic detection of sensitive content. In this dissertation, we define sensitive content as sex, extreme physical violence, gore, or any scenes potentially disturbing to the viewer. We present a sensitive video dataset for binary video classification (whether there is sensitive content in the video or not), containing 127 thousand tagged videos, Each with their extracted audio and visual embeddings. We also trained and evaluated four baseline models for the sensitive content detection in video task. The best performing model achieved 99 percent weighed F2-Score on our test subset and 88.83 percent on the Pornography-2k dataset.

[ES] La COVID-19 ha acelerado el proceso de adaptación al trabajo telemático donde el entorno académico también ha sufrido cambios. La comunidad investigadora ha normalizado la participación en congresos a través del ordenador. El objetivo de este estudio es analizar los elementos que intervienen en el discurso oral en las comunicaciones en congresos digitales académicos para identificar los patrones de comunicación más empleados por los investigadores. Desde inicios del siglo XXI, diversos autores han explorado este género discursivo al considerarlo el homólogo oral del artículo científico. Estos mismos autores demandan más atención a los géneros orales, al considerarlos infraestudiados en comparación con los géneros académicos escritos. Para el análisis se utiliza el Análisis del Discurso Multimodal (ADM). Esta metodología entiende que, solo considerando todos los elementos del discurso se puede tener una idea global de cómo es la comunicación. De este modo, considerar únicamente lo que se dice (el lenguaje) no permite descifrar todo el significado. Por ello, los autores del ADM proponen analizar elementos del aspecto visual, el lenguaje o los gestos del orador. Entre los resultados, destaca el protagonismo de las diapositivas en el discurso, el desarrollo de un patrón para la estructura del discurso y la poca relevancia de los gestos, que apenas son percibidos por la cámara. Estos datos permiten conocer los modelos de comunicación en los congresos digitales, así como atender a los elementos más urgentes que hay que tratar para mejorar la comunicación efectiva a través del ordenador y la cámara web. / [CA] La COVID-19 ha accelerat el procés d'adaptació al treball telemàtic on l'entorn acadèmic també ha patit canvis. La comunitat investigadora ha normalitzat la participació en congressos a través de l'ordinador. L'objectiu d'aquest estudi és analitzar els elements que intervenen en el discurs oral en les comunicacions en congressos digitals acadèmics per a identificar els patrons de comunicació més utilitzats pels investigadors. Des d'inicis del segle XXI, diversos autors han explorat aquest gènere discursiu al considerar-lo l'homòleg oral de l'article científic. Aquests mateixos autors demanden més atenció als gèneres orals, en considerar-los poc estudiats en comparació amb els gèneres acadèmics escrits. Per a l'anàlisi s'utilitza l'Anàlisi del Discurs Multimodal (ADM). Aquesta metodologia entén que, només considerant tots els elements del discurs es pot tindre una idea global de com és la comunicació. D'aquesta manera, considerar únicament el que es diu (el llenguatge) no permet desxifrar tot el significat. Per això, els autors de l'ADM proposen analitzar elements com l'aspecte visual, el llenguatge o els gestos de l'orador. Entre els resultats, destaca el protagonisme de les diapositives a les comunicacions, el desenvolupament d'un patró per a l'estructura del discurs i la poca rellevància dels gestos, que a penes són percebuts per la càmera. Aquestes dades permeten conèixer els models de comunicació en els congressos digitals, així com atendre els elements més urgents que cal tractar per a millorar la comunicació efectiva a través de l'ordinador i la càmera web. / [EN] The Covid-19 pandemic has accelerated the adaptation to remote work, leading to significant changes in the academic environment. The academic community has normalized participation in conferences through computer screens. The aim of this study is to analyze the elements involved in oral discourse during digital conference papers to identify the communication patterns most commonly used by researchers. Since the early 21st century, several authors have explored this discursive genre, considering it the oral counterpart of scientific articles. These authors advocate for more attention to oral genres, considering them understudied in comparison to written academic genres. This study employs Multimodal Discourse Analysis (MDA). This methodology understands that a comprehensive understanding of communication can only be achieved by considering all elements of discourse. Merely focusing on what is said (language) does not reveal the full meaning. Therefore, MDA advocates for the analysis of visual aspects, language, and gestures of the speaker. The results highlight the prominence of slides in discourse, the development of a pattern for discourse structure, and the limited relevance of gestures, which are barely perceived by the camera. These findings provide insights into communication models in digital conferences and draw attention to the most pressing elements that need to be addressed to enhance effective communication through computers and webcams. / López Navarrete, AJ. (2024). ¿Cómo se comunica la investigación a través de la pantalla? Análisis del discurso multimodal en las comunicaciones en congresos digitales en el área de comunicación [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/202718

This thesis draws critically on approaches to globalisation, translation and advertising to analyse current translation practices using multimodal advertising texts. The purpose of this study is both to show evidence of the need to approach advertising translation from an interdisciplinary and intersemiotic perspective, and to remark on the need to incorporate such an approach into translation training and practice. It is developed in the broader context of globalisation and its impacts on socio-economic changes and cultural interaction worldwide. The intensification of global communications together with technological advances enhance the steady flow of knowledge and information, whilst at the same time there is a tendency to accentuate commonalities between cultures, clearly illustrated by standardised strategies used in global advertising campaigns. However, one key obstacle in the spread of standardised messages is language diversity, something that has caused translation to gain an active role in global advertising since it facilitates dialogue between global companies and international consumers; thus, translation has not only become an activity that overcomes linguistic and cultural barriers but also a commercial tool. This thesis focuses on the translation of advertisements as an intentional communication act that involves a team process into which translation may be integrated. Translation encompasses concepts that go beyond linguistic matters such as contextual issues and extralinguistic communication sign systems. The translation of advertisements is a growing area of study, with current research mainly limited to the study of linguistic matters. Consequently, in order to improve the accuracy and effectiveness of multimodal translations (text and images in this study) in a professional context, this research incorporates concepts and knowledge from visual analysis and advertising. This thesis suggests a multimodal method of analysis consisting of different phases integrating visual analysis as a part of the translation process. The 2 proposed methodology is illustrated by a corpus of Spanish-English pairs of standardised print advertisements from the beauty and cosmetics sector. The conclusion is that it is vital to consider the extralinguistic aspects surrounding a multimodal text from both a descriptive and critical perspective in order to read connotative information from words, images and the text-image unit. Translation training might also benefit from including this multimodal approach and further collaborative work must be done between translators and marketers to better understand and consolidate the role of translation in this area.

Significant research efforts have been directed into smart home environments in the last decade creating abundant opportunities for the broader home services ecosystem to foster a wide range of innovative services. Research interest has been given on automatic identification and tracking of people within the home environment to support customised services such as care services for elderly and disadvantaged people to enable and prolong their independent living. Although various approaches have been proposed to tackle this problem, solutions still remain elusive due to various reasons (e.g. user acceptance). Literature reviews have indicated the need for an advanced non-tagged identification and tracking approach that is capable to provide the infrastructure support for realisation of context-aware services, satisfy users’ needs, and deal with the complexity of smart home environmental conditions. The aim of this study is to develop and implement an advanced approach that is capable to accurately detect, identify, and track people within opportune and calm home environment to be used as infrastructure for various application domains such as assisted living, healthcare, security and energy management. Accordingly, a novel multimodal approach for non-tagged human identification and tracking within home environment is proposed. The proposed approach combined floor pressure and PIR sensors through unique designed integration strategy aiming to merge the advantages of the two sensor types and overcome or minimise their weaknesses. The designed strategy enabled the PIR output signal pattern to afford explicit information indicating a person’s body surface area (size/shape). This information enhanced the identification accuracy, facilitated the custom designed smart floor, and reduced the overall cost. The conceptual framework of the proposed approach/strategy encompassed two key stages, hardware system design and implementation, and data processing. The hardware system design included the custom designed PIR and smart floor units. A test bed was designed and implemented for supporting the research studies, including proof of concept, concept demonstration, experimental and test cases studies. Data processing system has divided into different stages to accomplish the identification and tracking goals. First, the interested patterns were segmented and generated with threshold edge detection method and advanced pattern generation algorithm respectively. Second, limited set of features were extracted and selected from each pattern including ground reaction force GRF, gait, and body size/shape (PIR) features. Third, these features were merged at different fusion level, namely, feature-level and decision-level to provide comprehensive description about the person’s identity. Fourth, MLPNN multiclass classifier was adopted to process the feature vectors and recognise the person’s identity. Finally, the footstep patterns were tracked using weighted centroid tracking technique, in addition to MLPNN classifier to handle the footsteps association problems. Four test cases were designed and carried out to demonstrate, test, and evaluate the feasibility and effectiveness of the proposed non-tagged identification and tracking strategies/approach. The assessment outcomes have shown the potential of the proposed multimodal approach as an advanced strategy for implementation of an indoor non-tagged human identification and tracking system and to be used as infrastructure for supporting the delivery of various types of smart services within the smart home environments. In summary, the proposed multimodal approach has the potential to: (1) Identify up to 5 persons successfully with minimum 98.8% correct classification rate without tag, (2) detect, locate, and track multiple persons successfully without tag and the location error no more than 11.76 cm, approximately 1.5 times better in accuracy than the original set target (i.e. 30 cm), and (3) able to handle various tracking difficulties and solve 97.5% of data association problems.

This thesis is about the resources that speakers can draw on when producing actions, both verbal and non-vocal. It considers how identity categories, gaze and touch can contribute to action in everyday interactions. The study stemmed from an interest in how lesbian identity is made relevant by lesbian speakers in everyday co-present interaction. A corpus of approximately 23.5 hours of video-recordings was gathered: households self-designated as lesbian (including couples, families, and housemates) video recorded some of their everyday interactions (including mealtimes, watching television, and playing board games). Using the tools of Conversation Analysis and working with the video recordings and transcripts of the interactions, several ways of making a lesbian identity relevant through talk were identified. As the analysis progressed, it was found that many references to sexual identity were produced fleetingly; they were not part of or integral to the ongoing talk, and were not taken up as a topic by participants. Rather, this invoking of a participant s sexual identity appears to contribute to a particular action that is being produced. It was found that invokings of other identities, for example relating to occupation, nationality, and race, worked in a similar way, and this is explored in relation to explanations and accounts. Where the first half of the thesis focuses on verbal invokings of identity in relation to action, the second half of the thesis considers some of the non-vocal resources that participants incorporate into their actions. It was found that when launching a topic related to something in the immediate environment, speakers can use gaze to ensure recipiency. Also, when producing potentially face-threatening actions such as teases, reprimands or insults, speakers can use interpersonal touch to mitigate the threat. In addition to showing how identities can be made relevant in everyday interaction, the findings of this thesis highlight the complexity of action design, and that in co-present interaction the physical resources available to participants also need to be taken into account.

In the past few years, e-government has been a topic of much interest among those excited about the advent of Web technologies. Due to the growing demand for effective communication to facilitate real-time interaction between users and e-government applications, many governments are considering installing new tools by e-government portals to mitigate the problems associated with user – interface communication. Therefore, this study is to indicate the use of multimodal metaphors such as audio-visual avatars in e-government interfaces; to increase the user performance of communications and to reduce information overload and lack of trust that is common with many e-government interfaces. However, only a minority of empirical studies has been focused on assessing the role of audio-visual metaphors in e-government. Therefore, the subject of this thesis’ investigation was the use of novel combinations of multimodal metaphors in the presentation of messaging content to produce an evaluation of these combinations’ effects on the users’ communication performance as well as the usability of e-government interfaces and perception of trust. The thesis outlines research comprising three experimental phases. An initial experiment was to explore and compare the usability of text in the presentation of the messaging content versus recorded speech and text with graphic metaphors. The second experimental was to investigate two different styles of incorporating initial avatars versus the auditory channel. The third experiment examined a novel approach around the use of speaking avatars with human-like facial expressions, obverse speaking avatars full body gestures during the presentation of the messaging content to compare the usability and communication performance as well as the perception of trust. The achieved results demonstrated the usefulness of the tested metaphors to enhance e-government usability, improve the performance of communication and increase users’ trust. A set of empirically derived ground-breaking guidelines for the design and use of these metaphors to generate more usable e-government interfaces was the overall provision of the results.

This empirical work aims to investigate the impact of using multimodal communication metaphors on e-learning systems’ usability, overall user experience and affective state. The study proposed a triple evaluation approach to avoid the problem of conventional assessment relying only on usability measurements of efficiency, effectiveness and user satisfactions. Usability in that sense refers only to the functionality and pragmatic side of the product and neglects other aspects of the system. Learning is a cognitive and repetitive task, requiring learners’ attention as well as their interest. Therefore, when delivering content, in addition to the pragmatic functionality, an e-learning system should provide a constructive overall user experience and positive affective state. Doing so will ensure user engagement, facilitate the learning process and increase learners’ performance. The impact of using five different communication metaphors was evaluated in three dimensions using the proposed approach. Within the usability dimension, the evaluation criteria involved measuring system efficiency, effectiveness, user satisfaction and learning performance. Within the user experience dimension, the evaluation criteria involved measuring pragmatic aspects of the user experience, the hedonic aspects of user experience in terms of stimulation as well as identification and the overall system attractiveness. Within the affective state dimension a self-assessments manikin technique was used in conjunction with biofeedback measurements, and users’ valence, arousal and dominance were measured. The study found that system attractiveness and the hedonic user experience had a profound impact on users’ learning performance and attitude toward the tested system. Furthermore, they influenced users’ views and judgement of the system and its usability. The communication metaphors were not equal in their influence within the evaluation criteria. Empirically derived guidelines were produced for the use and integration of these metaphors in e-learning systems. The outcome of the study highlights the need to use the triple evaluation approach in the assessment of e-learning interfaces prior to their release for better adoption and acceptance by end users.

Examensarbetet handlar om förskollärarens förhållningssätt och kunskap kring användandet av lärplattan i förskolan. Syftet med arbetet är att undersöka vad som finns tillgängligt för förskollärare att ta del av för att få ökad kunskap kring lärplattan som verktyg för barns språkutveckling. I arbetet undersöks hur Litteraturhuset Trampolins ryggsäckar påverkar barns möjligheter till språkutveckling genom lärplattan samt kommunens fortbildning och dess utformning. Genom en innehållsanalys av Litteraturhuset Trampolins ryggsäckar, intervjuer med förskollärare som lånat en sådan ryggsäck samt genom en observation av kommunens fortbildning har empiri samlats in och analyserats utifrån ett multimodalt perspektiv.   I arbetets resultat belyses förskollärares upplevelse om att det är värdefullt att kunna ta del av de olika möjligheter som finns tillgängliga genom Trampolin och fortbildning i kommunen. Det blir meningsfullt dels för förskollärare som får möjlighet att utveckla sina egna kunskaper och inspireras till att integrera lärplattan i förskolan dels för barnen när förskollärarens kunskap och förståelse för barns lärande med IT som verktyg förbättras. Detta skapar en diskussion om didaktik kring hur Litteraturhuset Trampolins ryggsäckar och dess färdiga koncept påverkar barns språkutveckling samt hur utformningen av fortbildning inom området bidrar till förskollärares kunskap och engagemang kring lärplattan som pedagogiskt verktyg. Arbetets empiriska del visar att förskollärare önskar mer kunskap om integrering av lärplattan i förskolan vilket stöds av forskning inom området.

No description available.