I den här uppsatsen analyseras Södertörn högskolas kampanj jag är lärare, som lanserades år 2014, utifrån de texter och bilder används för att förmedla dess budskap. Texten analyseras med hjälp av Systemisk funktionell grammatik (SFG) och de visuella elementen med Systemisk funktionell Multimodal Diskusanalys (SF-MDA). Det material som ingår i uppsatsen är litet och undersöks djupgående. Uppsatsen drar slutsatser om hur kampanjen låter bild och text samverka, både för att förhöja enade budskap och för att enhetligt presentera skilda budskap.

Bakgrund: I litteraturen finns smärta, livskvalitet och arbetsåtergång beskrivet och utvärderat i nära anslutning till avslutad multimodal rehabilitering (MMR). Det saknas dock långtidsuppföljningar på mer än två år på patienter som genomgått MMR.Metod: Data om smärta (numerisk skala), livskvalitet (EuroQol 5 Dimensions (EQ5D)), aktivitetsbegränsning p.g.a. smärta (Multidimensional Pain Inventory (MPI)) och arbetsåtergång samlades in från frågeformulär. Datan jämfördes med insamlad data före MMR och efter MMR fyra år tillbaka i tiden för att undersöka om det förelåg skillnader. Datan analyserades med Wilcoxons tecken-rangtest och Chi2.Resultat: Undersökningsgruppen bestod av 31 personer. Resultatet visade statistiskt signifikanta skillnader i smärta och livskvalitet efter genomgången multimodal rehabilitering och även fyra år senare p<0,033. Det förelåg ingen signifikant skillnad i Delskala 3 MPI som mäter kontroll. I de övriga delskalorna som mäter minskad smärta, minskning av hur smärtan inverkar och minskning av känslomässig obalans visade statistisk signifikans direkt efter och fyra år efter avslutad MMR p<0,001.Konklusion: Resultatet i studien visade att det förelåg statistiskt signifikanta skillnader i minskad smärta och ökad livskvalitet hos en grupp deltagare med långvarig ryggsmärta direkt efter avslutad MMR2 och även fyra år senare. Trots att deltagarna upplever minskad smärta och ökad livskvalitet upplever de inte att de har kontroll över sitt liv, problem, värk och stressade situationer.

Mobile robots for rough terrain are of interest to researchers as their range of possible uses is large, including exploration activities for inhospitable areas on Earth and on other planets and bodies in the solar system, searching in disaster sites for survivors, and performing surveillance for military applications. Nature generally achieves land movement by walking using legs, but additional modes such as climbing, jumping and rolling are all produced from legs as well. Robotics tends not to use this integrated approach and adds additional mechanisms to achieve additional movements. The spherical device described within this thesis, called Jollbot, integrated a rolling motion for faster movement over smoother terrain, with a jumping movement for rougher environments. Jollbot was developed over three prototypes. The first achieved pause-and-leap style jumps by slowly storing strain energy within the metal elements of a spherical structure using an internal mechanism to deform the sphere. A jump was produced when this stored energy was rapidly released. The second prototype achieved greater jump heights using a similar structure, and added direction control to each jump by moving its centre of gravity around the polar axis of the sphere. The final prototype successfully combined rolling (at a speed of 0.7 m/s, up 4° slopes, and over 44 mm obstacles) and jumping (0.5 m cleared height), both with direction control, using a 0.6 m spherical spring steel structure. Rolling was achieved by moving the centre of gravity outside of the sphere’s contact area with the ground. Jumping was achieved by deflecting the sphere in a similar method to the first and second prototypes, but through a larger percentage deflection. An evaluation of existing rough terrain robots is made possible through the development of a five-step scoring system that produces a single numerical performance score. The system is used to evaluate the performance of Jollbot.

This study is an investigation into the science literacy of college genetics students who were given a modified curriculum to address specific teaching and learning problems from a previous class. This study arose out of an interest by the professor and researcher to determine how well students in the class Human Genetics in the 21st Century responded to a reorganized curriculum to address misconceptions that were prevalent after direct instruction in the previous year's class. One of the components to the revised curriculum was the addition of a multimodal representation requirement as part of their normal writing assignments. How well students performed in these writing assignments and the relationship they had to student learning the rest of the class formed the principle research interest of this study. Improving science literacy has been a consistent goal of science educators and policy makers for over 50 years (DeBoer, 2000). This study uses the conceptualization of Norris and Phillips (2003) in which science literacy can be organized into both the fundamental sense (reading and writing) and the derived sense (experience and knowledge) of science literacy. The fundamental sense of science literacy was investigated in the students' ability to understand and use multimodal representations as part of their homework writing assignments. The derived sense of science literacy was investigated in how well students were able to apply their previous learning to class assessments found in quizzes and exams. This study uses a mixed-methods correlational design to investigate the relationship that existed between students' writing assignment experiences connected to multimodal representations and their academic performance in classroom assessments. Multimodal representations are pervasive in science literature and communication. These are the figures, diagrams, tables, pictures, mathematical equations, and any other form of content in which scientists and science educators are communicating ideas and concepts to their audience with more than simple text. A focused holistic rubric was designed in this study to score how well students in this class were able to incorporate aspects of multimodality into their writing assignments. Using these scores and factors within the rubric (ex. Number of original modes created) they were correlated with classroom performance scores to determine the strength and direction of the relationship. Classroom observations of lectures and discussion sections along with personal interviews with students and teaching assistants aided the interpretation of the results. The results from the study were surprisingly complex to interpret given the background of literature which suggested a strong relationship between multimodal representations and science learning (Lemke, 2000). There were significant positive correlations between student multimodal representations and quiz scores but not exam scores. This study was also confounded by significant differences between sections at the beginning of the study which may have led to learning effects later. The dissimilarity between the tasks of writing during their homework and working on exams may be the reason for no significant correlations with exams. The power to interpret these results was limited by the number of the participants, the number of modal experiences by the students, and the operationalization of multimodal knowledge through the holistic rubric. These results do show that a relationship does exist between the similar tasks within science writing and quizzes. Students may also gain derived science literacy benefits from modal experiences on distal tasks in exams as well. This study shows that there is still much more research to be known about the interconnectedness of multimodal representational knowledge and use to the development of science literacy.

Doctor of Philosophy(PhD) / There is increasing awareness among linguists that discourse analysis inevitably involves analyses of meanings arising from the combination of multiple modes of communication. The evolving multimodal pedagogic environment for teaching English as a foreign language (henceforth EFL), among other communicative contexts, calls for a social, semiotic, and linguistic explanation. Situated within the theoretical landscape of social semiotics and in the pedagogic context of EFL education, the present study aims to elucidate how linguistic and visual semiotic resources are co-deployed to construe interpersonal meaning in multimodal textbooks. The data drawn upon are eighteen EFL textbooks for primary and secondary schooling, published by People’s Education Press between 2002 and 2006. The research design consists of three complementary sub-studies. First, it investigates the ways in which the semantic regions of ENGAGEMENT and GRADUATION can be modelled in multimodal texts, with special reference to the interplay of voices in textbook discourse. The second sub-study analyzes how verbal and visual semiotic resources are co-deployed to construe the ‘emotion and attitude’ goal highlighted in curriculum standards, with a particular focus on verbiage-image relations. Third, it extends the linguistic concept ‘modality’ to multimodal discourse, exploring coding orientation in texts for different educational contexts and between different constituent genres. The main findings of this thesis are as follows: (1) A range of multimodal resources (i.e. labelling, dialogue balloon, jointly-constructed text, illustration and highlighting) are identified as enabling editor voice to negotiate meanings with reader voice and character voice. It is found that the way in which an ENGAGEMENT value can be scaled is strongly associated with the intrinsic property of the given multimodal resource. The interaction between multiple voices is closely related to contact, social distance, and point of view. (2) It is shown that images play an essential role in realizing attitudinal meanings. Together with verbal APPRAISAL resources, visual semiotic features work to position the readers in ways that align them to set pedagogic goals, guiding them in completing jointly-constructed texts. Moreover, an attitudinal shift from an emotional release to a more institutionalized type of evaluation can be identified as students advance through the school years. (3) It is argued that what counts as real in multimodal texts is socially defined and specific to a given communicative context. The nature of pedagogic discourse should be taken into account when visual displays are produced for pedagogic materials. The implications of this study include both theoretical and pedagogic aspects。Theoretically it adapts and extends APPRAISAL analysis to multimodal discourse, exploring the intersemiotic complementarity and co-instantiation in construing global evaluative stance. This semiotic exploration, in return, suggests ways in which discourse analysis may help textbook users better understand and interpret the multimodal features. With the affordances as well as limitations of semiotic resources made explicit, we may have one step further towards a comprehensive and critical understanding of multimodal construal of interpersonal meaning in pedagogic materials.

Sketches are commonly used in the early stages of design. Our previous system allows users to sketch mechanical systems that the computer interprets. However, some parts of the mechanical system might be too hard or too complicated to express in the sketch. Adding speech recognition to create a multimodal system would move us toward our goal of creating a more natural user interface. This thesis examines the relationship between the verbal and sketch input, particularly how to segment and align the two inputs. Toward this end, subjects were recorded while they sketched and talked. These recordings were transcribed, and a set of rules to perform segmentation and alignment was created. These rules represent the knowledge that the computer needs to perform segmentation and alignment. The rules successfully interpreted the 24 data sets that they were given.

Undersökningens syfte har varit att jämföra de statliga styrdokumentens rekommendationer i relation till de identifierade ståndpunkterna inom forskning om didaktik och ADHD. Som design valde jag en systematisk litteraturstudie. Jag hämtade resultat från 15forskningsstudier inom svensk och internationell forskning, från flera vetenskaper. Empirin består av tre delar, där jag först har studerat statliga styrdokument, därefter didaktisk forskning samt forskning om didaktik och ADHD. Tolkningen av resultaten har gjorts med en jämförande design, för att belysa de likheter och skillnader som finns. Resultat visade på likhet mellan statliga styrdokument och forskningom didaktik och ADHD, inom följande områden: metoder oc htillvägagångssätt, medier, samt centralstimulerande mediciners inverkan på inlärningen. Resultaten visar att det fanns skillnader mellan statliga riktlinjer – i förhållande till forskning om didaktik och ADHD gällande: målet med inlärning, organisationsformer och undervisning, samt former för att kontrollera inlärningsresultat och inlärningssätt. Forskningen visar att det inte fanns någon generelllösning för hur man kunde förväntas undervisa elever med ADHD. Det fanns en mängd olika metoder och inlärningsstilar som alltid bör individanpassas till varje elev.

I gymnasieskolans nya läroplan står att undervisningen ska anpassas till varje elevs förutsättningar och behov. För att klara kursernas kunskapskrav måste eleven inneha eller utveckla en mängd olika förmågor. I den processen behöver läraren vara en delaktig handledare som också kan förstå och göra en rättvis bedömning av elevens utveckling. I undervisningen är det skrivna och talade språket den vanligaste uttrycksformen. Forskningen detta arbete tittat på visar att ny teknik inte bara möjliggör nya uttrycksformer utan att det även förändrar hur vi tänker. Genom en kvantitativ och kvalitativ textanalys av kursplaner inom GY11 studeras här vilka förutsättningar de ger för användningen av olika uttrycksformer. Arbetets kvantitativa analys visar att den mest efterfrågade förmågan för att uppfylla kunskapskravet för E i olika kurser är att kunna redogöra. Den kvalitativa analysen visar att kravet att eleven ska ha förmågan att nyansera för ett högre betyg riskerar att fortsätta ge, om inte till och med förstärka, det skrivna och talade språkets särställning inom skolan. Vad det innebär för så vitt skilda kurser som Golvläggning och Sociologi problematiseras.

During the past few decades, there has been increasing attention to multimodal adaptive language learning interface design. The purpose of this study was to examine users’ experiences with a chatbot language learning interface through the lens of cognitive emotions and emotions in learning. A particular focus of this study was on users’ interactions with a chatbot in a public setting and in a private environment. Focusing on the event of users’ interaction with a chatbot interface, seventy-five interactions were videotaped in this study, in which fifteen users were asked to interact with the chatbot “Lucy” for their language learning. The video-stimulated post interaction interviews with participants provided complementary data for understanding their experiences with the language learning system. Analysis of twenty-five interactions selected from a total of seventy-five revealed five main factors of chatbot language tutor interface design and their relative significance in the process of users’ meaning making and knowledge construction. Findings showed that users’ sensory, emotional, cultural, linguistic and relational engagement influenced their responses to the chatbot interface, which in turn, shaped their learning processes. Building on a theoretical framework of cognitive emotions and emotions in learning, this study documented users’ language learning processes with the chatbot language learning interface by investigating users’ experiences. The findings and techniques resulting from this study will help designers and researchers achieve a better understanding of users’ experiences with technology and the role of emotions in the processes of learning when using technology and assist them to improve the design of language learning environments.

Fusion of information from multiple modalities in Human Computer Interfaces (HCI) has gained a lot of attention in recent years, and has far reaching implications in many areas of human-machine interaction. However, a major limitation of current HCI fusion systems is that the fusion process tends to ignore the semantic nature of modalities, which may reinforce, complement or contradict each other over time. Also, most systems are not robust in representing the ambiguity inherent in human gestures. In this work, we investigate an evidential reasoning based approach for intelligent multimodal fusion, and apply this algorithm to a proposed multimodal system consisting of a Hand Gesture sensor and a Brain Computing Interface (BCI). There are three major contributions of this work to the area of human computer interaction. First, we propose an algorithm for reconstruction of the 3D hand pose given a 2D input video. Second, we develop a BCI using Steady State Visually Evoked Potentials, and show how a multimodal system consisting of the two sensors can improve the efficiency and the complexity of the system, while retaining the same levels of accuracy. Finally, we propose an semantic fusion algorithm based on Transferable Belief Models, which can successfully fuse information from these two sensors, to form meaningful concepts and resolve ambiguity. We also analyze this system for robustness under various operating scenarios.