La phase de conception préliminaire est déterminante lors de la réalisation d’un projet industriel. Elle exploite généralement des outils méthodologiques tels que le brainstorming, l’analyse causale et le chronogramme, qui permettent la collaboration entre des participants aux compétences et aux approches différentes. Ces activités se déroulent dans des salles de réunions traditionnelles, autour d'une table ou devant un tableau blanc, avec l’aide de nombreux papiers et Post-it, ce qui rend la préparation, l’exécution, et l’exploitation de ce processus particulièrement difficile. Jusqu’à présent, cette phase de conception préliminaire a résisté à la numérisation, notamment parce que l’addition d’un dispositif informatique au sein de ces activités perturbe la communication et la collaboration naturelles entre participants. Au cours des dix dernières années, de nombreuses avancées technologiques ont été réalisées en ce qui concerne les dispositifs numériques tels que les tables et les tableaux interactifs, les smartphones et les tablettes tactiles. La similarité des configurations physiques de ces dispositifs avec les dispositifs plus traditionnels permet d’exploiter les capacités préexistantes des utilisateurs (l’habileté motrice, le raisonnement spatial, le langage parlé, etc.). Les chercheurs se basent sur l’hypothèse que ces nouveaux dispositifs, travaillant de concert au sein d’espaces interactifs, pourront augmenter la collaboration co-localisée pour les équipes de conception préliminaire. L’objectif de cette thèse est, d’une part, d’étudier la conception d’un espace interactif pour la collaboration co-localisée durant la phase de conception préliminaire, et d’autre part, de proposer une architecture permettant de réunir les dispositifs hétérogènes et distribués composant cet espace.La première contribution consiste en une présentation détaillée d’un espace interactif utilisant une configuration physique encore peu exploitée dans la littérature scientifique : une table et un tableau multi-tactiles de grandes dimensions. La conception de cet espace interactif a été basée sur des observations d’utilisateurs dans un contexte de conception préliminaire traditionnel et sur une revue de la littérature visant à identifier des principes de conception. Lors de la conception de cet espace, une attention particulière a été portée à l’interface utilisateur qui s’étend sur des écrans partagés et qui maintient une séparation entre les activités d’un participant et les activités d’une équipe. Enfin, les évaluations, réalisées avec des groupes de cinq à six participants, démontrent une amélioration dans l’exploitation des outils méthodologiques sur supports numériques par rapport à une utilisation traditionnelle lors de la conception préliminaire.La conception d’une infrastructure distribuée basée sur un système multi-agents constitue la deuxième contribution de cette thèse. Cette infrastructure parvient à rassembler de nombreuses plateformes et des dispositifs hétérogènes. Elle représente une solution intéressante pour les espaces interactifs, en particulier parce qu’elle tolère particulièrement bien la défaillance de réseau et permet un prototypage rapide des dispositifs. / The preliminary design phase occurs near the launch of an engineering project, normally after an initial requirements gathering phase. Through a series of meetingswhich gathers the key actors of a project, effective preliminary design involves discussion and decision-making punctuated by group creativity techniques. These activities are designed to explore the potential solutions of the problem, such asbrainstorming or causal analysis, or to address the project itself, such as collaborative project planning. Such activities are usually conducted in traditional meeting rooms with pen and paper media, which requires significant time and effort to prepare, perform, and later render into a digitally exploitable format. These processes have resisted previous attempts of computer-supported solutions, because any additional instruments risk obstructing the natural collaboration and workflow that make these activities so beneficial. Over the past decade, technologies such as interactive table tops, interactive wall displays, speech recognition software, 3D motion sensing cameras, and handheld tablets and smartphones have experienced significant advances in maturity. Theirform factors resemble the physical configuration of traditional pen-and-paper environments,while their “natural” input devices (based on multi-touch, gestures, voice, tangibles, etc.) allow them to leverage a user’s pre-existing verbal, spatial,social, motor and cognitive skills. Researchers hypothesize that having these devices working in concert inside interactive spaces could augment collaboration forco-located (i.e. physically present) groups of users.There currently exist several interactive spaces in the literature, illustrating awide range of potential hardware configurations and interaction techniques. The goal of this thesis is first to explore what qualities these interactive spaces should exhibit in their interaction design, particularly with regard to preliminary designactivities, and second, to investigate how their heterogeneous and distributed computing devices can be unified into a flexible and extensible distributed computing architecture. The first main contribution of this thesis is an extensive presentation of an interactive space, which at its core uses a configuration not yet fully explored inprevious literature : a large multitouch table top and a large multitouch interactive Abstract board display. The design of this interactive space is driven by observations o fgroups engaged in preliminary design activities in traditional environments and a literature review aimed at extracting user-centered design guide lines. Special consideration is given to the user interface as it extends across multiple shared displays, and maintains a separation of concerns regarding personal and group work. Finally, evaluations using groups of five and six users show that using such an interactive space, coupled with our proposed multi-display interaction techniques, leads to a more effective construction of the digital artifacts used in preliminary design.The second main contribution of this thesis is a multi-agent infrastructure forthe distributed computing environment which effectively accommodates a widerange of platforms and devices in concerted interaction. By using agent-oriented programming and by establishing a common content language for messaging, the infrastructure is especially tolerant of network faults and suitable for rapid prototyping of heterogeneous devices in the interactive space.

O sistema de TV digital interativa está em fase de implantação no Brasil. O middleware Ginga, responsável por permitir a apresentação de programas interativos, prevê que usuários possam interagir com aplicações apresentadas na TV ao pressionar de teclas em um controle remoto. Considerando que controles remotos tradicionais apresentam limitações de usabilidade, este trabalho teve o objetivo investigar a aplicação de conceitos de computação ubíqua, em particular interfaces naturais e multimodais, como alternativas para prover interatividade entre usuários e programas de TV digital. Como resultado, um dispositivo móvel alternativo ao controle remoto tradicional foi utilizado no projeto de novos mecanismos de interação que incluem interfaces baseadas em telas sensíveis ao toque, interfaces sensíveis a gestos capturados por dispositivos que contêm acelerômetros, e interfaces que contêm microfones que permitem entrada de dados por voz. A construção de protótipos correspondentes foi beneficiada pela (assim como beneficiou) implementação prévia de um componente que oferece funcionalidades para envio de dados multimodais para um receptor de TV digital contendo o middleware Ginga, e de um componente que, instalado no receptor, permite a comunicação peer-to-peer entre dispositivos sem fio / The tradicional Brazilian TV system is being replaced by an interactive digital platform. The Ginga middleware, responsible for allowing the presentation of interactive programs, is able to support user interactions with TV applications by means of key presses on a remote control. Since traditional remotes have usability limitations, this work aimed at investigating the application of ubiquitous computing concepts, such as natural and multimodal interfaces, to provide alternatives for the interaction among users and TV applications. Considering the availability of mobile devices such as smartphones, prototype interfaces based on touch screens, as well as gesture-based, accelerometer-based, and voice-based interfaces have been designed and implemented to allow the interaction usually provided by remote controls. The implementation of those interfaces was supported by the previous development of components providing multimodal interaction and peer-to-peer communication in the context of the Brazilian interactive digital TV system middleware

Submitted by Renata Lopes (renatasil82@gmail.com) on 2018-03-12T18:07:31Z No. of bitstreams: 1 marcoaureliofreeszjunior.pdf: 20328090 bytes, checksum: 7633f57ccc559ade49b11d6db5026d22 (MD5) / Approved for entry into archive by Adriana Oliveira (adriana.oliveira@ufjf.edu.br) on 2018-03-12T18:09:30Z (GMT) No. of bitstreams: 1 marcoaureliofreeszjunior.pdf: 20328090 bytes, checksum: 7633f57ccc559ade49b11d6db5026d22 (MD5) / Made available in DSpace on 2018-03-12T18:09:30Z (GMT). No. of bitstreams: 1 marcoaureliofreeszjunior.pdf: 20328090 bytes, checksum: 7633f57ccc559ade49b11d6db5026d22 (MD5) Previous issue date: 2017-08-25 / Em meio à diversidade de veículos de comunicação atuais, as mídias Out-of-Home vêm desempenhando um papel de destaque tanto em espaços públicos quanto privados, ao permitir a difusão de informações de cunho coletivo a um grande número de pessoas. No que diz respeito à autoria de conteúdos multimídia nesse meio, grande parte do ferramental existente ou faz uso de linguagens de domínios distintos, ou faz uso de abstrações gráficas que facilitam a autoria, porém, utilizam-se de linguagens para representação hiper/multimídia com conceitos distantes aos utilizados nessas abstrações. Portanto, a representação desses documentos se torna complexa, fazendo uso ainda de linguagens de scripts que são elegíveis aos autores não-programadores e de difícil extração automatizada de informações. Com o foco nessa etapa, este trabalho elabora um modelo conceitual hipermídia com entidades de alto nível de abstração que busca se aproximar dos conceitos encontrados na indústria audiovisual, tais como cenas, trilhas e mídias. A partir desse modelo é criada uma linguagem declarativa em XML, utilizada para representar o conteúdo multimídia e, como consequência, permitir sua autoria textual. Essa linguagem procura tratar questões relativas às disposições espaciais, sincronização dos objetos de mídias, adaptação de conteúdos, além da interatividade e algumas de suas múltiplas modalidades de interação, como gestos e voz. Como resultado, alguns casos de uso são tratados por meio da criação de conteúdos DOOH utilizando-se da linguagem multimídia proposta e, em seguida, uma execução em um player de prova de conceito é realizada. Além disso, o trabalho propõe um pré-processamento específico, aplicado a cada instância do modelo, com o objetivo de otimizar a transmissão do conteúdo em arquiteturas broadcast e multicast, como IPTV. Esse pré-processamento aborda a renderização sucessiva de partes determinísticas da instância, de modo a facilitar o sincronismo do conteúdo entre terminais, reduzir a carga de processamento nestes e reduzir o uso de recursos da rede. / Among the variety of current social communication vehicles, Digital Out-of Home (DOOH) media have been playing a remarkable role in both public and private spaces, by allowing for the rapid dissemination of collective information to a large number of people. Regarding multimedia content authoring for this vehicle, much of the existing toolkits either makes use of languages from other domains or makes use of graphic abstracti ons that ease the authoring process, but making use of hyper/multimedia representation languages with distant concepts to those used in these abstractions. Therefore, docu ment representation becomes complex, sometimes makes use of scripting languages, and therefore is illegible by authors and even difficult for automated information extraction. Focusing on this step, this paper develops a conceptual hypermedia model with high-level entities of abstraction which seek to approach the concepts found in the audiovisual in dustry, such as scenes, tracks and media. From that model a XML declarative language is created, used to represent the multimedia content and, as a result, allow its textual authorship. This language seeks to deal with matters relating to spatial arrangements, synchronizing media objects, content adaptation, in addition to the interactivity and some of its multiple modalities of interaction, gesture and voice. As a result, some use cases are handled by creating DOOH content using the proposed multimedia language, and then an execution in a proof-of-concept player is performed. In addition, this paper proposes an improvement applied to the STorML instance with the aim to optimize the transmis sion of content in broadcast and multicast architectures, such as IPTV. This improvement covers subsequent rendering of deterministic parts of the instance to facilitate content timing between terminals, reducing the processing in the terminals and decreasing the use of network resources.

When an individual’s ability to communicate with speech does not function as expected, in many cases, augmentative and alternative communication, (AAC), can serve as a support interaction with other persons. The purpose of this study is to investigate how a specific child’s communicative possibilities and abilities develop during the first months after introducing a speech generating device. The study’s main participant Alex is 6 years old and has been diagnosed with autism spectrum disorder and ADHD. Alex also has confirmed limitations regarding gross and fine motor skills and further investigation will examine the presence of childhood apraxia of speech. Two play situations were videotaped at approximately four months intervals and then analysed via conversation analysis and multimodal interaction analysis. At the study’s initial recording session Alex’ communication consisted of gestures and body language together with isolated vocalisations. There were no observed occurrences of spoken words. At the start of the study Alex had been using their speech generating device, SGD, for approximately six weeks. At the end of the study the SGD is found to have been integrated in Alex’ communication during the observed play situations. Its role as a communicative resource has developed from being primarily a device which the child presses in order to answer direct questions, into being used to organise, for example, turn taking, initiate new play rounds or asking questions, by way of other types of manipulation. In addition, Alex’ use of vocalisations has increased and become more situation specific. Baseline measurements were not conducted in this study, wherefore it is not possible to indicate the implementation of an SGD as the reason for Alex’ communicative development. Continued research is needed in order to achieve individual and optimum solutions for all who are dependent on support in their communication. / När en individs möjligheter att kommunicera med tal av någon anledning inte fungerar som förväntat kan i många fall alternativ och kompletterande kommunikation, förkortat AKK, fungera som ett stöd i interaktionen med andra människor. Syftet med denna studie är att undersöka hur ett specifikt barns kommunikativa möjligheter och förmågor utvecklas under de första månaderna efter att ett högteknologiskt AKK introducerats. Studiens huvuddeltagare Alex är 6 år och har diagnoserna autism i barndomen och ADHD. Alex har även konstaterade grov- och finmotoriska begränsningar och vidare undersökningar kommer att undersöka förekomst av verbal dyspraxi. Två leksituationer videofilmades med cirka fyra månaders mellanrum och analyserades sedan med samtalsanalys och multimodal interaktionsanalys. Vid studiens inledande inspelning utgjordes Alex kommunikation av gester och kroppsspråk samt enstaka vokaliseringar. Inga talade ord förekom. Vid studiens slut konstateras att hAKK vid den observerade leksituationen integrerats i Alex kommunikation. Dess roll som kommunikativ resurs har utvecklats från att främst vara en apparat som barnet trycker på för att svara på direkta frågor, till att genom annan typ av manipulering exempelvis organisera turtagning, initiera en ny lekomgång och ställa frågor. Även Alex användning av vokaliseringar i leksituationen har ökat och blivit mer situationsspecifika. Alex mamma och en av hens förskollärare intervjuades och besvarade skattningsfrågor. Resultaten visar att de båda noterat en generellt ökad vilja till kommunikation hos Alex samt lyfter frågan om vilka begränsningar som fortfarande finns gällande Alex kommunikativa möjligheter vid studiens slut. Baselinemätningar saknas i denna studie, varför det inte är möjligt att ange implementeringen av hAKK som orsak till Alex kommunikativa utveckling. Fortsatt forskning behövs för att kunna uppnå individuella och optimala lösningar för alla som är beroende av stöd i sin kommunikation.

O sistema de TV digital interativa está em fase de implantação no Brasil. O middleware Ginga, responsável por permitir a apresentação de programas interativos, prevê que usuários possam interagir com aplicações apresentadas na TV ao pressionar de teclas em um controle remoto. Considerando que controles remotos tradicionais apresentam limitações de usabilidade, este trabalho teve o objetivo investigar a aplicação de conceitos de computação ubíqua, em particular interfaces naturais e multimodais, como alternativas para prover interatividade entre usuários e programas de TV digital. Como resultado, um dispositivo móvel alternativo ao controle remoto tradicional foi utilizado no projeto de novos mecanismos de interação que incluem interfaces baseadas em telas sensíveis ao toque, interfaces sensíveis a gestos capturados por dispositivos que contêm acelerômetros, e interfaces que contêm microfones que permitem entrada de dados por voz. A construção de protótipos correspondentes foi beneficiada pela (assim como beneficiou) implementação prévia de um componente que oferece funcionalidades para envio de dados multimodais para um receptor de TV digital contendo o middleware Ginga, e de um componente que, instalado no receptor, permite a comunicação peer-to-peer entre dispositivos sem fio / The tradicional Brazilian TV system is being replaced by an interactive digital platform. The Ginga middleware, responsible for allowing the presentation of interactive programs, is able to support user interactions with TV applications by means of key presses on a remote control. Since traditional remotes have usability limitations, this work aimed at investigating the application of ubiquitous computing concepts, such as natural and multimodal interfaces, to provide alternatives for the interaction among users and TV applications. Considering the availability of mobile devices such as smartphones, prototype interfaces based on touch screens, as well as gesture-based, accelerometer-based, and voice-based interfaces have been designed and implemented to allow the interaction usually provided by remote controls. The implementation of those interfaces was supported by the previous development of components providing multimodal interaction and peer-to-peer communication in the context of the Brazilian interactive digital TV system middleware

Syfte Studiens syfte är att undersöka hur samspel och kommunikation med personer med flerfunktionsnedsättning, deras assistenter och vattendansledare under vattendans kan analyseras med multimodal interaktionsanalys, för att genom tolkning av kommunikativa uttryck identifiera upplevelse av interventionen. Ytterligare ett syfte med studien är att identifiera vilka styrkor och svagheter analysmetoden har vid studier av interaktion mellan personer med flerfunktionsnedsättning och deras assistenter. Metod I studien videoinspelades och analyserades interaktion mellan personer med flerfunktionsnedsättning, deras medföljare samt personal som höll i vattendansen. Den metod som användes var multimodal interaktionsanalys, som är en metod där man analyserar interaktion i naturligt förekommande situationer. Analysen utfördes genom observation och transkription av videoinspelad interaktion. Resultat Ett flertal uttryck identifierades med multimodal interaktionsanalys. Dessa uttryck kunde säga något om upplevelsen hos personerna med flerfunktionsnedsättning under vattendansen. De kommunikativa uttrycken analyserades alltid i den kontext där de förekom. Uttrycken betraktades som kontextberoende och inte som isolerade handlingar. De modaliteter som personerna med flerfunktionsnedsättning använde för att kommunicera var bland annat blickar, vokalisationer och ansiktsmimik. Dessa uttryck studerades i termer av vilken mening eller sinnesstämning de tycktes förmedla, hur frekvent de förekom, samt vilken respons de väckte. Tolkning av uttrycken gjordes till stor del genom att studera omgivningens respons. Under arbetets gång identifieras styrkor såväl som svagheter med analysmetoden. Att analysera materialet med någon annan metod än just multimodal interaktionsanalys hade varit svårt, eftersom det i dagsläget finns mycket få alternativa sätt att analysera och transkribera den här typen av komplex interaktion på. Slutsats Genom multimodal interaktionsanalys kunde en varierad bredd av olika kommunikativa uttryck identifieras och tolkas, och författarna fick därmed en bild av hur personerna med flerfunktionsnedsättning och deras medföljare upplevde interventionsmetoden. Överlag förefaller upplevelsen av vattendansen vara positiv för personerna med flerfunktionsnedsättning såväl som för deras medföljare. Multimodal interaktionsanalys lämpade sig väl som analysmetod för att identifiera tecken som kan säga något om personernas med flerfunktionsnedsättnings upplevelse av vattendansen. / Aim The aim of this study is to investigate whether interaction between people with multiple disabilities, their assistants and waterdance staff during waterdance can be analysed using multimodal interaction analysis, in order to identify how the people with multiple disabilities experience the intervention. Identification of experience is conducted by recognizing and interpreting communicative actions. Another aim is to identify strengths and weaknesses of the method of analysis in the study of interaction between people with multiple functional impairments and their assistants. Method Interaction between three people with multiple disabilities, their assistants and waterdance staff was video recorded and then analysed. The method used is multimodal interaction analysis, which is a method for analysing naturally occurring interaction. The analysis was performed through observation and transcription of video-recorded interaction. Results Several communicative expressions were identified with multimodal interaction analysis. These expressions could say something about the people with multiple disabilities’ experience of the waterdance. The communicative actions and expressions were analysed in the context in which they occurred. The expressions were regarded as contextually dependent, and not as isolated, independent actions. Modalities that were used for communication were among others: gaze, vocalizations and facial expressions. These communicative actions were studied regarding what meaning or mood they seemed to convey, the frequency with which they occurred in, and in what response they evoked. Interpretation of the expressions was largely conducted by studying the response from people in the environment. Strengths as well as weaknesses were identified regarding the method of analysis. To analyse the material with a method other than multimodal interaction analysis would have been difficult, since other options of methods for analysing and transcribing this complex type of interaction are very limited. Conclusions A variety of communicative expressions were identified through multimodal interaction analysis. Through interpretation of the expressions, the authors were able to better understand how the people with multiple disabilities and their assistants experienced the intervention method. Multimodal interaction analysis is well suited as an analytical method for identifying signs that may tell us something about the experience of waterdance, of the three persons with multiple disabilities.

I dagens samhälle har användning av digitala medier ökat drastiskt och i fler områden. Användning av mobila pekskärmenheter har ökat successivt hos småbarn och förskolebarn, och åldern när barn först exponeras till dessa enheter har minskat. Till skillnad från en tidigare tidsperiod utsätts barn idag för en stor mängd stimuli från digital media under sina utvecklingsår. De kort- och långsiktiga effekterna av denna typ av exponering är fortfarande inte helt utforskade. Theory of Mind (ToM) är ett väsentligt område i utvecklingsprocessen hos barn. Vi har i denna studie undersökt barns ToM i en narrativ kontext, där vi bryter ner och analyserarderas yttranden och beteendemönster som visar ToM-förmåga både språkligt och interaktionellt. Multimodal interaktionsanalys låg som grund för analyserna i denna studie, där 45 barn speladesin när de berättar utifrån bilder i boken ”Frog, where are you?”. Resultatet redovisades både kvalitativt för att besvara frågan hur barn visar ToM-förmåga i ett narrativt sammanhang, och kvantitativt för att besvara frågan huruvida det finns en korrelation mellan ToM-förmåga ochdigital mediaanvänding. Det kvantitativa resultatet visade inget signifikant samband mellandigital mediaanvändning eller bokläsning och ToM-förmågan hos barn. Det kvalitativa resultatetvisar att barn integrerar flera olika kunskapsområden för att uppvisa sin ToM både språkligt ochinteraktionellt. / Currently, the use of digital media has increased significantly and has become dominant in manyareas in our society. The use of mobile touch screen devices has increased gradually in youngchildren and preschool children, and the age at which children are first exposed to these deviceshas decreased. Unlike earlier eras, children today are exposed to a substantial quantity of stimulifrom digital media during their developmental years. The short- and long-term effects of thistype of exposure are still not fully explored. Theory of Mind is an essential part of thedevelopmental process in children. In this study, we examined children's ToM in a narrative context, where we analyzed their statements and behavioural patterns that show ToM abilities both linguistically and interactively. Multimodal interaction analysis was the basis for the analyses in this study, where 45 children were recorded when they told a story called "Frog, where are you?". The results were reported qualitatively to answer the question of how children show ToM ability in a narrative context, and quantitatively to answer the question of whether there is acorrelation between ToM ability and children’s digital media use. The quantitative result showed no significant relation between digital media usage or book reading and children’s ToM ability.The qualitative result shows that children integrate several different areas of knowledge to present their ToM both linguistically and interactively, during the narrative activity.

This thesis presents scientific contributions to the field of multimodal interac- tive structured prediction (MISP). The aim of MISP is to reduce the human effort required to supervise an automatic output, in an efficient and ergonomic way. Hence, this thesis focuses on the two aspects of MISP systems. The first aspect, which refers to the interactive part of MISP, is the study of strate- gies for efficient human¿computer collaboration to produce error-free outputs. Multimodality, the second aspect, deals with other more ergonomic modalities of communication with the computer rather than keyboard and mouse. To begin with, in sequential interaction the user is assumed to supervise the output from left-to-right so that errors are corrected in sequential order. We study the problem under the decision theory framework and define an optimum decoding algorithm. The optimum algorithm is compared to the usually ap- plied, standard approach. Experimental results on several tasks suggests that the optimum algorithm is slightly better than the standard algorithm. In contrast to sequential interaction, in active interaction it is the system that decides what should be given to the user for supervision. On the one hand, user supervision can be reduced if the user is required to supervise only the outputs that the system expects to be erroneous. In this respect, we define a strategy that retrieves first the outputs with highest expected error first. Moreover, we prove that this strategy is optimum under certain conditions, which is validated by experimental results. On the other hand, if the goal is to reduce the number of corrections, active interaction works by selecting elements, one by one, e.g., words of a given output to be supervised by the user. For this case, several strategies are compared. Unlike the previous case, the strategy that performs better is to choose the element with highest confidence, which coincides with the findings of the optimum algorithm for sequential interaction. However, this also suggests that minimizing effort and supervision are contradictory goals. With respect to the multimodality aspect, this thesis delves into techniques to make multimodal systems more robust. To achieve that, multimodal systems are improved by providing contextual information of the application at hand. First, we study how to integrate e-pen interaction in a machine translation task. We contribute to the state-of-the-art by leveraging the information from the source sentence. Several strategies are compared basically grouped into two approaches: inspired by word-based translation models and n-grams generated from a phrase-based system. The experiments show that the former outper- forms the latter for this task. Furthermore, the results present remarkable improvements against not using contextual information. Second, similar ex- periments are conducted on a speech-enabled interface for interactive machine translation. The improvements over the baseline are also noticeable. How- ever, in this case, phrase-based models perform much better than word-based models. We attribute that to the fact that acoustic models are poorer estima- tions than morphologic models and, thus, they benefit more from the language model. Finally, similar techniques are proposed for dictation of handwritten documents. The results show that speech and handwritten recognition can be combined in an effective way. Finally, an evaluation with real users is carried out to compare an interactive machine translation prototype with a post-editing prototype. The results of the study reveal that users are very sensitive to the usability aspects of the user interface. Therefore, usability is a crucial aspect to consider in an human evaluation that can hinder the real benefits of the technology being evaluated. Hopefully, once usability problems are fixed, the evaluation indicates that users are more favorable to work with the interactive machine translation system than to the post-editing system. / Alabau Gonzalvo, V. (2014). Multimodal interactive structured prediction [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/35135 / TESIS / Premios Extraordinarios de tesis doctorales

Denna uppsats bidrar till forskningen om social och språklig kategorisering i interaktion genom att undersöka kategoriseringsmetoder i serviceinteraktioner. Uppsatsen undersöker vilka metoder personal, kunder och personal och kunder tillsammans använder sig av för att identifiera kunderna som pensionärer i samband med rabatt, samt vilka metoder som används för att hantera aspekter av kategoriseringen pensionär som inte har med serviceärendet i sig att göra. Teoretiskt och metodologiskt bygger undersökningen på Conversation Analysis (CA), Membership Categorization Analysis (MCA) och multimodal interaktionsanalys. Studiens data består av 18 videoinspelade, transkriberade samtal vid teaterkassor. I studien identifieras och analyseras tre metoder för kategorisering: alternativa kategorier, överlämnande självkategorisering och retrospektiv nyansering. Analysen av metoderna visar bland annat att olika typer av deltagare behandlar olika aspekter av kategoriseringen som olika känsliga och olika relevanta, att deltagarna orienterar sig mot kategoriseringen som ett gemensamt projekt och att det vid vissa skeden i de i för övrigt målfokuserade servicesamtalen finns utrymme för kunder som kategoriserats som pensionärer att ge uttryck för personliga nyanseringar och positioneringar i förhållande till kategorin pensionär.

Interest in comics as Swedish school material has risen in the last few years and the publication of comics for children and adolescents has also increased. Meanwhile, although research around new literacies has taken an interest in combinations of image and text, there is still little research on comics as a literacy material, especially as part of school practices. With comics’ rise in popularity, and their quality as examples of new literacies, this points to the relevance of exploring how meaning making with comics is done in schools. The purpose of this study is to contribute knowledge on how locally situated literacy practices are done, practices in which pupils and teachers make meaning with comics. The study combines literacy, comics and discursive psychology to investigate aspects of literacy not as individual, inner workings, but as part of participants’ social constructions, in line with New Literacy Studies. With this perspective, it is possible to investigate literary concepts such as narrative, and participants’ construction of story elements, through the way in which these aspects are utilized by participants to construct social action – what participants do with their utterances. To study this, video recordings have been made in one primary and one secondary school, in two different Swedish cities. The results of the study show constructions of a comics literacy, where participants engage with both visual and textual aspects of the material and negotiate focalization of narrative perspective and construction of narrative structure as well as narrative devices such as speech and thought bubbles. Furthermore, meaning making of comics literacy also includes the construction of discourses around comics as a specific type of story telling, either for material or literary reasons. The thesis discusses how participants construct classroom literature, and provides insight into how interaction around comics enables participants to construct and negotiate discourses around what comics literacy is and what it enables, as well as how to talk about, create, and read comics. / Intresset för serier som svenskt skolmaterial har stigit de senaste åren och publiceringen av serier för barn och ungdomar har också ökat. Även om forskning om new literacies har intresserat sig för kombinationer av bild och text så finns det fortfarande lite forskning på serier som literacymaterial, speciellt som en del av skolpraktik. Med det stigande intresset för serier och deras kvaliteter som exempel på new literacies, så pekar detta mot att det finns en relevans i att utforska hur meningsskapande med serier görs i skolan. Syftet med denna studie är att bidra med kunskap om hur lokalt situerad literacypraktik görs där elever och lärare skapar mening med serier. Studien kombinerar forskning om literacy, serier och diskursiv psykologi för att, i linje med New Literacy Studies, undersöka aspekter av literacy som en del av deltagarnas sociala konstruktioner – inte som ett individuellt, mentalt fenomen. Med detta perspektiv är det möjligt att undersöka litterära koncept som narrativ och deltagares konstruktion av berättelseinslag, genom det sätt på vilka dessa aspekter används av deltagare för att interagera – vad deltagare gör när de säger något. För att studera detta har videoobservationer använts i en lågstadieskola och en högstadieskola i två olika svenska städer. Resultaten från studien demonstrerar konstruktioner av serie-literacy där deltagarna engagerar sig i både text och bild i materialet, diskuterar berättandeperspektiv och konstruktioner av narrativ struktur, såväl som berättarverktyg, t.ex. prat- och tankebubblor. Därutöver inkluderar serie-literacy också deltagarnas skapande av seriediskurser där serier görs till en specifik typ av berättande, antingen på materiell eller litterär basis. Avhandlingen diskuterar hur deltagare konstruerar klassrumslitteratur, och studien erbjuder en insikt i hur interaktion runt serier möjliggör för deltagare att konstruera och förhandla diskurser om vad serieliteracy är och vad det erbjuder för möjligheter, såväl som hur deltagare kan prata om, skapa och läsa serier.