Due to the availability of technology, there has been a shift from traditional assessment methods to e-assessment methods designed to support learning. With this development there is a need to address the suitability and effectiveness of the e-assessment interface. One development in the e-assessment interface has been the use of the multimodal metaphor. Unfortunately, the associated effectiveness of multimodality in terms of usability and its suitability in achieving assessment aims has not been fully addressed. Thus, there is a need to determine the impact of multimodality on the effectiveness of e-assessment and to reveal the benefits, primarily to the user. Moreover, those involved in the development and assessment should be aware of potential impacts and benefits. This thesis investigates the role and effectiveness of multimodal metaphors in e-assessment, specifically; the thesis assesses the effect of multimodal metaphors, alone or in combination, on usability in e-assessment. Usability includes efficiency, effectiveness and user satisfaction. The empirical research described in this study consisted of three experiments of 30 participants each to evaluate the effect of description text, avatars and images individually, avatars, description text and recorded speech in combination with images, and finally, the use of avatars with whole body gestures, earcons and auditory icons. The experimental stages were designed as a progression towards the main focus of the study, which was the effectiveness of full body gesture avatar, considered to be the latest development in multimodal metaphors. The experimentation also assessed the role that an avatar could play as a tutor in e-assessment interfaces. The results proved the positive effectiveness and applicability of metaphors to enhance e-assessment usability. This was achieved through a more effective interaction between the user and the assessment interface. A set of empirically derived guidelines for the design and use of these metaphors to enhance e-assessment is also used in order to generate more usable e-assessment interfaces.

Thesis (M.S.)--Brigham Young University. School of Technology, 2007. / Includes bibliographical references (p. 87-89).

In work with clients having profound learning disability, music therapists may include in sessions assistants not trained as music therapists. This study is a qualitative inquiry addressing the questions: 1) How does improvised music influence the interaction between teenagers with profound and multiple disability and learning support assistants? 2) Which aspects of the music are associated with any influences found? A survey of music therapists, exploring how assistants are used and how effectively they perform their role, found that assistants are often used as ‘interaction partners’. To explore how the therapist may facilitate client-assistant interaction, about which little is known, video clips from the writer’s clinical practice were purposively selected in order to illustrate an approach entitled Triadic Support of Interaction by Improvisation (TSII). Seven learning support assistants (LSAs) each viewed a video clip showing her own interaction with a teenager having profound disability, supported by the writer’s improvised music. Semi-structured interviews explored the LSAs’ understanding of the behaviour and inferred mental processes of the teenagers, their own behaviour and mental processes and the music improvised by the therapist to support the interaction. A variant of Interpretative Phenomenological Analysis generated shared themes, which included concern for the teenagers’ autonomy, interest in their communicative behaviour and understanding of the mutuality of interaction. The therapist’s improvisation was seen by the LSAs as influencing only the teenagers. All the clips were also viewed by three music therapists, who used a mechanical continuous response device to register the influence of the therapist’s improvisation on four ‘scenarios’: the teenagers’ behaviour, their inferred mental processes, the LSAs’ behaviour and their inferred mental processes. Inter-rater agreement between the three therapists’ continuous responses was generally low, but some intra-rater correlations were found between pairs of scenarios, which the music was perceived as influencing in similar ways. This finding supports the conclusion that musical influences, although they may be analysed according to the four scenarios, actually function as a mutually inter-related system rather than as four independent processes. Each therapist selected decision points from the graphic record of her/his individual responses to discussed with the other therapists as a panel. Positive evaluations were made of the role of TSII in supporting the observed teenager-LSA interactions and the inferred underlying mental processes. This research design was exploratory, and not intended to test specific hypotheses about the mechanisms of musical influence. Tentative suggestions of associations between influences and musical features are however offered by the writer. Indications for the use of TSII are given and other applications suggested for novel aspects of the methodology developed for this study. A refinement of the continuous response task is proposed, and the requirements for any future formal evaluation of TSII are outlined.

Thesis advisor: Yukiko A. Hatasa, Judith E. Liskin-Gasparro. Includes bibliographic references (p. 165-172).

<p>Uppsatsen är genomförd vid Göteborgs universitet, inom ramen för forskarförberedande studier i musikpedagogik.</p>

Human infants rely on social interactions to acquire culturally relevant knowledge about their environment. Aside from active participation (“first-party perspective”), infants encounter social interactions through third-party observation (“third-party perspective”). Despite the absence of own involvement, the mere observation of others’ interactions represents an essential source of social learning opportunities. The overarching aim of this dissertation was to deepen our understanding of the foundations of infants’ observational learning from third-party interactions. This was achieved by investigating (a) social attentional developments and motivational influences driving infants’ attention toward third-party interactions (Study 1 & 2), and (b) factors influencing infants’ attention and memory while observing third-party interactions (Study 3). Study 1 investigated how infants’ attentional orienting to third-party interactions develops in parallel with their active social attention behavior. In Experiment 1, 9.5- to 11-month-old infants looked longer than 7- to 8.5-month-olds at videos showing two adults engaging in a face-to-face interaction, when simultaneously presented with a non-interactive back-to-back scene showing the same people acting individually. Moreover, older infants showed higher social engagement (including joint attention) during parent-infant free play. Experiment 2 replicated this age-related increase in both measures and showed that it follows continuous trajectories from 7 to 13 months of age. These findings suggest that infants’ attentional orienting to others’ social interactions coincides with developments in their social attention behavior during own social interactions. Study 2 examined the incentive value of social interactions as a proximal driver of infants’ attentional orienting to third-party interactions. In a gaze-contingent associative learning task, two geometrical shape cues were repeatedly paired with two kinds of target videos showing either a dyadic face-to-face interaction or a non interactive back-to-back scene. We found that 13-monthold infants performed faster saccadic latencies and more predictive gaze shifts toward the cued target region during social interaction trials. This suggests that social interaction targets can serve as primary reinforcers in an associative learning task, supporting the view that infants find it intrinsically rewarding to observe others’ social interactions. Study 3 investigated infants’ object encoding in the context of observed social interactions. In Experiment 1, 9-month-old infants were presented with four types of videos showing one object and two adults. The scenarios varied regarding the eye contact between the adults (eye contact or no eye contact) and the adults’ object directed gaze (looking toward or away from the object). Infants showed increased object encoding, but only when seeing two adults looking at an object together, following mutual eye contact. We found an identical pattern of results in a matched first-party design during which 9-month-old infants were directly addressed by one single adult on screen (Experiment 2). Together, these findings suggest that the capacity to learn about novel objects by observing third-party interactions emerges in the first postnatal year, and that it may depend on similar factors as infants’ learning through direct social interactions at this age. The findings of all three studies are integrated in a general discussion. In summary, the results of this thesis suggest that, throughout the first year after birth, infants develop abilities and preferences enabling them to approach and efficiently learn from third-party social interactions.:General Introduction 1 1.1 The Infant As an Active Learner 2 1.2 Social Attentional Requirements of Infants’ Learning From Social Interactions 3 1.3 Motivational Mechanisms Affording Opportunities to Learn From Social Interactions 16 1.4 Infants’ Learning From Social Interactions 21 1.5 Research Gaps 26 1.6 Focus of This Dissertation 27 Study I 33 2.1 Introduction 34 2.2 Experiment I 38 2.3 Experiment II 45 2.4 General Discussion 50 Study II 55 3.1 Introduction 56 3.2 Methods 57 3.3 Results 63 3.4 Discussion 64 Study III 69 4.1 Introduction 70 4.2 Experiment I 74 4.3 Experiment II 81 4.4 General Discussion 86 General Discussion 91 5.1 Summary of Results 91 5.2 Research Contributions 93 5.3 Limitations and Future Directions 101 5.4 Overall Conclusion 114 References 115 Appendix A – Supplementary Materials Study I 137 Appendix B – Supplementary Materials Study II 148 Appendix C – Supplementary Materials Study III 152 Curriculum Vitae 167 Scientific Publications and Conference Contributions 169 Contributions of Authors 171 Declaration of Authorship 175

Submitted by MARCIA ROVADOSCHI (marciar@unifra.br) on 2018-08-14T13:38:42Z No. of bitstreams: 2 license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Dissertacao_NadirLaciDieckelMainardi.pdf: 5429138 bytes, checksum: d4be23b9d4642d6e82afe1c1a85e8489 (MD5) / Made available in DSpace on 2018-08-14T13:38:42Z (GMT). No. of bitstreams: 2 license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Dissertacao_NadirLaciDieckelMainardi.pdf: 5429138 bytes, checksum: d4be23b9d4642d6e82afe1c1a85e8489 (MD5) Previous issue date: 2013-01-16 / This dissertation consists of a proposed interaction of existing technologies in our school with classroom content Electricity contemplating having as support concepts resulting from the perspective of Vygotsky (1997). The guiding question was: what is the contribution of the use of a virtual page in interaction with the classroom in the learning process of content on Electricity Electrical Resistors in the discipline of physics? The action was applied to third-year students "A" students with 23 members of the State College of Santo Antão of Bela Vista Caroba - PR. To accomplish this work was necessary to divide into two main phases: the first time we developed a virtual page (wikispaces) containing content Electricity headquartered initially with a controversial issue experienced by the student, causing him to interact with the simulations, virtual experiments , videos, current scientific texts and curiosities that contribute to student learning. In the second time in the classroom has developed content using electrical resistors as support for virtual page concomitantly with scientific text and slides, occurring just one student-computer interaction, student-student and student-teacher, with discussions, questions, interpretations and resolutions of the activities presented in the classroom. The methodology of qualitative observations proved efficient in the discussions among students instant access to the virtual page in the computer lab at school, but there was also efficient in the analysis of discussions and absorptions of content applied in the classroom. Three activities were prepared in the form of testing containing content electricity and applied to participants: the first questionnaire served as propellant forwarding all activities in this dissertation, his resulting demonstrated a low level of knowledge in solving exercises, with answers without nexus as usual without any scientific knowledge. In the second activity, students applied the issues between the two moments (virtual page - classroom), there were some concepts Association of Electrical Resistors more significantly, with some confusion, destabilizing the existing concepts, emerging discussions, reflections and training hypotheses of the concepts acquired in the computer room, taking the students curiosity and desire to understand and interpret the content. Applying the third activity after discussions, interactions and interpretation of students on the content presented in the classroom, it was necessary to determine whether the use of our interaction with the virtual classroom helped in the process of learning the content of Association Electrical Resistors. The results demonstrated satisfactorily in student learning through cultural mediation, appears in the interaction of ICT, teacher and classmates. In activities (testing III) developed by the students, was the domain knowledge of scientific concepts, creating stability and security in the discussions and resolutions of the exercises presented in the classroom. / A presente dissertação consiste em uma proposta de interação das tecnologias existentes em nossa escola com a sala de aula, contemplando o conteúdo de Eletricidade tendo como suporte conceitos decorrentes da perspectiva de Vygotsky (1997). A questão orientadora foi: qual a contribuição do uso de uma página virtual em interação com a sala de aula no processo da aprendizagem do conteúdo de Eletricidade em Resistores Elétricos na disciplina de Física? A ação foi aplicada em alunos do terceiro ano ―A‖ com 23 alunos integrantes do Colégio Estadual Santo Antão de Bela Vista da Caroba – PR. Para realizar este trabalho se fez necessário dividir em dois momentos principais: no primeiro momento foi desenvolvida uma página virtual (wikispaces), contendo o conteúdo Eletricidade sediado inicialmente com uma questão polêmica vivenciada pelo aluno, provocando-o a interagir com as simulações, experimentos virtuais, vídeos, textos científicos e curiosidades atuais que contribuem para aprendizagem dos alunos. No segundo momento em sala de aula se desenvolveu o conteúdo de resistores elétricos usando como apoio a página virtual concomitantemente com o texto científico e slides, ocorrendo assim uma interação entre aluno-computador, aluno-aluno e aluno-professor, com debates, questionamentos, interpretações e resoluções das atividades apresentadas em sala de aula. A metodologia utilizada de cunho qualitativo se mostrou eficiente nas observações das discussões entre os alunos no instante de acesso à página virtual no laboratório de informática da escola, como também houve a eficiência na analise das discussões e absorções do conteúdo aplicado em sala de aula. Foram elaboradas três atividades em forma de testagem contendo o conteúdo de eletricidade e aplicado aos participantes: o primeiro questionário serviu como propulsor do encaminhamento de todas as atividades desenvolvidas nesta dissertação, seu resultando demonstrou um baixo nível de conhecimento na resolução dos exercícios, com respostas sem nexo de forma usual sem nenhum conhecimento científico. Na segunda atividade, as questões aplicadas aos alunos entre os dois momentos (página virtual – sala de aula), surgiram alguns conceitos de Associação de Resistores Elétricos de forma mais significativa, com alguns equívocos, desestabilizando as concepções existentes, surgindo discussões, reflexões e formação de hipóteses dos conceitos adquiridos na sala de informática, levando os alunos a curiosidade e o desejo de entender e interpretar o conteúdo. Aplicando a terceira atividade após as discussões, interpretação e interações dos alunos sobre o conteúdo apresentado em sala de aula, houve a necessidade de verificar se o uso da página virtual em interação com a sala de aula auxiliou o processo da aprendizagem do conteúdo de Associação de Resistores Elétricos. Os resultados se manifestaram satisfatoriamente na aprendizagem dos alunos via mediação cultural, exibida na interação dos TICs, professor e colegas. Nas atividades (testagem III) desenvolvidas pelos alunos, houve domínio no conhecimento dos conceitos científicos, criando uma estabilidade e segurança nas discussões e nas resoluções dos exercícios apresentados na sala de aula.

L’interaction Humain-Robot est un domaine de recherche en pleine expansion parmi la communauté robotique. De par sa nature il réunit des chercheurs venant de domaines variés, tels que psychologie, sociologie et, bien entendu, robotique. Ensemble, ils définissent et dessinent les robots avec lesquels nous interagirons dans notre quotidien.Comme humains et robots commencent à travailler en environnement partagés, la diversité des tâches qu’ils peuvent accomplir augmente drastiquement. Cela créé de nombreux défis et questions qu’il nous faut adresser, en terme de sécurité et d’acceptation des systèmes robotiques.L’être humain a des besoins et attentes bien spécifiques qui ne peuvent être occultés lors de la conception des interactions robotiques. D’une certaine manière, il existe un besoin fort pour l’émergence d’une véritable interaction humain-robot ergonomique.Au cours de cette thèse, nous avons mis en place des méthodes pour inclure des critères ergonomiques et humains dans les algorithmes de prise de décisions, afin d’automatiser le processus de génération d’une interaction ergonomique. Les solutions que nous proposons se basent sur l’utilisation de fonctions de coût encapsulant les besoins humains et permettent d’optimiser les mouvements du robot et le choix des actions. Nous avons ensuite appliqué cette méthode à deux problèmes courants d’interaction humain-robot.Dans un premier temps, nous avons proposé une technique pour améliorer la lisibilité des mouvements du robot afin d’arriver à une meilleure compréhension des ses intentions. Notre approche ne requiert pas de modéliser le concept de lisibilité de mouvements mais pénalise les trajectoires qui amènent à une interprétation erronée ou tardive des intentions du robot durant l’accomplissement d’une tâche partagée. Au cours de plusieurs études utilisateurs nous avons observé un gain substantiel en terme de temps de prédiction et une réduction des erreurs d’interprétation.Puis, nous nous sommes attelés au problème du choix des actions et des mouvements qui vont maximiser l’ergonomie physique du partenaire humain. En utilisant une mesure d’ergonomie des postures humaines, nous simulons les actions et mouvements du robot et de l’humain pour accomplir une tâche donnée, tout en évitant les situations où l’humain serait dans une posture de travail à risque. Les études utilisateurs menées montrent que notre méthode conduit à des postures de travail plus sûr et à une interaction perçue comme étant meilleure. / Human-Robot Interaction (HRI) is a growing field in the robotic community. By its very nature it brings together researchers from various domains including psychology, sociology and obviously robotics who are shaping and designing the robots people will interact with ona daily basis. As human and robots starts working in a shared environment, the diversity of tasks theycan accomplish together is rapidly increasing. This creates challenges and raises concerns tobe addressed in terms of safety and acceptance of the robotic systems. Human beings havespecific needs and expectations that have to be taken into account when designing robotic interactions. In a sense, there is a strong need for a truly ergonomic human-robot interaction.In this thesis, we propose methods to include ergonomics and human factors in the motions and decisions planning algorithms, to automatize this process of generating an ergonomicinteraction. The solutions we propose make use of cost functions that encapsulate the humanneeds and enable the optimization of the robot’s motions and choices of actions. We haveapplied our method to two common problems of human-robot interaction.First, we propose a method to increase the legibility of the robot motions to achieve abetter understanding of its intentions. Our approach does not require modeling the conceptof legible motions but penalizes the trajectories that leads to late or mispredictions of therobot’s intentions during a live execution of a shared task. In several user studies we achievesubstantial gains in terms of prediction time and reduced interpretation errors.Second, we tackle the problem of choosing actions and planning motions that maximize thephysical ergonomics on the human side. Using a well-accepted ergonomic evaluation functionof human postures, we simulate the actions and motions of both the human and the robot,to accomplish a specific task, while avoiding situations where the human could be at risk interms of working posture. The conducted user studies show that our method leads to saferworking postures and a better perceived interaction.

This microanalytic study focuses on the mutimodal word look-up practices of Japanese foreign language learners of English at the novice level using electronic dictionaries (e-dictionaries) in pair conversations. Not yet investigated with a Conversation Analysis (CA) approach, this analysis examines reoccurring interactional and collaborative repair practices (Schegloff, Jefferson, & Sacks, 1977; Schegloff, 2000) of the learners' look-ups, and explicates from the sequential turn-taking procedures (Sacks, Schegloff, & Jefferson, 1974), the underlying social organization of the e-dictionary look-up sequence. Recent research has found that not-yet-fluent learners are capable of relatively smooth turn-taking (Carroll, 2000, 2004), and they employ various embodied actions (Olsher, 2004) to complete their turns. Nonvocal resources such as gaze movement (Goodwin, 1981) and gestures were also investigated in order to better understand how learners collaboratively utilize vocal and nonvocal resources in hybrid actions, to co-construct the meaning of look-up words, and maintain intersubjectivity. While enrolled in a university intensive English program, thirteen native speakers of Japanese video-recorded thirty-minute conversations; and during these conversations, they completed look-up sequences as interactional achievements. The results indicated that EFL novice learners display sophisticated competencies when using e-dictionaries for communication. While collaboratively completing look-up sequences, they display multimodal competencies by noticing trouble with words, initiating look-ups, making candidate proposals of word translations, correcting themselves, mutually acknowledging their understanding, and maintaining intersubjectivity and sequential relevance. In terms of language learning, learners' collaborative learning of words demonstrates instances of learning-as-interaction (Brouwer & Wagner, 2004; Firth & Wagner, 2007), making public the participants' socially situated cognition. Indications of a change in the participants' cognitive state can emerge in the look-up sequential organization. A lack of knowledge is displayed publically in before-look-up actions, encouraging collaboration in the look-up. Multiple proposals and acknowledgement sequences, often displayed in embodied expansions, provide multimodal indications of a possible change in cognitive state and possible gain in knowledge. Thus, the look-up sequence organization is proposed as an interactional organization for the learning of vocabulary. Finally, the understanding of sequential structures and practices that interactants use in looking up words can inform teachers concerning the efficacy of e-dictionary use in the classroom. / CITE/Language Arts

Cette thèse prend pour objet des situations d’apprentissage guidé du français, en face à face et en dehors de cadres institutionnels, se déroulant dans le contexte multilingue du Luxembourg. Elle décrit et analyse des interactions entre des locuteurs plurilingues adultes dont la première langue est le chinois ou le coréen et des locuteurs plurilingues agissant en tant qu’experts pour la langue française. Plus particulièrement, dans l’optique d’une analyse qualitative des données, ce travail s’efforce d’appliquer les outils de l’analyse conversationnelle d’inspiration plutôt anglo-américaine à une vision didactique de tradition de langue française. Pour ce faire, il est fait appel aux notions de compétence communicative (Hymes 1972), de dramatisation (Goffman 1991) et de rôle social (Cicurel 1988). L’analyse montre que dans des situations d’apprentissage-en-interaction, les apprenants et les experts ont recours à une diversité de ressources interactionnelles liées à des activités de dramatisation : le dialogue-en-situation, la voix, la séquence préfabriquée, la séquentialité discursive, la réparation, la séquence explicative, le récit préenregistré, l’évaluation, le récit enchâssé, l’identité, le récit conversationnel de l’expert, l’interview, le récit conversationnel de l’apprenant et le mode éditeur. Pour conclure, un rapprochement est opéré entre ces activités de dramatisation et la didactique du FLE, à plusieurs niveaux, sous la forme de recommandations suggestions. / The current thesis focuses on guided language learning exchanges in French, in a face-to-face non-institutional setting in the multilingual context of Luxembourg. It describes and analyzes interactions between adult plurilingual speakers whose first language is Chinese or Korean and multilingual speakers acting as experts for the French language. Taking a qualitative analysis approach, our work strives to apply the tools of conversation analysis of a rather Anglo-American origin to a vision of “didactique” corresponding to the French language tradition. To this end, we rely in particular on the notions of communicative competence(Hymes 1972), dramatization (Goffman 1991) and social role (Cicurel 1988). The analysis of learning-in-interaction data shows the enactment of a variety of dramatization-related interactional resources by both learners and experts: the situated dialogue, the voice, the formulaic language, the discursive sequentiality, the repair, the explanatory sequence, the pre-recorded conversational narrative, the evaluation, the embedded narrative, the identity, the conversational narrative of the expert, the interview, the conversational narrative of the learner and the editor mode. A few recommendations-suggestions are proposed in the conclusion, focusing on how these dramatization activities could inform, at different levels, the development of French teaching and learning.