abstract: Research has shown that the learning processes can be enriched and enhanced with the presence of affective interventions. The goal of this dissertation was to design, implement, and evaluate an affective agent that provides affective support in real-time in order to enrich the student’s learning experience and performance by inducing and/or maintaining a productive learning path. This work combined research and best practices from affective computing, intelligent tutoring systems, and educational technology to address the design and implementation of an affective agent and corresponding pedagogical interventions. It included the incorporation of the affective agent into an Exploratory Learning Environment (ELE) adapted for this research. A gendered, three-dimensional, animated, human-like character accompanied by text- and speech-based dialogue visually represented the proposed affective agent. The agent’s pedagogical interventions considered inputs from the ELE (interface, model building, and performance events) and from the user (emotional and cognitive events). The user’s emotional events captured by biometric sensors and processed by a decision-level fusion algorithm for a multimodal system in combination with the events from the ELE informed the production-rule-based behavior engine to define and trigger pedagogical interventions. The pedagogical interventions were focused on affective dimensions and occurred in the form of affective dialogue prompts and animations. An experiment was conducted to assess the impact of the affective agent, Hope, on the student’s learning experience and performance. In terms of the student’s learning experience, the effect of the agent was analyzed in four components: perception of the instructional material, perception of the usefulness of the agent, ELE usability, and the affective responses from the agent triggered by the student’s affective states. Additionally, in terms of the student’s performance, the effect of the agent was analyzed in five components: tasks completed, time spent solving a task, planning time while solving a task, usage of the provided help, and attempts to successfully complete a task. The findings from the experiment did not provide the anticipated results related to the effect of the agent; however, the results provided insights to improve diverse components in the design of affective agents as well as for the design of the behavior engines and algorithms to detect, represent, and handle affective information. / Dissertation/Thesis / Doctoral Dissertation Computer Science 2018

abstract: Languages, specially gestural and sign languages, are best learned in immersive environments with rich feedback. Computer-Aided Language Learning (CALL) solu- tions for spoken languages have successfully incorporated some feedback mechanisms, but no such solution exists for signed languages. Computer Aided Sign Language Learning (CASLL) is a recent and promising field of research which is made feasible by advances in Computer Vision and Sign Language Recognition(SLR). Leveraging existing SLR systems for feedback based learning is not feasible because their decision processes are not human interpretable and do not facilitate conceptual feedback to learners. Thus, fundamental research is needed towards designing systems that are modular and explainable. The explanations from these systems can then be used to produce feedback to aid in the learning process. In this work, I present novel approaches for the recognition of location, movement and handshape that are components of American Sign Language (ASL) using both wrist-worn sensors as well as webcams. Finally, I present Learn2Sign(L2S), a chat- bot based AI tutor that can provide fine-grained conceptual feedback to learners of ASL using the modular recognition approaches. L2S is designed to provide feedback directly relating to the fundamental concepts of ASL using an explainable AI. I present the system performance results in terms of Precision, Recall and F-1 scores as well as validation results towards the learning outcomes of users. Both retention and execution tests for 26 participants for 14 different ASL words learned using learn2sign is presented. Finally, I also present the results of a post-usage usability survey for all the participants. In this work, I found that learners who received live feedback on their executions improved their execution as well as retention performances. The average increase in execution performance was 28% points and that for retention was 4% points. / Dissertation/Thesis / Doctoral Dissertation Engineering 2020

This work presents an interactive environment for learning about fractions, with mechanisms to support cooperative and adaptive interactions offered by tutors agents to human learners, focusing mainly on activities to solve problems. For this purpose, an architecture based on software agents and semantic Web services was proposed, therefore, we verify the functional viability of the proposal and, posteriorly, to present a revision of that architecture to suply some requirements not previously covered, beyond models that support to those interactions. With respect to interactions, the learner will receive support from both a pedagogical agent tutor, as some of their peers who are part of the environment. Particularly, a tutor agent has an open learner model, from which it obtains information to guide their actions. The idea of this model be opened is to allow the learner seeing the evaluation that the system has about him, and also the opportunity to disagree with this assessment, and thus contribute to the refinement of the content of such a model / Fundação de Amparo a Pesquisa do Estado de Alagoas / Este trabalho apresenta um ambiente interativo de aprendizagem sobre Frações, dotado de mecanismos de suporte a interações cooperativas e adaptativas oferecidas por seus agentes tutores aos aprendizes humanos, focando principalmente em atividades de resolução de problemas. Para isso, propõe-se uma arquitetura baseada em agentes de software e serviços Web semânticos, daí, pôde-se verificar a viabilidade funcional da proposta e, posteriormente, apresentar uma revisão de tal arquitetura para suprir alguns requisitos anteriormente não visados, além de modelos que dão suporte às referidas interações. No que diz respeito às interações, o aprendiz receberá suporte pedagógico tanto de um agente tutor, quanto de algum de seus pares que fazem parte do ambiente. Particularmente, um agente tutor conta com um modelo aberto do aprendiz, a partir do qual passa a dispor de informações úteis para orientar suas ações. A idéia deste modelo ser aberto é a de permitir que o aprendiz possa ver qual a avaliação que o sistema tem a seu respeito, tendo ainda a oportunidade de discordar de tal avaliação, e assim contribuir para o refinamento do conteúdo de tal modelo. Palavras-chave: Modelagem Aberta do Aprendiz; Sistemas Tutores Inteligentes; Sistemas Multi-agentes