Gli obiettivi della presente tesi sono di analizzare, nei bambini con sviluppo tipico e atipico, la costruzione dell'intersoggettività attraverso l'uso di oggetti nello sviluppo atipico (ASD) e attraverso l'interazione con un robot umanoide nello sviluppo tipico. Per quanto riguarda i robot umanoidi, il problema è osservare la possibile attribuzione di caratteristiche simili all'uomo a un robot che possa rendere l'interazione uomo-robot simile all'interazione uomo-uomo che è una classica interazione intersoggettiva per la presenza di due soggetti umani. Questo scopo viene indagato considerando due quadri teorici: 1) la Teoria della Mente che spiegano la costruzione dell'intersoggettività attraverso l'attribuzione di stati mentali ad altri; 2) Prospettiva socio-materiale che postula che in molti casi la costruzione dell'intersoggettività sia mediata dagli oggetti. Quindi, la domanda che sorge in questa dissertazione sui robot umanoidi è: cosa succede quando l'oggetto mediatore (il robot) è anche il partner interattivo. Per raggiungere questo scopo, la presente tesi investiga due temi principali: 1) i modelli di interazione con e l'attribuzione di una mente a un robot da parte di bambini con sviluppo tipico; 2) i modelli di interazione nello sviluppo atipico - bambini autistici - in un'interazione bambino-adulto mediata da oggetti. Il primo tema viene analizzato in una prospettiva teorica e un approccio metodologico innovativi, che porta a una comprensione dell'interazione bambino-robot. La prospettiva teorica collega il ruolo della Teoria della Mente alll'HRI; l'approccio metodologico osserva le strategie decisionali dei bambini nell'HRI, confrontando questi modelli comportamentali quando i bambini interagiscono con un essere umano e con un robot. Il secondo tema della tesi riguarda il ruolo dell'oggetto come mediatore della relazione tra bambini autistici e adulti. Questo viene studiato attraverso la prospettiva socio-materiale che suggerisce che le caratteristiche materiali degli oggetti e le loro qualità sociali intrinseche siano strettamente connesse. / The aims of the present dissertation are to analyse, in children with typical and atypical development, the construction of the intersubjectivity through the use of objects in atypical development (ASD) and through the interaction with a humanoid robot in typical development. About the humanoid robots, the issue is to observe the possible attribution of human-like features to a robot that can make the human-robot interaction similar to the human-human interaction that is a classic intersubjective interaction because of the presence of two human subjects. This purpose is investigated considering two theoretical frameworks: 1) Theory of Mind that explain the construction of the intersubjectivity through the attribution of mental states to others; 2) Socio-material perspective which postulates that the construction of the intersubjectivity is mediated by objects in many cases. Thus, the question arises in this dissertation about the humanoid robot is: what happen when the mediator object (the robot) is also the interactive partner. To achieve this purposes, the present thesis studies two main topics: 1) the interactional patterns with and the attribution of a mind to a robot by typically developing children; 2) the interactional patterns in atypical development - autistic children - in a child-adult interaction mediated by objects. The first topic is analysed in an innovative theoretical perspective and a through novel methodological approach, leading to an innovative understanding of the child-robot interaction. The theoretical perspective connects the role of Theory of Mind in HRI; the methodological approach observes children’s decision-making strategies in the HRI, comparing these behavioural patterns when children interact with a human and with a robot. The second issue of the thesis is about the role of the object as mediator of the relationship between autistic children and adults. This is studied through the Socio- material perspective that hypothesised that material features of the objects and their intrinsic social qualities are strictly connected.

Social practices are acceptable ways of doing things, contextual and materially mediated, shared between actors, and routinized over time. These social practices are interactions and can be used to define rule protocols those drive human-robot interactions. We follow these practices in our day-to-day life in almost all the interactions with other people. Social practices also allow computer scientists to model human computer interactions. Human interactions are very dynamic and so the social practices and they naturally follow some versions. Even though they might deviate from pre-specified patterns they will usually expect to follow the general flow of a social practice and adapt it to a specific circumstances. Now we wanted to take these social practices and apply them in a human-robot-interactions setup by choosing a more applicable scenario for the definition of social practices. In our case a simple scenario in elderly care. More specifically, we want to show how these social practices influence the way AI planning in a Pepper robot in the elderly care setting. Given a specific representation of Social Practices we show how the elements of the Social Practice are used in the planning process. We then show briefly how this plan is executed on the Pepper robot in the scenario. We briefly touch on the hardware aspect of the Pepper robot’s API and the inclusion of external APIs to make it for better results. We design and implement the social practices on Pepper robot and conduct experiments and present the results here.

In everyday communication, humans utilise eye gaze due to its importance as a communication tool. As technology evolves, social robots are expected to become more adopted in society and, since they interact with humans, they should similarly use eye gaze to elevate the level of the interaction and increase humans’ perception of them. Previous studies have shown that robots possessing human-like gaze behaviour increase the interactants’ task performance and their perception of the robot. However, social robots must also be able to behave and respond appropriately when humans act inappropriately, and failure in doing so may normalize bad behaviour even towards other humans. Additionally, with the recent progress of wearable eyetracking technology, there is interest to see how this technology can be used to help generate human gaze into a robot. This thesis work investigates how the eye gaze behaviour from a human being can be modeled into the robot Furhat to behave more human-like in a confrontational scenario. It further investigates how a robot possessing the developed human-like gaze model compares to a robot using a believable heuristic gaze model. We created a pipeline which concerned selecting scenarios, conducting roleplays between actors of these scenarios to collect gaze, extracting and processing that gaze data and extracting probability distributions that the human-like model would utilise. Our model used frequencies to determine where to gaze and head rotation, while gamma distributions were used to sample gaze length. We then executed an online video study with the two robot conditions where participants rated either robot by filling out a questionnaire. The results show that while no statistical difference could be found, the human-like condition scored higher on the measures anthropomorphism/human-likeness and composure, whereas the heuristic condition rated higher on expertise and extroversion. As such, the human-like model did not yield a significant benefit on robot perception to opt for it. Still, we suggest that the pipeline used in this thesis work may help HRI researchers to perform gaze studies and possibly build a foundation for further development. / I vardaglig kommunikation använder människor sig av blickar på grund av dess betydelse som kommunikationsverktyg. Då teknologi ständigt utvecklas förväntas det att sociala robotar kommer att bli mer involverade i samhället, och eftersom de integrerar med människor så bör de på samma sätt använda sig av blickar och ögonrörelser för att höja nivån på interaktionen och därmed förbättra människors uppfattning av dem. Tidigare studier har visat att robotar som använder sig av blickar likt människor kan förbättra deltagarnas utförande av uppgifter samt deras uppfattning av roboten. Sociala robotar måste dock även kunna agera och svara på ett lämpligt sätt när människor beter sig olämpligt, och gör dem inte det finns risken att det olämpliga beteendet normaliseras, även i interaktioner med andra människor. Med de senaste framstegen av portabla eye-tracking enheter finns det därför ett intresse att se hur denna teknologi kan användas för att generera människolikt blickbeteende som sedan används i en robot. Denna studie undersöker hur en människas sätt att blicka och titta kan modelleras i roboten Furhat för att bete sig mer människolikt i ett scenario där konfrontation behövs. Studien undersöker dessutom hur en robot som bär ett människolikt blickbeteende jämför sig med en robot som bär ett trovärdigt heuristiskt blickbeteende. Vi skapade en struktur som involverade att välja scenarion, utföra rollspel mellan skådespelare i dessa scenarier för att samla data om deras blickmönster, extrahera och bearbeta denna data, och extraherade sannolikhetsfördelningar som den människolika modellen skulle använda sig av. Vår modell använde sig av frekvenser för att besluta var roboten skulle blicka, medan gammafördelningar användes för att generera blickens längd. Vi utförde därefter en videostudie online med de två robotvarianterna, där deltagare bedömde någon av robotarna genom att svara på en enkät. Resultaten visar att ingen statistisk signifikant skillnad kunde påvisas. Trender visade dock att modellen med människolik blickbeteende bedömdes högre i mätningen av attributerna antropomorfism/mänsklighet och fattning, medan den heuristiska modellen bedömdes högre i expertis och utåtvändighet. Därav erhöll den människolika modellen ingen signifikant framgång för att föredra den. Vi föreslår ändå att strukturen som användes i studien kan hjälpa MRI forskare att utföra studier som involverar blickbeteende hos människor, och möjligtvis bygga en grund för vidareutveckling av strukturen.

L'interaction homme-robot arrive dans une phase intéressante ou la relation entre un homme et un robot est envisage comme 'un partenariat plutôt que comme une simple relation maitre-esclave. Pour que cela devienne une réalité, le robot a besoin de comprendre le comportement humain. Il ne lui suffit pas de réagir de manière appropriée, il lui faut également être socialement proactif. Pour que ce comportement puis être mise en pratique le roboticien doit s'inspirer de la littérature déjà riche en sciences sociocognitives chez l'homme. Dans ce travail, nous allons identifier les éléments clés d'une telle interaction dans le contexte d'une tâche commune, avec un accent particulier sur la façon dont l'homme doit collaborer pour réaliser avec succès une action commune. Nous allons montrer l'application de ces éléments au cas un système robotique afin d'enrichir les interactions sociales homme-robot pour la prise de décision. A cet égard, une contribution a la gestion du but de haut niveau de robot et le comportement proactif est montre. La description d'un modèle décisionnel d'collaboration pour une tâche collaboratif avec l'humain est donnée. Ainsi, l'étude de l'interaction homme robot montre l'intéret de bien choisir le moment d'une action de communication lors des activités conjointes avec l'humain / Human Robot Interaction is entering into the interesting phase where the relationship with a robot is envisioned more as one of companionship with the human partner than a mere master-slave relationship. For this to become a reality, the robot needs to understand human behavior and not only react appropriately but also be socially proactive. A Companion Robot will also need to collaborate with the human in his daily life and will require a reasoning mechanism to manage thecollaboration and also handle the uncertainty in the human intention to engage and collaborate. In this work, we will identify key elements of such interaction in the context of a collaborative activity, with special focus on how humans successfully collaborate to achieve a joint action. We will show application of these elements in a robotic system to enrich its social human robot interaction aspect of decision making. In this respect, we provide a contribution to managing robot high-level goals and proactive behavior and a description of a coactivity decision model for collaborative human robot task. Also, a HRI user study demonstrates the importance of timing a verbal communication in a proactive human robot joint action

In advanced robot applications, the challenge today is that the robot should perform different successive subtasks to achieve one or more complicated tasks similar to human. Hence, this kind of tasks required to combine different kind of sensors in order to get full information about the work environment. However, from the point of view of control, more sensors mean more possibilities for the structure of the control system. As shown previously, vision and force sensors are the most common external sensors in robot system. As a result, in scientific papers it can be found numerous control algorithms and different structures for vision/force robot control, e.g. shared, traded control etc. The lacks in integration of vision/force robot control could be summarized as follows: • How to define which subspaces should be vision, position or force controlled? • When the controller should switch from one control mode to another one? • How to insure that the visual information could be reliably used? • How to define the most appropriated vision/force control structure? In many previous works, during performing a specified task one kind of vision/force control structure has been used which is pre-defined by the programmer. In addition to that, if the task is modified or changed, it would be much complicated for the user to describe the task and to define the most appropriated vision/force robot control especially if the user is inexperienced. Furthermore, vision and force sensors are used only as simple feedback (e.g. vision sensor is used usually as position estimator) or they are intended to avoid the obstacles. Accordingly, much useful information provided by the sensors which help the robot to perform the task autonomously is missed. In our opinion, these lacks of defining the most appropriate vision/force robot control and the weakness in the utilization from all the information which could be provided by the sensors introduce important limits which prevent the robot to be versatile, autonomous, dependable and user-friendly. For this purpose, helping to increase autonomy, versatility, dependability and user-friendly in certain area of robotics which requires vision/force integration is the scope of this thesis. More concretely: 1. Autonomy: In the term of an automatic decision system which defines the most appropriated vision/force control modes for different kinds of tasks and chooses the best structure of vision/force control depending on the surrounding environments and a priori knowledge. 2. Versatility: By preparing some relevant scenarios for different situations, where both the visual servoing and force control are necessary and indispensable. 3. Dependability: In the term of the robot should depend on its own sensors more than on reprogramming and human intervention. In other words, how the robot system can use all the available information which could be provided by the vision and force sensors, not only for the target object but also for the features extraction of the whole scene. 4. User-friendly: By designing a high level description of the task, the object and the sensor configuration which is suitable also for inexperienced user. If the previous properties are relatively achieved, the proposed robot system can: • Perform different successive and complex tasks. • Grasp/contact and track imprecisely placed objects with different poses. • Decide automatically the most appropriate combination of vision/force feedback for every task and react immediately to the changes from one control cycle to another because of occurrence of some unforeseen events. • Benefit from all the advantages of different vision/force control structures. • Benefit from all the information provided by the sensors. • Reduce the human intervention or reprogramming during the execution of the task. • Facilitate the task description and entering of a priori-knowledge for the user, even if he/she is inexperienced.

In this thesis, we aim at modeling some aspects of the functional role of emotions on an autonomous embodied agent. We begin by describing our robotic prototype, Cherry--a robot with the task of being a tour guide and an office assistant for the Computer Science Department at the University of Central Florida. Cherry did not have a formal emotion representation of internal states, but did have the ability to express emotions through her multimodal interface. The thesis presents the results of a survey we performed via our social informatics approach where we found that: (1) the idea of having emotions in a robot was warmly accepted by Cherry's users, and (2) the intended users were pleased with our initial interface design and functionalities. Guided by these results, we transferred our previous code to a human-height and more robust robot--Petra, the PeopleBot™--where we began to build a formal emotion mechanism and representation for internal states to correspond to the external expressions of Cherry's interface. We describe our overall three-layered architecture, and propose the design of the sensory motor level (the first layer of the three-layered architecture) inspired by the Multilevel Process Theory of Emotion on one hand, and hybrid robotic architecture on the other hand. The sensory-motor level receives and processes incoming stimuli with fuzzy logic and produces emotion-like states without any further willful planning or learning. We will discuss how Petra has been equipped with sonar and vision for obstacle avoidance as well as vision for face recognition, which are used when she roams around the hallway to engage in social interactions with humans. We hope that the sensory motor level in Petra could serve as a foundation for further works in modeling the three-layered architecture of the Emotion State Generator.

Achieving appropriate and natural backchanneling behavior in social robots remains a challenge in Human-Robot Interaction (HRI). This thesis addresses this issue by utilizing methods from Safe Reinforcement Learning in particular shielding to improve social robot backchanneling behavior. The aim of the study is to develop and implement a safety shield that guarantees appropriate backchanneling. In order to achieve that, a Recurrent Neural Network (RNN) is trained on a human-human conversational dataset. Two agents are built; one uses a random algorithm to backchannel and another uses shields on top of its algorithm. The two agents are tested using a recorded human audio, and later evaluated in a between-subject user study with 41 participants. The results did not show any statistical significance between the two conditions, for the chosen significance level of α < 0.05. However, we observe that the agent with shield had a better listening behavior, more appropriate backchanneling behavior and missed less backchanneling opportunities than the agent without shields. This could indicate that shields have a positive impact on the robot’s behavior. We discuss potential explanations for why we did not obtain statistical significance and shed light on the potential for further exploration. / Att uppnå lämpligt och naturligt upbbackningsbeteende i sociala robotar är fortfarande en utmaning i Människa-Robot Interaktion (MRI). Den här avhandlingen tar upp detta problem genom att använda metoder från säker förstärkningsinlärning i synnerhet avskärmning för att förbättra sociala robotars upbbackningsbeteende. Syftet med studien är att utveckla och implementera en säkerhetsavskärmning som garanterar lämplig upbbackning. För att uppnå det, tränas ett återkommande neuralt nätverk på en människa-människa konversationsdatamängd. Två agenter byggs; en använder en slumpmässig algoritm för att upbbacka och en annan använder avskärmninng ovanpå sin algoritm. De två agenterna testas med hjälp av ett inspelat mänskligt ljud och utvärderas senare i en användarstudie med 41 deltagare. Resultaten visade inte någon statistisk signifikans mellan de två skicken, för den valda signifikansnivån < 0, 05. Vi observerar dock att agenten med avskärmning hade ett bättre lyssningsbeteende, mer lämplig upbbackningsbeteende och missade mindre upbbacknings-möjligheter än agenten utan avskärmning. Detta kan indikera att avskärmning har en positiv inverkan på robotarnas beteende. Vi diskuterar potentiella förklaringar till varför vi inte fick statistisk signifikans och belyser potentialen för ytterligare utforskning.

Previous works in Human-Robot Interaction have demonstrated the positive potential benefit of designing highly anthropomorphic robots. This includes physical appearance but also whether they can express emotions, behave in a congruent manner, etc. This work wants to explore the creation of a robot that is able to express a given personality consistently throughout a dialogue while also manifesting congruent emotional expressions. Personality defines many aspects of the character of a person and it can influence how one speaks, behaves, reacts to events, etc. Here, we only focus our attention on language and on how it changes depending on one particular personality trait, the extraversion. To this end, we tested different language models to automate the process of generating language according to a particular personality. We also compared large language models such as GPT-3 to smaller ones, to analyse how size can correlate to performance in this task. We initially evaluated these methods through a fairly small user study in order to confirm the correct manipulation of personality in a text-only context. Results suggest that personality manipulation and how well it is understood highly depend on the context of a dialogue, with a more ‘personal’ dialogue being more successful in manifesting personality. Also, the performance of GPT-3 is comparable to smaller models, specifically trained, with the main difference only given in the perceived fluency of the generations. We then conducted a follow-up study where we chose to use a robot that is capable of showing different facial expressions used to manifest different emotions, the Furhat robot. We integrated into the robot the generations from our language models together with an emotion classification method that is used to guide its facial expressions. Whilst the output of our models did trigger different emotional expressions, resulting in robots which differed both in their language and nonverbal behaviour, resultant perception of these robots’ personality only approached significance (p ∼ 0.08). In this study, GPT3 performed very similarly to much smaller models, with the difference in fluency also being much smaller than before. We did not see any particular change in the perception of the robots in terms of likeability nor uncanniness. / Tidigare arbeten inom Människa-robotinteraktion har visat den positiva potentiella fördelen med att designa mycket antropomorfa robotar. Detta inkluderar fysiskt utseende men också huruvida de kan uttrycka känslor, bete sig på ett kongruent sätt, etc. Detta arbete vill utforska skapandet av en robot som kan uttrycka en given personlighet konsekvent under en dialog samtidigt som den manifesterar kongruenta känslomässiga uttryck. Personlighet definierar många aspekter av en persons karaktär och den kan påverka hur man talar, beter sig, reagerar på händelser etc. Här fokuserar vi vår uppmärksamhet endast på språket och på hur det förändras beroende på ett särskilt personlighetsdrag, extraversion. För detta ändamål testade vi olika språkmodeller för att automatisera processen att skapa språk enligt en viss personlighet. Vi jämförde även stora språkmodeller som GPT-3 med mindre, för att analysera hur storlek kan relatera till prestanda i denna uppgift. Vi utvärderade inledningsvis dessa metoder genom en mindre användarstudie för att bekräfta att personligheten kan manipuleras på rätt sätt i en textbaserad kontext. Resultaten tyder på att personlighetsmanipulation och hur väl den förstås i hög grad beror på sammanhanget i en dialog, där en mer ‘personlig’ dialog är mer framgångsrik när det gäller att manifestera personlighet. Prestandan hos GPT-3 är också jämförbar med mindre modeller, specifikt tränade på en uppgift, där den största skillnaden var i den genererade textens upplevda flyt. Vi gjorde sedan en uppföljningsstudie där vi valde att använda en robot som är kapabel att visa olika ansiktsuttryck och därigenom kapabel att manifestera olika känslor, Furhat-roboten. Vi integrerade talet som genererades från våra språkmodeller i roboten tillsammans med en känsloklassificeringsmetod som används för att styra dess ansiktsuttryck. Medan resultatet av våra modeller framkallade olika känslomässiga uttryck, vilket resulterade i robotar som skilde sig åt både i språk och icke-verbal kommunikation, närmade sig endast den resulterande uppfattningen av dessa robotars personlighet signifikans (p ∼ 0.08). I denna studie presterade GPT-3 mycket likartat med mycket mindre modeller, med skillnaden i flyt också mycket mindre än tidigare. Vi såg ingen speciell förändring i uppfattningen av robotarna när det gäller sympati eller obehaglighet.

Att läsa har en avgörande roll i barn och ungdomars utveckling, men läsmotivationen bland elever i Sverige har sjunkit de senaste åren. Sociala robotar har potential att väcka intresse för läsning, men forskningen är fortfarande i ett tidigt skede. Studien syftar till att utforska och stödja utformningen av designförslag för läsaktiviteter med den sociala roboten Bokbotten. Målet är att undersöka hur läsaktiviteter med Bokbotten kan stimulera mellanstadieelevers intresse och motivation för läsning. För att uppnå studiens syfte användes Design Research Methodology (DRM). DRM är en iterativ metod med tre faser som kombinerar forskning, design och utvärdering för att generera insikter och lösningar. I förståelsefasen analyserades videomaterial från tidigare prototyptestning för att identifiera effektiva och problematiska aspekter av interaktionen mellan elever och Bokbotten. I designfasen utvecklades designförslag baserat på dessa insikter och kvalitativ innehållsanalys av expertintervjuer. Slutligen utvärderades designförslagen av en expertgrupp för att identifiera svagheter, utmaningar och intressanta aspekter. Resultaten visar att Bokbotten har potential att skapa en trygg och engagerande lärmiljö som främjar läslust genom att stödja elevernas autonomi, kompetens och samhörighet. Baserat på de insikter som gavs av studien föreslås följande designriktlinjer: förstärkt autonomi, samarbete med lärare, konceptuellt lärande, samhörighet och inkludering och bidra till känsla av kompetens. / Reading plays a crucial role in the development of children and adolescents, yet reading motivation among students in Sweden has declined in recent years. Social robots have the potential to spark interest in reading, but the research is still in its early stages. This study aims to explore and develop design proposals for reading activities using the BookBot. The overall goal is to examine how reading activities with the BookBot can stimulate middle school students’ interest and motivation for reading. To achieve the study’s objective, the Design Research Methodology (DRM) was employed. DRM is an iterative method comprising three phases - Descriptive Study I, Prescriptive Study, and Descriptive Study II - that integrate research, design, and evaluation to generate insights and solutions. In Descriptive Study I, recorded material from previous prototype testing was analyzed to identify effective and problematic aspects of the interaction between students and the BookBot. In Prescriptive Study, design proposals were developed based on the insights from Descriptive Study I and qualitative content analysis of expert interviews. Finally, the design proposals were evaluated by an expert group to identify weaknesses, challenges, and interesting aspects. The results indicate that the Bookbot has the potential to create a safe and engaging learning environment that fosters motivation for reading. This effect is achieved by supporting three key student needs identified in Self-Determination Theory: autonomy, competence, and relatedness. Based on the insights provided by the study, the following design guidelines are proposed: enhanced autonomy, collaboration with teachers, conceptual learning, relatedness and inclusion, and fostering a sense of competence.

This study investigates if preferences for mental capacities in robots vary across different application domains and identifies influential factors, both in individuals’ characteristics and attributes specific to each robot domain. Employing a between-subject design, participants (N=271) completed a survey collecting both quantitative and qualitative data on preferences for 12 mental capacities across six robot types situated in a specific application domain (medicine, defense, household, social, education, customer service). Half of the mental capacities align with each dimension (experience and agency) in the two-dimensional model of mind (Gray et al., 2007; McMurtrie, 2023). Key findings reveal a general preference for high agency ability and low experience ability across all application domains. Exceptions were found in preference for lower agency ability in the cleaning robot and higher experience ability in the companion robot. Qualitative analysis indicates a desire for objective and logical robots functioning without emotions, while demonstrating empathy for human emotions. Additionally, gender and educational background emerged as factors influencing preference for lower experience abilities in robots. While previous research has mainly focused on attribution of mental capacities to technical agents, this study provides insights into human preferences and factors affecting them. These insights can guide responsible and ethics-driven development and design of robot technology within the field of human-robot interaction.