Les humains et les robots travaillant en toute sécurité et en parfaite harmonie dans un environnement est l'un des objectifs futurs de la communauté robotique. Quand les humains et les robots peuvent travailler ensemble dans le même espace, toute une catégorie de tâches devient prête à l'automatisation, allant de la collaboration pour l'assemblage de pièces, à la manutention de pièces et de materiels ainsi qu'à leur livraison. Garantir la sûreté des humains nécessite que le robot puisse être capable de surveiller la zone de travail, déduire l'intention humaine, et être conscient suffisamment tôt des dangers potentiels afin de les éviter.Des normes existent sur la collaboration entre robots et humains, cependant elles se focalisent à limiter les distances d'approche et les forces de contact entre l'humain et le robot. Ces approches s'appuient sur des processus qui se basent uniquement sur la lecture des capteurs, et ne tiennent pas compte des états futurs ou des informations sur les tâches en question. Un outil clé pour la sécurité entre des robots et des humains travaillant dans un environnement inclut la reconnaissance de l'intention dans lequel le robot tente de comprendre l'intention d'un agent (l'humain) en reconnaissant tout ou partie des actions de l'agent pour l'aider à prévoir les actions futures de cet agent. La connaissance de ces actions futures permettra au robot de planifier sa contribution aux tâches que l'humain doit exécuter ou au minimum, à ne pas se mettre dans une position dangereuse.Dans cette thèse, nous présentons une approche qui est capable de déduire l'intention d'un agent grâce à la reconnaissance et à la représentation des informations de l'état. Cette approche est différente des nombreuses approches présentes dans la littérature qui se concentrent principalement sur la reconnaissance de l'activité (par opposition à la reconnaissance de l'état) et qui « devinent » des raisons pour expliquer les observations. Nous déduisons les relations détaillées de l'état à partir d'observations en utilisant Region Connection Calculus 8 (RCC-8) et ensuite nous déduisons les relations globales de l'état qui sont vraies à un moment donné. L'utilisation des informations sur l'état sert à apporter une contribution plus précise aux algorithmes de reconnaissance de l'intention et à générer des résultats qui sont equivalents, et dans certains cas, meilleurs qu'un être humain qui a accès aux mêmes informations. / Humans and robots working safely and seamlessly together in a cooperative environment is one of the future goals of the robotics community. When humans and robots can work together in the same space, a whole class of tasks becomes amenable to automation, ranging from collaborative assembly to parts and material handling to delivery. Proposed standards exist for collaborative human-robot safety, but they focus on limiting the approach distances and contact forces between the human and the robot. These standards focus on reactive processes based only on current sensor readings. They do not consider future states or task-relevant information. A key enabler for human-robot safety in cooperative environments involves the field of intention recognition, in which the robot attempts to understand the intention of an agent (the human) by recognizing some or all of their actions to help predict the human’s future actions.We present an approach to inferring the intention of an agent in the environment via the recognition and representation of state information. This approach to intention recognition is different than many ontology-based intention recognition approaches in the literature as they primarily focus on activity (as opposed to state) recognition and then use a form of abduction to provide explanations for observations. We infer detailed state relationships using observations based on Region Connection Calculus 8 (RCC-8) and then infer the overall state relationships that are true at a given time. Once a sequence of state relationships has been determined, we use a Bayesian approach to associate those states with likely overall intentions to determine the next possible action (and associated state) that is likely to occur. We compare the output of the Intention Recognition Algorithm to those of an experiment involving human subjects attempting to recognize the same intentions in a manufacturing kitting domain. The results show that the Intention Recognition Algorithm, in almost every case, performed as good, if not better, than a human performing the same activity.

Until recently, few technologies have been applicable to increase flexibility in the manufacturers’ assembly applications, but the introduction of industrial robots in collaborative assembly applications provides such opportunities. Specifically, these collaborative assembly applications present an opportunity to, in a fenceless environment, combine the flexibility of the human with the accuracy, repeatability, and strengths of the robot while utilizing less floor space and allowing portable applications. However, despite the benefits of industrial robots in collaborative assembly applications, there are significant gaps in the literature preventing their implementation. Based on this background, the objective of this work is to support the implementation of industrial robots in collaborative assembly applications. To fulfill this objective, this work included two empirical studies; first, an interview study mapped the attributes of industrial robots in collaborative assembly applications. Second, a multiple-case study mapped the critical challenges and enabling activities when implementing these collaborative assembly applications. The studies were also combined with literature reviews aiming to fill the theoretical gaps.  The work provides an implementation process with enabling activities that can mitigate critical challenges when implementing industrial robots in collaborative assembly applications. The implementation process shows enabling activities in the three first phases: pre-study, collaborative assembly application design, and assembly installation. These enabling activities are mapped to the 7M dimensions as a way to clearly show how they can support the implementation of industrial robots in collaborative assembly applications. The implementation process contributes to filling the identified gaps in the literature and provides practitioners with activities that managers could consider when implementing collaborative robots in collaborative assembly applications. Finally, this work suggests that future research could aim to validate the implementation process in a case study or investigate further the last two phases of the process. / Hittills har få tekniker kunnat öka flexibiliteten i tillverkarnas monteringsapplikationer, men introduktion av industrirobotar i samarbetande monteringsapplikationer öppnar upp för sådana möjligheter. Specifikt så presenterar dessa samarbetande monteringsapplikationer en möjlighet att, i en staketlös miljö, kombinera människans flexibilitet med industrirobotens precision, repeterbarhet och styrka men samtidigt nyttja litet golvutrymme och tillåta bärbarhet. Emellertid, trots fördelarna med industrirobotar i samarbetande monteringsapplikationer, finns det signifikanta gap i litteraturen som förhindrar dess implementering.  Baserat på denna bakgrund är syftet med detta arbete att stödja implementeringen av industrirobotar i samarbetande monteringsapplikationer.  För att fullfölja detta syfte inkluderade detta arbete två empiriska studier. Först, en intervjustudie som kartlagde attributen för industrirobotar i samarbetande monteringsapplikationer. För det andra, en flerfallstudie som kartlagde de kritiska utmaningarna och möjliggörande aktiviteterna för implementeringen av dessa samarbetande monteringsapplikationer. Studierna kombinerades också med litteraturstudier med målet att fylla de teoretiska gapen.  Detta arbete ger en implementeringsprocess med möjliggörande aktiviteter som kan mildra de kritiska utmaningarna under implementeringen av industrirobotar i samarbetande monteringsapplikationer. Implementeringsprocessen visar möjliggörande aktiviteter i de tre första faserna; förstudie, design av samarbetande monteringsapplikationer och monteringsinstallation.  Dessa möjliggörande aktiviteter är kartlagda mot 7M dimensionerna som ett sätt att tydligt visa hur dessa kan stödja implementeringen av industrirobotar i samarbetande monteringsapplikationer. Implementeringsprocessen bidrar till att fylla de identifierade gapen i litteraturen och ger till praktiker aktiviteter som ledare kan beakta vid implementeringen av industrirobotar i samarbetande monteringsapplikationer. Slutligen, detta arbete föreslår att framtida forskning syftar att validera implementeringsprocessen genom en fallstudie eller vidare undersöka de två sista faserna av denna process.

Collaborative robots are becoming more commonplace within factories to work alongside their human counterparts. With this newfound perspective towards robots being seen as collaborative partners comes the question of how interacting with these machines will change. This thesis therefore focuses on investigating the connection between facial expression communication in industrial robots and users' perceptions. Experiments were conducted to investigate the relationship between users' perceptions towards both existing facial expressions of the Baxter robot (an industrial robot by Rethink Robotics) and redesigned versions of these facial expressions. Findings reveal that the redesigned facial expressions provide a better match to users’ expectations. In addition, insights into improving the expressive communication between humans and robots are discussed, including the need for additional solutions which can complement the facial expressions displayed by providing more detailed information as needed. The last section of this thesis presents future research directions towards building a more intuitive and user-friendly human-robot cooperation space for future industrial robots.

Industries are becoming more and more demanding with new technology and trying to improve the productivity of their lines by adding technological methods and thus obtaining greater flexibility and time savings. One of these methods is to train their workers with the new augmented reality technology, which saves training time since the user can learn independently by following the steps indicated on the device. Another method is to add robots to the production line to carry out those tasks that are supposed to be repetitive and tiring for humans. To squeeze more and get the best out of the robot and the human, companies choose to combine their virtues and put them working together hand in hand as a human-robot collaboration.In this project, the demonstrator includes an assembly car process realized in a human-robot collaboration system, where the human and the collaborative robot communicate through an augmented reality device, Hololens2. This demonstrator might be used for user experience studies to evaluate if the human can realize an assembly process following the instructions in a head-mounted device without previous experience and collaborate with a robot. / <p>Program: - (Utbytesstudenter)</p>

<p> </p> <p>The usage of robots has increased in different areas of society and human work, including medicine, transportation, education, space exploration, and the service industry. This phenomenon has generated a sudden enthusiasm to develop more intelligent robots that are better equipped to perform tasks in a manner that is equivalently good as those completed by humans. Such jobs require human involvement as operators or teammates since robots struggle with automation in everyday settings. Soon, the role of humans will be far beyond users or stakeholders and include those responsible for training such robots. A popular teaching form is to allow robots to mimic human behavior. This method is intuitive and natural and does not require specialized knowledge of robotics. While there are other methods for robots to complete tasks effectively, collaborative tasks require mutual understanding and coordination that is best achieved by mimicking human motion. This mimicking problem has been tackled through skill imitation, which reproduces human-like motion during a task shown by a trainer. Skill imitation builds on faithfully replicating the human pose and requires two steps. In the first step, an expert's demonstration is captured and pre-processed, and motion features are obtained; in the second step, a learning algorithm is used to optimize for the task. The learning algorithms are often paired with traditional control systems to transfer the demonstration to the robot successfully. However, this methodology currently faces a generalization issue as most solutions are formulated for specific robots or tasks. The lack of generalization presents a problem, especially as the frequency at which robots are replaced and improved in collaborative environments is much higher than in traditional manufacturing. Like humans, we expect robots to have more than one skill and the same skills to be completed by more than one type of robot. Thus, we address this issue by proposing a human motion imitation framework that can be efficiently computed and generalized for different kinematic structures (e.g., different robots).</p> <p> </p> <p>This framework is developed by training an algorithm to augment collaborative demonstrations, facilitating the generalization to unseen scenarios. Later, we create a model for pose imitation that converts human motion to a flexible constraint space. This space can be directly mapped to different kinematic structures by specifying a correspondence between the main human joints (i.e., shoulder, elbow, wrist) and robot joints. This model permits having an unlimited number of robotic links between two assigned human joints, allowing different robots to mimic the demonstrated task and human pose. Finally, we incorporate the constraint model into a reward that informs a Reinforcement Learning algorithm during optimization. We tested the proposed methodology in different collaborative scenarios. Thereafter, we assessed the task success rate, pose imitation accuracy, the occlusion that the robot produces in the environment, the number of collisions, and finally, the learning efficiency of the algorithm.</p> <p> </p> <p>The results show that the proposed framework creates effective collaboration in different robots and tasks.</p>

<p>  </p> <p>Manufacturing flexibility and human-centered designs are promising approaches to face the demand for individualized products. Human-robot assembly cells still lack flexibility and adaptability (VDI, 2017) using static control architectures (Bessler et al., 2020). Autonomous adaptation to human operators in short time horizons increases the willingness to work with cobots. Besides, monotonous static assembling in manufacturing operations does not accommodate the human way of working. Therefore, Human-Robot Collaboration (HRC) workstations require a work behavior adaptation accommodating varying work behavior regarding human mental and physical conditions (Weiss et al., 2021). The thesis presents the development of a cyber-physical HRC assembly station.</p> <p>Moreover, the thesis includes an experimental study investigating the influence of a cobot’s speed on human work behavior. The Cyber-Physical System (CPS) integrates the experiment's findings with event-based software architecture and a semantic knowledge representation. Thereby, the work focuses on demonstrating the feasibility of the CPS and the semantic model, allowing the self-adaptation of the system. Finally, the conclusion identifies the need for further research in human work behavior detection and fuzzy decision models. Such detection and decision models could improve self-adaptation in human-centered assembly systems.</p>

As Industry 4.0 continues to evolve, human-robot collaboration, HRC, has become more common in industries. This study aimed to explore perceived safety in HRC within manufacturing, with a focus on the assembly processes at Volvo. The goal was to promote perceived safety among operators by applying the Human-Technology-Organization, HTO, perspective, including Safety-I, -II, and -III. A framework was developed to illustrate the aim in relation to the theory and the approach taken in the study. The Volvo case RITA, a collaborative robot designed to assist with kitting, was used as a use case in the study. Numerous interviews were conducted with organizational representatives and assembly line operators with a complementary questionnaire. Since RITA was not operational, a video of the case was utilized extensively throughout the study. Operator interviews were centered on gathering their insights on perceived safety, drawing from the above safety perspectives. The formulated recommendations emphasized the importance of comprehensive operator training and early involvement in new development processes. Various traffic rules were devised for different collaboration scenarios, and the significance of clear workspaces was underscored to maintain system efficiency. These recommendations were later validated by an organizational representative from Volvo. Lastly, the study emphasizes that while technical solutions for safety are necessary, they are not sufficient without a strong safety culture that encourages openness and collaboration. By considering technical, organizational, and human aspects of safety, this study contributes to a deeper understanding of the dynamics in HRC and lays the foundation for safe and efficient manufacturing processes.

Planning is the sub-area of artificial intelligence that studies the process of selecting actions to lead an agent, e.g. a robot or a softbot, to a goal state. In many realistic scenarios, any choice of actions can lead the robot into a dead-end state, that is, a state from which the goal cannot be reached. In such cases, the robot can, pro-actively, resort to human help in order to reach the goal, an approach called symbiotic autonomy. In this work, we propose two different approaches to tackle this problem: (I) contingent planning, where the initial state is partially observable, configuring a belief state, and the outcomes of the robot actions are non-deterministic; and (II) probabilistic planning, where the initial state may be partially or totally observable and the actions have probabilistic outcomes. In both approaches, the human help is considered a scarce resource that should be used only when necessary. In contingent planning, the problem is to find a policy (a function mapping belief states into actions) that: (i) guarantees the agent will always reach the goal (strong policy); (ii) guarantees that the agent will eventually reach the goal (strong cyclic policy), or (iii) does not guarantee achieving the goal (weak policy). In this scenario, we propose a contingent planning system that considers human help to transform weak policies into strong (cyclic) policies. To do so, two types of human help are included: (i) human actions that modify states and/or belief states; and (ii) human observations that modify belief states. In probabilistic planning, the problem is to find a policy (a function mapping between world states and actions) that can be one of these two types: a proper policy, where the agent has probability 1 of reaching the goal; or an improper policy, in the case of unavoidable dead-ends. In general, the goal of the agent is to find a policy that minimizes the expected accumulated cost of the actions while maximizes the probability of reaching the goal. In this scenario, this work proposes probabilistic planners that consider human help to transform improper policies into proper policies however, considering two new (alternative) criteria: either to minimize the probability of using human actions or to minimize the expected number of human actions. Furthermore, we show that optimal policies under these criteria can be efficiently computed either by increasing human action costs or given a penalty when a human help is used. Solutions proposed in both scenarios, contingent planning and probabilistic planning with human help, were evaluated over a collection of planning problems with dead-ends. The results show that: (i) all generated policies (strong (cyclic) or proper) include human help only when necessary; and (ii) we were able to find policies for contingent planning problems with up to 10^15000 belief states and for probabilistic planning problems with more than 3*10^18 physical states. / Planejamento é a subárea de Inteligência Artificial que estuda o processo de selecionar ações que levam um agente, por exemplo um robô, de um estado inicial a um estado meta. Em muitos cenários realistas, qualquer escolha de ações pode levar o robô para um estado que é um beco-sem-saída, isto é, um estado a partir do qual a meta não pode ser alcançada. Nestes casos, o robô pode, pró-ativamente, pedir ajuda humana para alcançar a meta, uma abordagem chamada autonomia simbiótica. Neste trabalho, propomos duas abordagens diferentes para tratar este problema: (I) planejamento contingente, em que o estado inicial é parcialmente observável, configurando um estado de crença, e existe não-determinismo nos resultados das ações; e (II) planejamento probabilístico, em que o estado inicial é totalmente observável e as ações tem efeitos probabilísticos. Em ambas abordagens a ajuda humana é considerada um recurso escasso e deve ser usada somente quando estritamente necessária. No planejamento contingente, o problema é encontrar uma política (mapeamento entre estados de crença e ações) com: (i) garantia de alcançar a meta (política forte); (ii) garantia de eventualmente alcançar a meta (política forte-cíclica), ou (iii) sem garantia de alcançar a meta (política fraca). Neste cenário, uma das contribuições deste trabalho é propor sistemas de planejamento contingente que considerem ajuda humana para transformar políticas fracas em políticas fortes (cíclicas). Para isso, incluímos ajuda humana de dois tipos: (i) ações que modificam estados do mundo e/ou estados de crença; e (ii) observações que modificam estados de crenças. Em planejamento probabilístico, o problema é encontrar uma política (mapeamento entre estados do mundo e ações) que pode ser de dois tipos: política própria, na qual o agente tem probabilidade 1 de alcançar a meta; ou política imprópria, caso exista um beco-sem-saída inevitável. O objetivo do agente é, em geral, encontrar uma política que minimize o custo esperado acumulado das ações enquanto maximize a probabilidade de alcançar a meta. Neste cenário, este trabalho propõe sistemas de planejamento probabilístico que considerem ajuda humana para transformar políticas impróprias em políticas próprias, porém considerando dois novos critérios: minimizar a probabilidade de usar ações do humano e minimizar o número esperado de ações do humano. Mostramos ainda que políticas ótimas sob esses novos critérios podem ser computadas de maneira eficiente considerando que ações humanas possuem um custo alto ou penalizando o agente ao pedir ajuda humana. Soluções propostas em ambos cenários, planejamento contingente e planejamento probabilístico com ajuda humana, foram empiricamente avaliadas sobre um conjunto de problemas de planejamento com becos-sem-saida. Os resultados mostram que: (i) todas as políticas geradas (fortes (cíclicas) ou próprias) incluem ajuda humana somente quando necessária; e (ii) foram encontradas políticas para problemas de planejamento contingente com até 10^15000 estados de crença e para problemas de planejamento probabilístico com até 3*10^18 estados do mundo.

Que ce soit dans un contexte industriel ou quotidien, les robots deviennent de plus en plus présents dans notre environnement et sont désormais capables d'interagir avec des humains. Dans les milieux industriels, des robots viennent notamment assister les opérateurs des chaînes d'assemblage pour des tâches fatigantes et dangereuses. Robots et opérateurs sont alors amenés à partager le même espace physique (coprésence) et à effectuer des tâches en commun (collaboration). Alors que la sécurité des humains à proximité des robots doit être garantie à tout instant, il convient également de déterminer si le travail collaboratif est accepté par les opérateurs, en termes d'utilisabilité et d'utilité.La première problématique de la thèse consiste à déterminer quelles sont les composantes importantes rentrant en jeu dans l'acceptabilité de la collaboration homme-robot, du point de vue des opérateurs. Différents facteurs peuvent influencer cette acceptabilité : l'apparence des robots et leurs mouvements, la distance de sécurité ou encore le mode d'interaction avec le robot.Afin d'étudier le maximum de facteurs, nous proposons d'utiliser la réalité virtuelle pour mener des tests utilisateurs en environnement virtuel. Nous utilisons des questionnaires pour recueillir les impressions subjectives des opérateurs et des mesures physiologiques pour estimer leur état affectif (stress, effort). La deuxième problématique de la thèse consiste à déterminer si une méthodologie utilisant la réalité virtuelle est pertinente pour cette évaluation : les résultats issus des tests en environnement virtuel rendent-ils bien compte de la situation réelle ?Pour répondre aux problématiques de la thèse, trois cas d'étude ont été mis en place et quatre expérimentations ont été menées. Deux de ces expérimentations ont été reproduites à la fois en environnements réel et virtuel afin d'évaluer la pertinence des résultats issus de la situation virtuelle par rapport à la situation réelle. / Either in the context of the industry or of the everyday life, robots are becoming more and more present in our environment and are nowadays able to interact with humans. In industrial environments, robots now assist operators on the assembly lines for difficult and dangerous tasks. Then, robots and operators need to share the same physical space (copresence) and to manage common tasks (collaboration). On the one side, the safety of humans working near robots has to be guaranteed at all time. On the other hand, it is necessary to determine if such a collaborative work is accepted by the operators, in terms of usability and utility.The first problematic of the thesis consists in determining the important criteria that play a role in the acceptability, from the operators' point of view. Different factors can influence this acceptability: robot appearance, robot movements, safety distance or interaction modes with the robot.In order to study as many factors as possible, we intend to use virtual reality to perform user studies in virtual environments. We are using questionnaires to gather subjective impressions from operators and physiological measures to estimate their affective states (stress, effort). The second problematic of the thesis consists in determining if a methodology using virtual reality is relevant for this evaluation: can the results from studies in virtual environments be reproducible in equivalent physical situations?In order to answer the problematics of the thesis, three use cases have been implemented and four studies have been performed. Two of those studies rely on a physical situation and its virtual reality counterpart in order to evaluate the relevance of the results of the virtual situation compared to the real situation.

In Human-Robot Collaboration scenarios, safety must be ensured by a risk management process that requires the execution of computationally expensive perception models (e.g., based on computer vision) in real-time. However, robots usually have constrained hardware resources that hinder timely responses, resulting in unsafe operations. Although Multi-access Edge Computing allows robots to offload complex tasks to servers on the network edge to meet real-time requirements, this might not always be possible due to dynamic changes in the network that can cause congestion or failures. This work proposes a safety-based task offloading strategy to address this problem. The goal is to intelligently use edge resources to reduce delays in the risk management process and consequently enhance safety. More specifically, depending on safety and network metrics, a Reinforcement Learning (RL) solution is implemented to decide whether a less accurate model should run locally on the robot or a more complex one should run remotely on the network edge. A third possibility is to reuse the previous output through verification of temporal coherence. Experiments are performed in a simulated warehouse scenario where humans and robots have close interactions. Results show that the proposed RL solution outperforms the baselines in several aspects. First, the edge is used only when the network performance is good, reducing the number of failures (up to 47%). Second, the latency is also adapted to the safety requirements (risk X latency reduced up to 48%), avoiding unnecessary network congestion in safe situations and letting other robots in hazardous situations use the edge. Overall, the latency of the risk management process is largely reduced (up to 68%), and this positively affects safety (time in safe zone increased up to 3:1%). / I scenarier med människa-robotkollaboration måste säkerheten säkerställas via en riskhanteringsprocess. Denna process kräver exekvering av beräkningstunga uppfattningsmodeller (t.ex. datorseende) i realtid. Robotar har vanligtvis begränsade hårdvaruresurser vilket förhindrar att respons uppnås i tid, vilket resulterar i osäkra operationer. Även om Multi-access Edge Computing tillåter robotar att avlasta komplexa uppgifter till servrar på edge, för att möta realtidskraven, så är detta inte alltid möjligt på grund av dynamiska förändringar i nätverket som kan skapa överbelastning eller fel. Detta arbete föreslår en säkerhetsbaserad uppgiftsavlastningsstrategi för att hantera detta problem. Målet är att intelligent använda edge-resurser för att minska förseningar i riskhanteringsprocessen och följaktligen öka säkerheten. Mer specifikt, beroende på säkerhet och nätverksmätvärden, implementeras en Reinforcement Learning (RL) lösning för att avgöra om en modell med mindre noggrannhet ska köras lokalt eller om en mer komplex ska köras avlägset på edge. En tredje möjlighet är att återanvända sista utmatningen genom verifiering av tidsmässig koherens. Experimenten utförs i ett simulerat varuhusscenario där människor och robotar har nära interaktioner. Resultaten visar att den föreslagna RL-lösningen överträffar baslinjerna i flera aspekter. För det första används edge bara när nätverkets prestanda är bra, vilket reducerar antal fel (upp till 47%). För det andra anpassas latensen också till säkerhetskraven (risk X latens reducering upp till 48%), undviker onödig överbelastning i nätverket i säkra situationer och låter andra robotar i farliga situationer använda edge. I det stora hela reduceras latensen av riskhanterings processen kraftigt (upp till 68%) och påverkar på ett positivt sätt säkerheten (tiden i säkerhetszonen ökas upp till 4%).