The main task in this thesis is to design a robot vision system with face detection and tracking capability. Hence there are two main works in the thesis: Firstly, the detection of the face on an image that is taken from the camera on the robot must be achieved. Hence this is a serious real time image processing task and time constraints are very important because of this reason. A processing rate of 1 frame/second is tried to be achieved and hence a fast face detection algorithm had to be used. The Eigenface method and the Subspace LDA (Linear Discriminant Analysis) method are implemented, tested and compared for face detection and Eigenface method proposed by Turk and Pentland is decided to be used. The images are first passed through a number of preprocessing algorithms to obtain better performance, like skin detection, histogram equalization etc. After this filtering process the face candidate regions are put through the face detection algorithm to understand whether there is a face or not in the image. Some modifications are applied to the eigenface algorithm to detect the faces better and faster. Secondly, the robot must move towards the face in the image. This task includes robot motion. The robot to be used for this purpose is a Pioneer 2-DX8 Plus, which is a product of ActivMedia Robotics Inc. and only the interfaces to move the robot have been implemented in the thesis software. The robot is to detect the faces at different distances and arrange its position according to the distance of the human to the robot. Hence a scaling mechanism must be used either in the training images, or in the input image taken from the camera. Because of timing constraint and low camera resolution, a limited number of scaling is applied in the face detection process. With this reason faces of people who are very far or very close to the robot will not be detected. A background independent face detection system is tried to be designed. However the resultant algorithm is slightly dependent to the background. There is no any other constraints in the system.

This thesis describes a system which is able to track and imitate human motion. The system is divided into two major parts: computer vision system and robot arm motion control system. Through the use of two real-time video cameras, computer vision system identifies the moving object depending on the colour features, as the object colour is matched within the colour range in the current image frame, a method that employs two vectors is used to calculate the coordinates of the object. After the object is detected and tracked coordinates are saved to a pre-establish database in the purpose of further data processing, a mathematical algorithm is performed to the data in order to give a better robotic motion control. Robot arm manipulator responds with a move within its workspace which corresponds to a consequential human-type motion. Experimental outcomes have shown that the system is reliable and can successfully imitate a human hand motion in most cases.

L’audition est une modalité utile pour aider un robot à explorer et comprendre son environnement sonore. Dans cette thèse, nous nous intéressons à la tâche de localiser une ou plusieurs sources sonores mobiles et intermittentes à l’aide d’un robot mobile équipé d’une antenne de microphones en exploitant la mobilité du robot pour améliorer la localisation. Nous proposons d’abord un modèle bayésien pour localiser une seule source mobile intermittente. Ce modèle estime conjointement la position et l’activité de la source au cours du temps et s’applique à tout type d’antenne. Grâce au mouvement du robot, il peut estimer la distance de la source et résoudre l’ambiguïté avant-arrière qui apparaît dans le cas des antennes linéaires. Nous proposons deux implémentations de ce modèle, l’une à l’aide d’un filtre de Kalman étendu basé sur des mélanges de gaussiennes et l’autre à l’aide d’un filtre à particules, que nous comparons en termes de performance et de temps de calcul. Nous étendons ensuite notre modèle à plusieurs sources intermittentes et mobiles. En combinant notre filtre avec un joint probability data association filter (JPDAF), nous pouvons estimer conjointement les positions et activités de deux sources sonores dans un environnement réverbérant. Enfin nous faisons une contribution à la planification de mouvement pour réduire l’incertitude sur la localisation d’une source sonore. Nous définissons une fonction de coût avec l’alternative entre deux critères: l’entropie de Shannon ou l’écart-type sur l’estimation de la position. Ces deux critères sont intégrés dans le temps avec un facteur d’actualisation. Nous adaptons alors l’algorithme de Monte-Carlo tree search (MCTS) pour trouver, efficacement, le mouvement du robot qui minimise notre fonction de coût. Nos expériences montrent que notre méthode surpasse, sur le long terme, d’autres méthodes de planification pour l’audition robotique / Robot audition provides hearing capability for robots and helps them explore and understand their sound environment. In this thesis, we focus on the task of sound source localization for a single or multiple, intermittent, possibly moving sources using a mobile robot and exploiting robot motion to improve the source localization. We propose a Bayesian filtering framework to localize the position of a single, intermittent, possibly moving sound source. This framework jointly estimates the source location and its activity over time and is applicable to any micro- phone array geometry. Thanks to the movement of the robot, it can estimate the distance to the source and solve the front-back ambiguity which appears in the case of a linear microphone array. We propose two implementations of this framework based on an extended mixture Kalman filter (MKF) and on a particle filter, that we compare in terms of performance and computation time. We then extend our model to the context of multiple, intermittent, possibly moving sources. By implementing an extended MKF with joint probabilistic data association filter (JPDAF), we can jointly estimate the locations of two sources and their activities over time. Lastly, we make a contribution on long-term robot motion planning to optimally reduce the uncertainty in the source location. We define a cost function with two alternative criteria: the Shannon entropy or the standard deviation of the estimated belief. These entropies or standard deviations are integrated over time with a discount factor. We adapt the Monte Carlo tree search (MCTS) method for efficiently finding the optimal robot motion that will minimize the above cost function. Experiments show that the proposed method outperforms other robot motion planning methods for robot audition in the long run

The use of robotics has recently seen significant growth in various domains such as unmanned ground/underwater/aerial vehicles, smart manufacturing, and humanoid robots. However, one of the most important and essential capabilities required for long term autonomy, which is the ability to operate robustly and safely in real-world environments, in contrast to industrial and laboratory setup is largely missing. Designing robots that can operate reliably and efficiently in cluttered and changing environments is non-trivial, especially for high degree-of-freedom (DoF) systems, i.e. robots with multiple actuators. On one hand, the dexterity offered by the kinematic redundancy allows the robot to perform dexterous manipulation tasks in complex environments, whereas on the other hand, such complex system also makes controlling and planning very challenging. To address such two interrelated problems, we exploit robot motion synthesis from three perspectives that feed into each other: end-pose planning, motion planning and motion adaptation. We propose several novel ideas in each of the three phases, using which we can efficiently synthesise dexterous manipulation motion for fixed-base robotic arms, mobile manipulators, as well as humanoid robots in cluttered and potentially changing environments. Collision-free inverse kinematics (IK), or so-called end-pose planning, a key prerequisite for other modules such as motion planning, is an important and yet unsolved problem in robotics. Such information is often assumed given, or manually provided in practice, which significantly limiting high-level autonomy. In our research, by using novel data pre-processing and encoding techniques, we are able to efficiently search for collision-free end-poses in challenging scenarios in the presence of uneven terrains. After having found the end-poses, the motion planning module can proceed. Although motion planning has been claimed as well studied, we find that existing algorithms are still unreliable for robust and safe operations in real-world applications, especially when the environment is cluttered and changing. We propose a novel resolution complete motion planning algorithm, namely the Hierarchical Dynamic Roadmap, that is able to generate collision-free motion trajectories for redundant robotic arms in extremely complicated environments where other methods would fail. While planning for fixed-base robotic arms is relatively less challenging, we also investigate into efficient motion planning algorithms for high DoF (30 - 40) humanoid robots, where an extra balance constraint needs to be taken into account. The result shows that our method is able to efficiently generate collision-free whole-body trajectories for different humanoid robots in complex environments, where other methods would require a much longer planning time. Both end-pose and motion planning algorithms compute solutions in static environments, and assume the environments stay static during execution. While human and most animals are incredibly good at handling environmental changes, the state-of-the-art robotics technology is far from being able to achieve such an ability. To address this issue, we propose a novel state space representation, the Distance Mesh space, in which the robot is able to remap the pre-planned motion in real-time and adapt to environmental changes during execution. By utilizing the proposed end-pose planning, motion planning and motion adaptation techniques, we obtain a robotic framework that significantly improves the level of autonomy. The proposed methods have been validated on various state-of-the-art robot platforms, such as UR5 (6-DoF fixed-base robotic arm), KUKA LWR (7-DoF fixed-base robotic arm), Baxter (14-DoF fixed-base bi-manual manipulator), Husky with Dual UR5 (15-DoF mobile bi-manual manipulator), PR2 (20-DoF mobile bi-manual manipulator), NASA Valkyrie (38-DoF humanoid) and many others, showing that our methods are truly applicable to solve high dimensional motion planning for practical problems.

This project proposes an adaptation of the dRRT* algorithm, a samplingbased multi-robot planner, for real-time industrial automation scenarios. The main objectives include optimizing computationally expensive sections of the algorithm, solving partially specified multi-robot problems, and evaluating the performance of the resulting method in various industry-like scenarios. The proposed algorithm, called Fast-dRRT*, aims to achieve highquality collision-free paths within strict time constraints. To accomplish this, the project introduces modifications to the dRRT* algorithm, such as the utilization of pre-computed swept volumes for efficient collision detection. The performance of four multi-robot planners, namely dRRT, ao-dRRT, dRRT*, and Fast-dRRT*, is evaluated through experiments on toy scenarios and industrial use cases. The results show that the proposed Fast-dRRT* algorithm outperforms the other planners in terms of finding solutions within the given time limits. It demonstrates improved efficiency, speed, and resilience to increased search spaces and the number of robots. The algorithm’s performance is further evaluated in real-world scenarios, including automotive assembly lines and warehouse automation, where it consistently outperforms dRRT* in terms of search speed and total planning time. Additionally, the algorithm successfully handles partially specified multi-robot problems, optimizing the overall movements’ cost. The study concludes that Fast-dRRT* is a promising option for real-time planning in industrial automation, offering reduced computation time and feasible solutions to complex multi-robot motion planning problems. / Detta projekt föreslår en anpassning av dRRT*-algoritmen, en samplingsbaserad multirobotplanerare, för realtidsscenarier inom industriell automation.. De huvudsakliga målen inkluderar optimering av beräkningskrävande delar av algoritmen, lösning av delvis specificerade multirobotproblem och utvärdering av den resulterande metodens prestanda i olika industriliknande scenarier. Den föreslagna algoritmen, kallad Fast-dRRT*, syftar till att uppnå högkvalitativa kollisionsfria banor inom strikta tidsbegränsningar. För att uppnå detta introducerar projektet modifieringar av dRRT*-algoritmen, såsom användning av förberäknade svepta volymer för effektiv kollisionsdetektering. Prestandan hos fyra multirobotplanerare, nämligen dRRT, ao-dRRT, dRRT* och Fast-dRRT*, utvärderas genom experiment på leksaksscenarier och industriella användningsfall. Resultaten visar att den föreslagna Fast-dRRT*- algoritmen överträffar de andra planerarna när det gäller att hitta lösningar inom de givna tidsgränserna. Den visar förbättrad effektivitet, hastighet och motståndskraft mot ökade sökutrymmen och antalet robotar. Algoritmens prestanda utvärderas vidare i verkliga scenarier, inklusive monteringslinjer för bilar och lagerautomation, där den konsekvent överträffar dRRT* när det gäller sökhastighet och total planeringstid. Dessutom hanterar algoritmen framgångsrikt delvis specificerade multirobotproblem och optimerar den totala rörelsekostnaden. Studien drar slutsatsen att Fast-dRRT* är ett lovande alternativ för realtidsplanering inom industriell automation, eftersom den erbjuder kortare beräkningstid och genomförbara lösningar på komplexa problem med rörelseplanering för multirobotar.

Pour les robots mobiles autonomes conçus pour partager notre environnement, la sécurité et l'efficacité de leur trajectoire ne sont pas les seuls aspects à prendre en compte pour la planification de leur mouvement: ils doivent respecter des règles sociales afin de ne pas gêner les personnes environnantes. Dans un tel contexte social, la plupart des techniques de planification de mouvement actuelles s'appuient fortement sur le concept d'espaces sociaux; de tels espaces sociaux sont cependant difficiles à modéliser et ils sont d'une utilisation limitée dans le contexte d'interactions homme-robot où l'intrusion dans les espaces sociaux est nécessaire. Ce travail présente une nouvelle approche pour la planification de mouvements dans un contexte social qui permet de gérer des environnements complexes ainsi que des situation d’interaction homme-robot. Plus précisément, le concept d'attention est utilisé pour modéliser comment l'influence de l'environnement dans son ensemble affecte la manière dont le mouvement du robot est perçu par les personnes environnantes. Un nouveau modèle attentionnel est introduit qui estime comment nos ressources attentionnelles sont partagées entre les éléments saillants de notre environnement. Basé sur ce modèle, nous introduisons le concept de champ attentionnel. Un planificateur de mouvement est ensuite développé qui s'appuie sur le champ attentionnel afin de produire des mouvements socialement acceptables. Notre planificateur de mouvement est capable d'optimiser simultanément plusieurs objectifs tels que la sécurité, l'efficacité et le confort des mouvements. Les capacités de l'approche proposée sont illustrées sur plusieurs scénarios simulés dans lesquels le robot est assigné différentes tâches. Lorsque la tâche du robot consiste à naviguer dans l'environnement sans causer de distraction, notre approche produit des résultats prometteurs même dans des situations complexes. Aussi, lorsque la tâche consiste à attirer l'attention d'une personne en vue d'interagir avec elle, notre planificateur de mouvement est capable de choisir automatiquement une destination qui exprime au mieux son désir d'interagir, tout en produisant un mouvement sûr, efficace et confortable. / For autonomous mobile robots designed to share their environment with humans, path safety and efficiency are not the only aspects guiding their motion: they must follow social rules so as not to cause discomfort to surrounding people. Most socially-aware path planners rely heavily on the concept of social spaces; however, social spaces are hard to model and they are of limited use in the context of human-robot interaction where intrusion into social spaces is necessary. In this work, a new approach for socially-aware path planning is presented that performs well in complex environments as well as in the context of human-robot interaction. Specifically, the concept of attention is used to model how the influence of the environment as a whole affects how the robot's motion is perceived by people within close proximity. A new computational model of attention is presented that estimates how our attentional resources are shared amongst the salient elements in our environment. Based on this model, the novel concept of attention field is introduced and a path planner that relies on this field is developed in order to produce socially acceptable paths. To do so, a state-of-the-art many-objective optimization algorithm is successfully applied to the path planning problem. The capacities of the proposed approach are illustrated in several case studies where the robot is assigned different tasks. Firstly, when the task is to navigate in the environment without causing distraction our approach produces promising results even in complex situations. Secondly, when the task is to attract a person's attention in view of interacting with him or her, the motion planner is able to automatically choose a destination that best conveys its desire to interact whilst keeping the motion safe, efficient and socially acceptable.

The aim of this work was to design the construction of transport equipment under industrial robot, pallet holder or other equipment. This manipulation device can be moved in one axis on a rigid guide. The device is designed for use in a linear manufacturing system for handling workpieces and tools. It is capable of operating two rows of machines when it is placed in the middle. The main function of this device is to extend the workspace to the carried devices. This type of construction can be used in combination with an industrial robot for other technological tasks such as welding or light machining. The main parts of this device include statically mounted beds that can be stacked behind each other to form optionally long conveyor paths and a positioning plate that moves along the path formed by these beds. An industrial robot, pallet holder or other device is attached to this positioning plate. In addition to the design itself, the thesis also includes a thorough research of the issue, further possibilities of solution in the form of concepts, verification of the structure by means of calculations and simulations and implementation of the solution into a specific production cell.