Autonomous robots are set to become ubiquitous in the future, with applications ranging from autonomous cars to assistive household robots. These systems must operate in close proximity of dynamic and static objects, including humans and other non-autonomous systems, adding complexity to their decision-making processes. The behaviour of such objects is often stochastic and hard to predict. Making robust decisions under such uncertain scenarios can be challenging for these autonomous robots. In the past, researchers have used deterministic approach to predict the motion of surrounding objects. However, these approaches can be over-confident and do not capture the stochastic behaviour of surrounding objects necessary for safe decision-making. In this dissertation, we show the importance of probabilistic prediction of surrounding dynamic objects and their incorporation into planning for safety-critical decision making. We utilise Bayesian inference models such as Monte Carlo dropout and deep ensemble to probabilistically predict the motion of surrounding objects. Our probabilistic trajectory forecasting model showed improvement over standard deterministic approaches and could handle adverse scenarios such as sensor noise and occlusion during prediction. The uncertainty-inclusive prediction of surrounding objects has been incorporated into planning. The inclusion of predicted states of surrounding objects with associated uncertainty enables the robot make proactive decisions while avoiding collisions. / Doctor of Philosophy / In future, humans will greatly rely on the assistance of autonomous robots in helping them with everyday tasks. Drones to deliver packages, cars for driving to places autonomously and household robots helping with day-to-day activities. In all such scenarios, the robot might have to interact with their surrounding, in particular humans. Robots working in close proximity to humans must be intelligent enough to make safe decisions not affecting or intruding the human. Humans, in particular make abrupt decisions and their motion can be unpredictable. It is necessary for the robot to understand the intention of human for navigating safely without affecting the human. Therefore, the robot must capture the uncertain human behaviour and predict its future motion so that it can make proactive decisions. We propose to capture the stochastic behaviour of humans using deep learning based prediction models by learning motion patterns from real human trajectories. Our method not only predicts future trajectory of humans but also captures the associated uncertainty during prediction. In this thesis, we also propose how to predict human motion under adverse scenarios like bad weather leading to noisy sensing as well as under occlusion. Further, we integrate the predicted stochastic behaviour of surrounding humans into the planning of the robot for safe navigation among humans.

No description available.

Doctor of Philosophy / Self-driving cars promise a future with safer roads and reduced traffic incidents and fatalities. This future hinges on the car's accurate understanding of its surrounding environment; however, the reliability of the algorithms that form this perception is not always guaranteed and adverse traffic and environmental conditions can significantly diminish the performance of these algorithms. To solve this problem, this research builds on the idea that enabling cars to share and exchange information via communication allows them to extend the range and quality of their perception beyond their capability. To that end, this research formulates a robust and flexible framework for cooperative perception, explores how connected vehicles can learn to collaborate to improve their perception, and introduces an affordable, experimental vehicle platform for connected autonomy research.

With connected and autonomous vehicles, no optimal standard or framework currently exists, outlining the right level of information sharing for cooperative autonomous driving. Cooperative Perception is proposed among vehicles, where every vehicle is transformed into a moving sensor platform that is capable of sharing information collected using its on-board sensors. This helps extend the line of sight and field of view of autonomous vehicles, which otherwise suffer from blind spots and occlusions. This increase in situational awareness promotes safe driving over a short range and improves traffic flow efficiency over a long range. This thesis proposes a methodology for cooperative perception for autonomous vehicles over a short range. The problem of cooperative perception is broken down into sub-tasks of cooperative relative localization and map merging. Cooperative relative localization is achieved using visual and inertial sensors, where a computer-vision based camera relative pose estimation technique, augmented with position information, is used to provide a pose-fix that is subsequently updated by dead reckoning using an inertial sensor. Prior to map merging, a technique for object localization using a monocular camera is proposed that is based on the Inverse Perspective Mapping technique. A mobile multi-robot testbed was developed to emulate autonomous vehicles and the proposed method was implemented on the testbed to detect pedestrians and also to respond to the perceived hazard. Potential traffic scenarios where cooperative perception could prove crucial were tested and the results are presented in this thesis. / MS / Perception in Autonomous Vehicles is limited to the field of view of the vehicles’ onboard sensors and the environment may not be fully perceivable due to the presence of blind spots and occlusions. To overcome this limitation, Vehicle-to-Vehicle wireless communication could be leveraged to exchange locally sensed information among vehicles within the vicinity. Vehicles may share information about their own position, heading and velocity or go one step further and share information about their surroundings as well. This latter form of cooperative perception extends each vehicle’s field of view and line of sight, and helps increase situational awareness. The result is an increase in safety over a short range whereas communication over a long range could help improve traffic flow efficiency. This thesis proposes one such technique for cooperative perception over a short range. The system uses visual and inertial sensors to perform cooperative localization between two vehicles sharing a common field of view, which allows one vehicle to locate the other vehicle in its frame of reference. Subsequently, information about objects in the surroundings of one vehicle, localized using a visual sensor is relayed to the other vehicle through communication. A mobile multi-robot testbed was developed to emulate autonomous vehicles and to experimentally evaluate the proposed method through a series of driving scenario test cases in which cooperative perception could be effective and crucial to the safety and comfort of driving.

The research theme of this dissertation is the multiple-vehicles cooperative perception (or cooperative perception) applied in the context of intelligent vehicle systems. The general methodology of the presented works in this dissertation is to realize multiple-intelligent vehicles cooperative perception, which aims at providing better vehicle perception result compared with single vehicle perception (or non-cooperative perception). Instead of focusing our research works on the absolute performance of cooperative perception, we focus on the general mechanisms which enable the realization of cooperative localization and cooperative mapping (and moving objects detection), considering that localization and mapping are two underlying tasks for an intelligent vehicle system. We also exploit the possibility to realize certain augmented reality effect with the help of basic cooperative perception functionalities; we name this kind of practice as cooperative augmented reality. Naturally, the contributions of the presented works consist in three aspects: cooperative localization, cooperative local mapping and moving objects detection, and cooperative augmented reality.

The understanding of the environment is the critical ability not only for the living creature but also for automation fields like the robot, automated car, and intelligent system. Especially for some essential task in the domain of automotive such as autonomous driving, path planning, localization, and object detection, the more information we gather, the better the result we get. Intelligent vehicle technology relies on sensorial perception to understand the surroundings of the vehicle. The objective of the research is developing a cooperative online perception system with semantic segmentation for automated driving and improving the current semantic segmentation framework to make it more robust and more suitable for our future projects.

Le thème de recherche de cette thèse est la perception coopérative multi-véhicules appliquée au contexte des systèmes de véhicules intelligents. L’objectif général des travaux présentés dans cette thèse est de réaliser la perception coopérative de plusieurs véhicules (dite « perception coopérative »), visant ainsi à fournir des résultats de perception améliorés par rapport à la perception d’un seul véhicule (ou « perception non-coopérative »). Au lieu de concentrer nos recherches sur la performance absolue de la perception coopérative, nous nous concentrons sur les mécanismes généraux qui permettent la réalisation de la localisation coopérative et de la cartographie de l’environnement routier (y compris la détection des objets), considérant que la localisation et la cartographie sont les deux tâches les plus fondamentales pour un système de véhicule intelligent. Nous avons également exploité la possibilité d’explorer les techniques de la réalité augmentée, combinées aux fonctionnalités de perception coopérative. Nous baptisons alors cette approche « réalité augmentée coopérative ». Par conséquent, nous pouvons d’ores et déjà annoncer trois contributions des travaux présentés: la localisation coopérative, la cartographie locale coopérative, et la réalité augmentée coopérative. / The research theme of this dissertation is the multiple-vehicles cooperative perception (or cooperative perception) applied in the context of intelligent vehicle systems. The general methodology of the presented works in this dissertation is to realize multiple-intelligent vehicles cooperative perception, which aims at providing better vehicle perception result compared with single vehicle perception (or non-cooperative perception). Instead of focusing our research works on the absolute performance of cooperative perception, we focus on the general mechanisms which enable the realization of cooperative localization and cooperative mapping (and moving objects detection), considering that localization and mapping are two underlying tasks for an intelligent vehicle system. We also exploit the possibility to realize certain augmented reality effect with the help of basic cooperative perception functionalities; we name this kind of practice as cooperative augmented reality. Naturally, the contributions of the presented works consist in three aspects: cooperative localization, cooperative local mapping and moving objects detection, and cooperative augmented reality.

Dans le cadre des systèmes de transport intelligents (STI), les véhicules peuvent avoir beaucoup de capteurs (caméras, lidars, radars, etc.) et d’applications (évitement des collisions, surveillance du trafic, etc.) générant des données. Ils représentent alors une source d’information importante. Les applications locales peuvent augmenter considérablement leur efficacité en partageant une telle information au sein du réseau. La précision des données, la confiance et la pertinence peuvent être vérifiées lors de la réception de données provenant d’autres nœuds. Par conséquent, nous croyons qu’une question importante à répondre dans ce contexte est: “Comment partager efficacement les données dans un tel environnement?” Le partage de données est une tâche complexe dans les réseaux dynamiques. De nombreuses problèmes telles que les connexions intermittentes, la variation de la densité du réseau et la congestion du médium de communication se posent. Une approche habituelle pour gérer ces problèmes est basée sur des processus périodiques. En effet, un message envoyé plusieurs fois peut atteindre sa destination même avec des connexions intermittentes et des réseaux à faible densité. Néanmoins, dans les réseaux à haute densité, ils peuvent entraîner une congestion du médium de communication. Dans cette thèse, nous abordons le problème du partage de données dans des réseaux dynamiques en nous appuyant sur des horizons de pertinence. Un horizon est défini comme une zone dans laquelle une information devrait être reçue. Nous commençons par nous concentrer sur le partage de données au sein des voisins directs (à 1 saut de distance). Ensuite, nous proposons une solution pour construire une carte des voisins, centrée sur le nœud ego, dans un horizon à n sauts. Enfin, nous relâchons la définition de l’horizon pour la définir de façon dynamique, où différents éléments de données peuvent atteindre des distances différentes (sauts). En ce qui concerne la solution pour les horizons à 1 saut, notre technique adaptative prend en compte la dynamique des nœuds et la charge du réseau. Afin d’assurer une diffusion efficace des données dans différents scénarios, la fréquence d’envoi des messages est définie en fonction des mouvements des véhicules et d’une estimation du taux de perte du réseau. Après, nous nous concentrons sur la carte des voisins jusqu’à n sauts de distance. Comme la communication avec des nœuds éloignés apporte des problèmes supplémentaires (actions de transfert, retards plus importants, informations périmées), une évaluation de confiance des nœuds identifiés et une estimation de fiabilité du chemin vers chaque voisin sont ajoutées à la carte. Au lieu d’exécuter des processus de diffusion séparés, notre troisième contribution porte sur une stratégie de coopération dont l’objectif principal est de diffuser des données tout en satisfaisant la plupart des nœuds. À cette fin, une trame unique est transmise de nœud en nœud. Sa charge utile est mise à jour localement afin qu’elle contienne les éléments de données les plus pertinents en fonction de certains critères (par exemple, urgence, pertinence). Une telle stratégie définit ainsi un horizon centré sur les données. Nous validons nos propositions au moyen d’émulations de réseaux réalistes. De toutes nos études et des résultats obtenus, nous pouvons affirmer que notre approche apporte des perspectives intéressantes pour le partage de données dans des réseaux dynamiques comme les VANET. / In the context of Intelligent Transportation Systems - ITS, vehicles may have a lot of sensors (e.g. cameras, lidars, radars) and applications (collision avoidance, traffic monitoring, etc.) generating data. They represent then an important source of information. Local applications can significantly increase their effectiveness by sharing such an information within the network. Data accuracy, confidence and pertinence can be verified when receiving data from other nodes. Therefore, we believe that an important question to answer in this context is: “How to efficiently share data within such an environment?” Data sharing is a complex task in dynamic networks. Many concerns like intermittent connections, network density variation and communication spectrum congestion arise. A usual approach to handle these problems is based on periodic processes. Indeed, a message sent many times can reach its destination even with intermittent connections and low density networks. Nevertheless, within high density networks, they may lead to communication spectrum scarcity. In this thesis we address the problem of data sharing in dynamic networks by relying in so-called horizons of pertinence. A horizon is defined as an area within which an information is expected to be received. We start focusing on data sharing within direct neighbors (at 1-hop of distance). Then we propose a solution to construct a map of neighbors, centered in the ego-node, within a horizon of n-hops. Finally, we relax the horizon definition to a dynamic defined one where different data items may reach different distances (hops). Regarding the solution for 1-hop horizons, our adaptive technique takes into account nodes’ dynamics and network load. In order to ensure an effective data dissemination in different scenarios, the sending messages frequency is defined according to vehicles movements and an estimation of the network loss rate. Following, we focus on the map of neighbors up to n-hops of distance. As communicationwith distant nodes brings additional concerns (forwarding actions, larger delays, out-of-date information), a trust evaluation of identified nodes and a reliability estimation of the multi-hop path to each neighbor is added to the map. Instead of running separated disseminating processes, our third contribution deals with a cooperative strategy with the main goal of disseminating data while satisfying most of the nodes. For this purpose a unique frame is forwarded from node to node. Its payload is locally updated so that it contains the most relevant data items according to some criteria (e.g. urgency, relevance). Such a strategy defines thus a data-centered horizon. We validate our proposals by means of realistic network emulations. From all our studies and achieved results we can state that our approach brings interesting insights for data sharing in dynamic networks like VANETs.