This thesis investigates the use of visual-motion information sampled through optical flow for the task of indoor obstacle avoidance on autonomous mobile robots. The methods focus on the practical use of optical flow and visual motion information in performing the obstacle avoidance task in real indoor environments. The methods serve to identify visual-motion properties that must be used in synergy with visual-spatial properties toward the goal of a complete robust visual-only obstacle avoidance system, as is evidently seen within nature. A review of vision-based obstacle avoidance techniques shows that early research mainly focused on visual-spatial techniques, which heavily rely on various assumptions of their environments to function successfully. On the other hand, more current research that looks toward the use of visual-motion information (sampled through optical flow) tends to focus on using optical flow in a subsidiary manner, and does not completely take advantage of the information encoded within an optical flow field. In the light of the current research limitations, this thesis describes two different approaches and evaluates their use of optical flow to perform the obstacle avoidance task. The first approach begins with the construction of a conventional range map using optical flow that stems from the structure-from-motion domain and the theory that optical flow encodes 3D environmental information under certain conditions. The second approach investigates optical flow in a causal mechanistic manner using machine learning of motor responses directly from optical flow - motivated from physical and behavioural evidence observed in biological creatures. Specifically, the second approach is designed with three main objectives in mind: 1) to investigate whether optical flow can be learnt for obstacle avoidance; 2) to create a system capable of repeatable obstacle avoidance performance in real-life environments; and 3) to analyse the system to determine what optical flow properties are actually being used for the motor control task. The range-map reconstruction results have demonstrated some good distance estimations through the use of a feature-based optical flow algorithm. However, the number of flow points were too sparse to provide adequate obstacle detection. Results froma differential-based optical flow algorithm helped to increase the density of flow points, but highlighted the high sensitivity of the optical flow field to the rotational errors and outliers that plague the majority of frames under real-life robot situations. Final results demonstrated that current optical flow algorithms are ill-suited to estimate obstacle distances consistently, as range-estimation techniques require an extremely accurate optical flow field with adequate density and coverage for success. This is a difficult problem within the optical flow estimation domain itself. In the machine learning approach, an initial study to examine whether optical flow can be machine learnt for obstacle avoidance and control in a simple environment was successful. However,there were certain problems. Several critical issues which arise with the use of a machine learning approach were highlighted. These included sample set completeness, sample set biases, and control system instability. Consequently, an extended neural network was proposed that had several improvements made to overcome the initial problems. Designing an automated system for gathering training data helped to eliminate most of the sample set problems. Key changes in the neural network architecture, optical flow filters, and navigation technique vastly improved the control system stability. As a result, the extended neural network system was able to successfully perform multiple obstacle avoidance loops in both familiar and unfamiliar real-life environments without collisions. The lap times of the machine learning approach were comparable to those of the laser-based navigation technique. The the machine learning approach was 13% slower in the familiar and 25% slower in the unfamiliar environment. Furthermore, through analysis of the neural network approach, flow magnitudes were revealed to be learnt for range information in an absolute manner, while flow directions were used to detect the focus of expansion (FOE) in order to predict critical collision situations and improve control stability. In addition, the precision of the flow fields was highlighted as an important requirement, as opposed to the high accuracy of flow vectors. For robot control purposes, image-processing techniques such as region finding and object boundary detection were employed to detect changes between optical flow vectors in the image space.

This thesis investigates the use of visual-motion information sampled through optical flow for the task of indoor obstacle avoidance on autonomous mobile robots. The methods focus on the practical use of optical flow and visual motion information in performing the obstacle avoidance task in real indoor environments. The methods serve to identify visual-motion properties that must be used in synergy with visual-spatial properties toward the goal of a complete robust visual-only obstacle avoidance system, as is evidently seen within nature. A review of vision-based obstacle avoidance techniques shows that early research mainly focused on visual-spatial techniques, which heavily rely on various assumptions of their environments to function successfully. On the other hand, more current research that looks toward the use of visual-motion information (sampled through optical flow) tends to focus on using optical flow in a subsidiary manner, and does not completely take advantage of the information encoded within an optical flow field. In the light of the current research limitations, this thesis describes two different approaches and evaluates their use of optical flow to perform the obstacle avoidance task. The first approach begins with the construction of a conventional range map using optical flow that stems from the structure-from-motion domain and the theory that optical flow encodes 3D environmental information under certain conditions. The second approach investigates optical flow in a causal mechanistic manner using machine learning of motor responses directly from optical flow - motivated from physical and behavioural evidence observed in biological creatures. Specifically, the second approach is designed with three main objectives in mind: 1) to investigate whether optical flow can be learnt for obstacle avoidance; 2) to create a system capable of repeatable obstacle avoidance performance in real-life environments; and 3) to analyse the system to determine what optical flow properties are actually being used for the motor control task. The range-map reconstruction results have demonstrated some good distance estimations through the use of a feature-based optical flow algorithm. However, the number of flow points were too sparse to provide adequate obstacle detection. Results froma differential-based optical flow algorithm helped to increase the density of flow points, but highlighted the high sensitivity of the optical flow field to the rotational errors and outliers that plague the majority of frames under real-life robot situations. Final results demonstrated that current optical flow algorithms are ill-suited to estimate obstacle distances consistently, as range-estimation techniques require an extremely accurate optical flow field with adequate density and coverage for success. This is a difficult problem within the optical flow estimation domain itself. In the machine learning approach, an initial study to examine whether optical flow can be machine learnt for obstacle avoidance and control in a simple environment was successful. However,there were certain problems. Several critical issues which arise with the use of a machine learning approach were highlighted. These included sample set completeness, sample set biases, and control system instability. Consequently, an extended neural network was proposed that had several improvements made to overcome the initial problems. Designing an automated system for gathering training data helped to eliminate most of the sample set problems. Key changes in the neural network architecture, optical flow filters, and navigation technique vastly improved the control system stability. As a result, the extended neural network system was able to successfully perform multiple obstacle avoidance loops in both familiar and unfamiliar real-life environments without collisions. The lap times of the machine learning approach were comparable to those of the laser-based navigation technique. The the machine learning approach was 13% slower in the familiar and 25% slower in the unfamiliar environment. Furthermore, through analysis of the neural network approach, flow magnitudes were revealed to be learnt for range information in an absolute manner, while flow directions were used to detect the focus of expansion (FOE) in order to predict critical collision situations and improve control stability. In addition, the precision of the flow fields was highlighted as an important requirement, as opposed to the high accuracy of flow vectors. For robot control purposes, image-processing techniques such as region finding and object boundary detection were employed to detect changes between optical flow vectors in the image space.

This thesis investigates the questions of where to grasp and how to grasp a given object with an articulated robotic grasping device. To this end, aspects of grasp synthesis and hand motion planning and control are investigated. Grasp synthesis is the process of determining a palm pose with respect to the target object, as well as a hand joint configuration and/or grasp contact points such that a successful grasp execution is allowed. Existing methods tackling the grasp synthesis problem can be categorized in analytical and empirical approaches. Analytical approaches are based on geometric, kinematic and/or dynamic formulations, whereas empirical methods aim at mimicking human strategies.An overarching idea throughout this thesis is to circumvent the curse of dimensionality, which is inherent in high-dimensional planning problems, by incorporating empirical data in analytical approaches. To this end, tools from the field of constrained optimization are used (i) to synthesize grasp families based on available prototype grasps, (ii) to incorporate heuristics capturing human grasp strategies in the grasp synthesis process and (iii) to encode demonstrated grasp motions in primitive motion controllers.The first contribution is related to the computation and analysis of grasp families which are represented as Independent Contact Regions (ICR) on a target object’s surface. To this end, the well-known concept of the Grasp Wrench Space for a single grasp is extended to be applicable for a set of grasps. Applications of ICR include grasp qualification by capturing the robustness of a grasp to position inaccuracies and the visual guidance of a demonstrator in a teleoperating scenario. In the second main contribution of this thesis, it is shown how to reduce the grasp solution space during the synthesis process by accounting for human approach strategies. This is achieved by imposing appropriate constraints to a corresponding optimization problem. A third contribution in this dissertation is made to reactive motion planning. Here, primitive controllers are synthesized by estimating the free parameters of corresponding dynamical systems from multiple demonstrated trajectories. The approach is evaluated on an anthropomorphic robot hand/arm platform. Also, an extension to a Model Predictive Control (MPC) scheme is presented which allows to incorporate state constraints for auxiliary tasks such as obstacle avoidance.

Dans cette thèse nous avons contribué à la commande floue de deux types de robots mobiles : deux robots de type unicycle (Khepera II et fauteuil roulant). Ensuite, nous avons utilisé une architecture de pilotage réactive permettant d’intégrer la commande floue ainsi qu’un algorithme d’évitement d’obstacles réactif utilisant la théorie de Zones de Déformation Virtuelles (ZDV). Des résultats de simulation et expérimentales ont permis de valider l’approche développée. / In this thesis we contributed on developing a fuzzy control of two types of mobile robots : two unicycle robots (Khepera II and wheelchair). Then, we used a reactive pilotingarchitecture insuring the integration of both the fuzzy controller and an obstacle avoidance algorithm using the deformable virtual zones theory (DVZ). Simulation and experimental results validate the developed approach.

abstract: Toward the ambitious long-term goal of a fleet of cooperating Flexible Autonomous Machines operating in an uncertain Environment (FAME), this thesis addresses several critical modeling, design and control objectives for ground vehicles. One central objective was to show how off-the-shelf (low-cost) remote-control (RC) “toy” vehicles can be converted into intelligent multi-capability robotic-platforms for conducting FAME research. This is shown for two vehicle classes: (1) six differential-drive (DD) RC vehicles called Thunder Tumbler (DDTT) and (2) one rear-wheel drive (RWD) RC car called Ford F-150 (1:14 scale). Each DDTT-vehicle was augmented to provide a substantive suite of capabilities as summarized below (It should be noted, however, that only one DDTT-vehicle was augmented with an inertial measurement unit (IMU) and 2.4 GHz RC capability): (1) magnetic wheel-encoders/IMU for(dead-reckoning-based) inner-loop speed-control and outer-loop position-directional-control, (2) Arduino Uno microcontroller-board for encoder-based inner-loop speed-control and encoder-IMU-ultrasound-based outer-loop cruise-position-directional-separation-control, (3) Arduino motor-shield for inner-loop motor-speed-control, (4)Raspberry Pi II computer-board for demanding outer-loop vision-based cruise- position-directional-control, (5) Raspberry Pi 5MP camera for outer-loop cruise-position-directional-control (exploiting WiFi to send video back to laptop), (6) forward-pointing ultrasonic distance/rangefinder sensor for outer-loop separation-control, and (7) 2.4 GHz spread-spectrum RC capability to replace original 27/49 MHz RC. Each “enhanced”/ augmented DDTT-vehicle costs less than 􀀀175 but offers the capability of commercially available vehicles costing over 􀀀500. Both the Arduino and Raspberry are low-cost, well-supported (software wise) and easy-to-use. For the vehicle classes considered (i.e. DD, RWD), both kinematic and dynamical (planar xy) models are examined. Suitable nonlinear/linear-models are used to develop inner/outer-loopcontrol laws. All demonstrations presented involve enhanced DDTT-vehicles; one the F-150; one a quadrotor. The following summarizes key hardware demonstrations: (1) cruise-control along line, (2) position-control along line (3) position-control along curve (4) planar (xy) Cartesian stabilization, (5) cruise-control along jagged line/curve, (6) vehicle-target spacing-control, (7) multi-robot spacing-control along line/curve, (8) tracking slowly-moving remote-controlled quadrotor, (9) avoiding obstacle while moving toward target, (10) RC F-150 followed by DDTT-vehicle. Hardware data/video is compared with, and corroborated by, model-based simulations. In short, many capabilities that are critical for reaching the longer-term FAME goal are demonstrated. / Dissertation/Thesis / Masters Thesis Electrical Engineering 2015

Este trabalho apresenta um novo método de detecção de obstáculos usados para navegação de um barco autônomo. O método desenvolvido é baseado em visão computacional e permite medir a distância e direção do obstáculo à câmera de video. A distância do possível obstáculo à câmera de vídeo, e o vetor de contorno predominante da imagem são os parâmetros usados para identificar os obstáculos. Imagens estereoscópicas adquiridas nas margens da lago do clube Náutico de Araraquara, usando bóias de navegação como obstáculos, foram usadas para extrair as características significantes das experiências. Para validar a experiência, foram utilizadas imagens do Reservatório do Broa (Itirapina, SP). A proposta desenvolvida mostrou ser mais eficiente que o método tradicional usando a teoria de Campos Potenciais. As imagens foram propositadamente tomadas contra o sol, onde o brilho das ondas são erroneamente indicadas como obstáculos pelo método de campos potenciais. Esta proposta filtra as ondas de forma a diminuir sua interferência no diagnóstico. / This work presents the results of new obstacle detection methods used for an autonomous boat navigation. The developed method is based on computer vision and allows to measure the distance and direction of the obstacle to the boat. The distance of the possible obstacle to the camera, and the obstacle outline predominant vector are the parameters used to identify the obstacles. Stereo images acquired from the margins of the Nautical Araraquara lake, using navigation buoys as obstacles, were used to extract the meaningful characteristics of the experiments. To validate the experiment, images from the Broa Reservoir (Itirapina, SP) where used. The developed proposal showed to be more efficient than the traditional method using the potential fields theory. The images were taken willfully against the sun, where the brightness of the waves are erroneously identified as obstacles by the method of potential fields. This method filters the waves so as to reduce its interference in the diagnosis.

UAVs employed for low altitude jobs are more liable to collide with the urban structures on their way to the goal point. In this thesis, the problem of reactive obstacle avoidance is addressed by an innovative partial integrated guidance and control (PIGC) approach using the Six-DOF model of real UAV unlike the kinematic models used in the existing literatures. The guidance algorithm is designed which uses the collision cone approach to predict any possible collision with the obstacle and computes an alternate aiming direction for the vehicle. The aiming direction of the vehicle is the line of sight line tangent to the safety ball surrounding the obstacle. The point where the tangent touches the safety ball is the aiming point. Once the aiming point is known, the obstacle is avoided by directing the vehicle (on the principles of pursuit guidance) along the tangent to the safety ball. First, the guidance algorithm is applied successfully to the point mass model of UAV to verify the proposed collision avoidance concept. Next, PIGC approach is proposed for reactive obstacle avoidance of UAVs. The reactive nature of the avoidance problem within the available time window demands simultaneous reaction from the guidance and control loop structures of the system i.e, in the IGC framework (executes in single loop). However, such quick maneuvers cause the faster dynamics of the system to go unstable due to inherent separation between the faster and slower dynamics. On the contrary, in the conventional design (executes in three loops), the settling time of the response of different loops will not be able to match with the stringent time-to-go window for obstacle avoidance. This causes delay in tracking in all the loops which will affect the system performance adversely and hence UAV will fail to avoid the obstacle. However, in the PIGC framework, it overcomes the disadvantage of both the IGC design and the conventional design, by introducing one more loop compared to the IGC approach and reducing a loop compared to the conventional approach, hence named as Partial IGC. Nonlinear dynamic inversion technique based PIGC approach utilizes the faster and slower dynamics of the full nonlinear Six-DOF model of UAV and executes the avoidance maneuver in two loops. In the outer loop, the vehicle guidance strategy attempts to reorient the velocity vector of the vehicle along the aiming point within a fraction of the available time-to-go. The orientation of the velocity vector is achieved by enforcing the angular correction in the horizontal and vertical flight path angles and enforcing turn coordination. The outer loop generates the body angular rates which are tracked as the commanded signal in the inner loop. The enforcement of the desired body rates generates the necessary control surface deflections required to stir the UAV. Control surface deflections are realized by the vehicle through the first order actuator dynamics. A controller for the first order actuator model is also proposed in order to reduce the actuator delay. Every loop of the PIGC technique uses nonlinear dynamic inversion technique which has critical issues like sensitiveness to the modeling inaccuracies of the plant model. To make it robust against the parameter inaccuracies of the system, it is reinforced with the neuro-adaptive design in the inner loop of the PIGC design. In the NA design, weight update rule based on Lyapunov’s theory provides online training of the weights. To enhance fast and stable training of the weights, preflight maneuvers are proposed. Preflight maneuvers provide stabilized pre-trained weights which prevents any misapprehensions in the obstacle avoidance scenario. Simulation studies have been carried out with the point mass model and with the Six-DOF model of the real fixed wing UAV in the PIGC framework to test the performance of the nonlinear reactive guidance scheme. Various simulations have been executed with different number and size of the obstacles. NA augmented PIGC design is validated with different levels of uncertainties in the plant model. A comparative study in NA augmented PIGC design was performed between the pre-trained weights and zero weights as used for weight initialization in online training. Various comparative study shows that the NA augmented PIGC design is quite effective in avoiding collisions in different scenarios. Since the NDI technique involved in the PIGC design gives a closed loop solution and does not operate with iterative steps, therefore the reactive obstacle avoidance is achieved in a computationally efficient manner.

Ce manuscrit traite de la conception d’un robot quadrirotor bio-inspiré. Ce robot, nommé QuadMorphing, s’inspire de l’oiseau et possède la capacité de se replier en vol afin de diminuer son envergure. Cette particularité est intéressante pour des problématiques d’évitement d’obstacles dans des milieux encombrés.Le travail présenté ici contient une présentation du robot où la plateforme mécatronique y est décrite en détails. Puis, des résultats expérimentaux sont présentés et commentés afin de quantifier les performances du prototype QuadMorphing durant des scénarios de franchissement d’obstacles.La deuxième partie de cette thèse traite de l’estimation de la taille d’obstacles en vol grâce à une perception visuelle monoculaire. Deux algorithmes d’estimation ont été simulés afin d’être validés pour être ensuite mis en place sur une nouvelle version du robot qui a été testée expérimentalement. Ces estimations permettent par la suite de rendre le robot plus autonome pour éviter les collisions avec son environnement et actionner son système de changement de forme si cela est nécessaire. / This paper describes a bio-inspired quadrotor design. This robot, called QuadMorphing, is inspired by birds and has the ability to fold its mechanical structure to reduce its wingspan during the flight. This feature could be useful for obstacle avoidance task in cluttered environments.The work presented here contains a full description of the mechatronic structure. Then, experimental results are presented and discussed in order to quantify the QuadMorphing performances during obstacle avoidance scenarios.The second part of this thesis deals with estimating obstacle size during flight using monocular visual perception. Two estimation algorithms were simulated in order to be validated and then implemented for experimental testing on a new version of the robot. In order to make the robot autonomous, the estimation of the size of the obstacle allows the robot to avoid collisions with its environment and to perform its morphological reduction if necessary.

No description available.

Obstacle detection is one of the major focus area on image processing. For mobile robots, obstacle detection and collision avoidance is a notorious problem and is still a part of the modern research. There are already a lot of research have been done so far for obstacle detection and collision avoidance. This thesis evaluates the existing various well-known methods and sensors for collision free navigation of the mobile robot. For moving obstacle detection purpose the frame difference approach is adopted. Robotino® is used as the mobile robot platform and additionally Microsoft Kinect is used as 3D sensor. For getting information from the environment about obstacle, the 9-built-in distance sensor of Robotino® and 3D depth image data from the Kinect is used. The combination is done to get the maximum advantages for obstacles information. The detection of moving object in front of the sensor is a major interest of this work.