Made available in DSpace on 2014-12-17T14:55:05Z (GMT). No. of bitstreams: 1 AndersonASS_TESE.pdf: 3250611 bytes, checksum: 4e87cd6efd2a74f4715e56d6e2aa0064 (MD5) Previous issue date: 2012-08-03 / This work introduces a new method for environment mapping with three-dimensional information from visual information for robotic accurate navigation. Many approaches of 3D mapping using occupancy grid typically requires high computacional effort to both build and store the map. We introduce an 2.5-D occupancy-elevation grid mapping, which is a discrete mapping approach, where each cell stores the occupancy probability, the height of the terrain at current place in the environment and the variance of this height. This 2.5-dimensional representation allows that a mobile robot to know whether a place in the environment is occupied by an obstacle and the height of this obstacle, thus, it can decide if is possible to traverse the obstacle. Sensorial informations necessary to construct the map is provided by a stereo vision system, which has been modeled with a robust probabilistic approach, considering the noise present in the stereo processing. The resulting maps favors the execution of tasks like decision making in the autonomous navigation, exploration, localization and path planning. Experiments carried out with a real mobile robots demonstrates that this proposed approach yields useful maps for robot autonomous navigation / Este trabalho apresenta um novo m?todo de mapeamento de ambientes com rob?s m?veis com informa??es tridimensionais para navega??o. Muitas abordagens de mapeamento 3D, usam o m?todo em grade de ocupa??o, o que resulta no uso de muito recurso computacional tanto na constru??o como no armazenamento desses mapas. A presente pesquisa apresenta o mapeamento 2,5-D em grade de ocupa??o-eleva??o, a qual ? definida como uma representa??o discreta, onde cada c?lula armazena uma probabilidade de ocupa??o, a altura do espa?o mapeado e a vari?ncia desse valor de altura. Essa representa??o permite que um rob? m?vel tenha a ci?ncia se um lugar do seu ambiente est? ocupado por um obst?culo e qual a altura desse obst?culo. Dessa forma, ele pode decidir se ? poss?vel navegar sobre o obst?culo ou n?o, de acordo com suas habilidades motoras. As informa??es sensoriais necess?rias para construir o mapa s?o providas por um sistema de vis?o est?reo, o qual foi modelado atrav?s de uma robusta an?lise estat?stica, considerando os ru?dos presentes no processamento est?reo. Os mapas resultantes favorecem a execu??o de tarefas como tomadas de decis?es na navega??o aut?noma, explora??o, localiza??o e planejamento de caminhos. Experimentos pr?ticos reais mostram que o m?todo de mapeamento apresentado ? ?til para a navega??o de rob?s aut?nomos

In the industrial area, robots are an important part of the technological resources available to perform manipulation tasks in manufacturing, assembly, the transportation of dangerous waste, and a variety of applications. Specialized systems of computer vision have entered the market to solve problems that other technologies have been unable to address. This document analyzes a stereo vision system that is used to provide the center of mass of an object in three dimensions. This kind of application is mounted using two or more cameras that are aligned along the same axis and give the possibility to measure the depth of a point in the space. The stereoscopic system described, measures the position of an object using a combination between the 2D recognition, which implies the calculus of the coordinates of the center of mass and using moments, and the disparity that is found comparing two images: one of the right and one of the left. This converts the system into a 3D reality viewfinder, emulating the human eyes, which are capable of distinguishing depth with good precision.The proposed stereo vision system is integrated into a 5 degree of freedom pneumatic robot, which can be programmed using the GRAFCET method by means of commercial software. The cameras are mounted in the lateral plane of the robot to ensure that all the pieces in the robot's work area can be observed.For the implementation, an algorithm is developed for recognition and position measurement using open sources in C++. This ensures that the system can remain as open as possible once it is integrated with the robot. The validation of the work is accomplished by taking samples of the objects to be manipulated and generating robot's trajectories to see if the object can be manipulated by its end effector or not. The results show that is possible to manipulate pieces in a visually crowded space with acceptable precision. However, the precision reached does not allow the robot to perform tasks that require higher accuracy as the one is needed in manufacturing assembly process of little pieces or in welding applications. / En el área industrial los robots forman parte importante del recurso tecnológico disponible para tareas de manipulación en manufactura, ensamble, manejo de residuos peligrosos y aplicaciones varias. Los sistemas de visión computacional se han ingresado al mercado como soluciones a problemas que otros tipos de sensores y métodos no han podido solucionar. El presente trabajo analiza un sistema de visión estereoscópico aplicado a un robot. Este arreglo permite la medición de coordenadas del centro de un objeto en las tres dimensiones, de modo que, le da al robot la posibilidad de trabajar en el espacio y no solo en un plano. El sistema estereoscópico consiste en el uso de dos o más cámaras alineadas en alguno de sus ejes, mediante las cuales, es posible calcular la profundidad a la que se encuentran los objetos. En el presente, se mide la posición de un objeto haciendo una combinación entre el reconocimiento 2D y la medición de las coordenadas y de su centro calculadas usando momentos. En el sistema estereoscópico, se añade la medición de la última coordenada mediante el cálculo de la disparidad encontrada entre las imágenes de las cámaras inalámbricas izquierda y derecha, que convierte al sistema en un visor 3D de la realidad, emulando los ojos humanos capaces de distinguir profundidades con cierta precisión. El sistema de visión computacional propuesto es integrado a un robot neumático de 5 grados de libertad el cual puede ser programado desde la metodología GRAFCET mediante software de uso comercial. Las cámaras del sistema de visión están montadas en el plano lateral del robot de modo tal, que es posible visualizar las piezas que quedan dentro de su volumen de trabajo. En la implementación, se desarrolla un algoritmo de reconocimiento y medición de posición, haciendo uso de software libre en lenguaje C++. De modo que, en la integración con el robot, el sistema pueda ser lo más abierto posible. La validación del trabajo se logra tomando muestras de los objetos a ser manipulados y generando trayectorias para el robot, a fin de visualizar si la pieza pudo ser captada por su garra neumática o no. Los resultados muestran que es posible lograr la manipulación de piezas en un ambiente visualmente cargado y con una precisión aceptable. Sin embargo, se observa que la precisión no permite que el sistema pueda ser usado en aplicaciones donde se requiere precisión al nivel de los procesos de ensamblado de piezas pequeñas o de soldadura.

Submitted by Liliane Ferreira (ljuvencia30@gmail.com) on 2018-11-26T13:24:36Z No. of bitstreams: 2 Dissertação - Gabriel da Silva Vieira - 2018.pdf: 13740412 bytes, checksum: ddb7d4353e4f2d7650b087dd0d4bd796 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Approved for entry into archive by Luciana Ferreira (lucgeral@gmail.com) on 2018-11-26T13:43:18Z (GMT) No. of bitstreams: 2 Dissertação - Gabriel da Silva Vieira - 2018.pdf: 13740412 bytes, checksum: ddb7d4353e4f2d7650b087dd0d4bd796 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) / Made available in DSpace on 2018-11-26T13:43:18Z (GMT). No. of bitstreams: 2 Dissertação - Gabriel da Silva Vieira - 2018.pdf: 13740412 bytes, checksum: ddb7d4353e4f2d7650b087dd0d4bd796 (MD5) license_rdf: 0 bytes, checksum: d41d8cd98f00b204e9800998ecf8427e (MD5) Previous issue date: 2018-10-05 / Outro / Disparity maps are key components of a stereo vision system. Autonomous navigation, 3D reconstruction, and mobility are examples of areas of research which use disparity maps as an important element. Although a lot of work has been done in the stereo vision field, it is not easy to build stereo systems with concepts such as reuse and extensible scope. In this study, we explore this gap and it presents a software architecture that can accommodate different stereo methods through a standard structure. Firstly, it introduces some scenarios that illustrate use cases of disparity maps and it shows a novel architecture that foments code reuse. A Disparity Computation Framework (DCF) is presented and we discuss how its components are structured. Then we developed a prototype which closely follows the proposal architecture and we prepared some test cases to be performed. Furthermore, we have implemented disparity methods for validation purposes and to evaluate our disparity refinement method. This refinement method, named as Segmented Consistency Check (SCC), was designed to increase the robustness of stereo matching algorithms. It consists of a segmentation process, statistical analysis of grouping areas and a support weighted function to find and to fill in unknown disparities. The experimental results show that the DCF can satisfy different scenarios on-demand. Besides, they show that SCC method is an efficient approach that can make some enhancements in disparity maps, as reducing the disparity error measure. / Mapas de disparidade são elementos cruciais em sistemas de visão estéreo. Navegação autônoma, reconstrução 3D e mobilidade são exemplos de área de pesquisa que utilizam mapas de disparidade como elementos-chave. Embora muitos trabalhos têm sido feitos na área de visão estéreo, ainda assim, não é trivial construir sistemas estéreos com aplicação de conceitos como reutilização e escopo extensível. Neste estudo, exploramos essa lacuna e apresentamos uma arquitetura de software capaz de acomodar diferentes métodos de visão estéreo através de uma estrutura bem definida. Inicialmente, cenários que ilustram usos de mapa de disparidade são introduzidos e uma arquitetura que fomenta reutilização de código é apresentada. Dessa forma, um Framework de Cálculo de Disparidade (FCD) é apresentado e seus componentes são discutidos a fim de especificar a sua estrutura. Em seguida, um protótipo que segue a arquitetura proposta é apresentado e alguns casos de teste são preparados e executados. Além disso, métodos de cálculo de disparidade foram implementados para propostas de validação e para avaliar o método de refinamento de disparidade proposto pelos autores. Esse método de refinamento, chamado de Checagem de Consistência de Segmento (CCS), foi projetado para aumentar a robustez de algoritmos de combinação estéreo. Trata-se de um método que utiliza um processo de segmentação preliminar, análise estatística de áreas definidas e função ponderada de suporte para encontrar e preencher disparidades marcadas como desconhecidas. Os resultados dos experimentos realizados apontam que o FCD pode satisfazer diferentes cenários sob demanda. Além disso, os resultados mostram que o método CCS é uma abordagem eficiente que pode trazer certos melhoramentos em mapas de disparidade, como reduzir a medida de erro no cálculo de correspondências estéreo.

Object tracking has become really important in the field of robotics in the last years. Frequently, the goal is to obtain the trajectory of the tracked target over time and space by acquiring and processing information from the sensors. In this thesis we are interested in tracking objects at a very short range. The primary application of our approach is targeting the domain of object tracking during grasp execution with a hand-in-eye sensor setup. To this end, a promising approach investigated in this work is based on the Leap Motion sensor, which is designed for tracking human hands. However, we are interested in tracking grasped objects thus we need to extend its functionality. The main goal of the thesis is to track the 3D position and orientation of an object from a set of simple primitives (cubes, cylinders, triangles) over a video sequence. That is the reason we have designed and developed two different approaches for tracking objects with the Leap Motion device as stereo vision system.

Stereo vision is one of the most popular elements in the field of mobile robots and significantly contributes to their autonomous behaviour. The aim of the diploma thesis was to design and implement a camera module as a hardware sensor input, which is independent, with the possibility of supplementing the system with other cameras, and to create a depth map from a pair of cameras. The diploma thesis consists of theoretical and practical part, including the conclusion of results. The theoretical part introduces the ROS framework, discusses methods of creating depth maps, and provides an overview of the most popular stereo cameras in robotics. The practical part describes in detail the preparation of the experiment and its implementation. It also describes the camera calibration and the depth map creating. The last chapter contains an evaluation of the experiment.

This thesis deals with a concept of system for automatic passenger counting in different  modes of transport. Counting units are placed in top of the door area in the vehicle. Passengers are detected at the disparity map counted from the stereo-camera images. Object tracking is achieved with Global nearest neighbor and Multiple hypothesis tracking algorithm. This system is used for public transportation surveys.

Dans le cadre de systèmes d’aide à la conduite, nous avons contribué aux approches de stéréovision pour l’extraction de contour, la mise en correspondance des images stéréoscopiques et la détection de véhicules. L’extraction de contour réalisée est basée sur le concept declivity line que nous avons proposé. La declivity line est construite en liant des déclivités selon leur position relative et similarité d’intensité. L’extraction de contour est obtenue en filtrant les declivity lines construites basées sur leurs caractéristiques. Les résultats expérimentaux montrent que la declivity lines méthode extrait plus de l’informations utiles comparées à l’opérateur déclivité qui les a filtrées. Des points de contour sont ensuite mis en correspondance en utilisant la programmation dynamique et les caractéristiques de declivity lines pour réduire le nombre de faux appariements. Dans notre méthode de mise en correspondance, la declivity lines contribue à la reconstruction détaillée de la scène 3D. Finalement, la caractéristique symétrie des véhicules sont exploitées comme critère pour la détection de véhicule. Pour ce faire, nous étendons le concept de carte de symétrie monoculaire à la stéréovision. En conséquence, en effectuant la détection de véhicule sur la carte de disparité, une carte de symétrie (axe; largeur; disparity) est construite au lieu d’une carte de symétrie (axe; largeur). Dans notre concept, des obstacles sont examinés à différentes profondeurs pour éviter la perturbation de la scène complexe dont le concept monoculaire souffre. / In the framework of driving assistance systems, we contributed to stereo vision approaches for edge extraction, matching of stereoscopic pair of images and vehicles detection. Edge extraction is performed based on the concept of declivity line we introduced. Declivity line is constructed by connecting declivities according to their relative position and intensity similarity. Edge extraction is obtained by filtering constructed declivity lines based on their characteristics. Experimental results show that declivity line method extracts additional useful information compared to declivity operator which filtered them out. Edge points of declivity lines are then matched using dynamic programming, and characteristics of declivity line reduce the number of false matching. In our matching method, declivity line contributes to detailed reconstruction of 3D scene. Finally, symmetrical characteristic of vehicles are exploited as a criterion for their detection. To do so, we extend the monocular concept of symmetry map to stereo concept. Consequently, by performing vehicle detection on disparity map, a (axis; width; disparity) symmetry map is constructed instead of an (axis; width) symmetry map. In our stereo concept, obstacles are examined at different depths thus avoiding disturbance of complex scene from which monocular concept suffers.

De nos jours, les réseaux de capteurs multimédia sans fils sont prometteurs pour différentes applications et domaines, en particulier avec le développement de l’IoT et des capteurs de caméra efficaces et bon marché. La stéréo vision est également très importante pour des objectifs multiples comme la Cinématographie, les jeux, la Réalité Virtuelle, la Réalité Augmentée, etc. Cette thèse vise à développer un système de reconstruction de scène en 3D prouvant l’utilisation de cartes de disparités stéréoscopiques multi-angles dans le contexte des réseaux de capteurs multimedia. Notre travail peut être divisé en trois parties. La première se concentre sur l’étude de toutes les applications, composants, topologies, contraintes et limitations de ces réseaux. En plus, les méthodes de calcul de disparité de vision stéréoscopique afin de choisir la ou les meilleures méthodes pour réaliser une reconstruction en 3D sur le réseau à faible coût en termes de complexité et de consommation d’énergie. Dans la deuxième partie, nous expérimentons et simulons différents calculs de cartes de disparités sur quelques nœuds en changeant les scénarios (intérieur et extérieur), les distances de couverture, les angles, le nombre de nœuds et les algorithmes. Dans la troisième partie, nous proposons un modèle de réseau basé sur l’arbre pour calculer des cartes de disparités précises sur des nœuds de capteurs de caméra multicouches qui répond aux besoins du serveur pour faire une reconstruction de scène 3D de la scène ou de l’objet d’intérêt. Les résultats sont acceptables et assurent la preuve du concept d’utilisation des cartes de disparités dans le contexte des réseaux de capteurs multimédia. / Nowadays, the WMSNs are promising for different applications and fields, specially with the development of the IoT and cheap efficient camera sensors. The stereo vision is also very important for multiple purposes like Cinematography, games, Virtual Reality, Augmented Reality, etc. This thesis aim to develop a 3D scene reconstruction system that proves the concept of using multiple view stereo disparity maps in the context of WMSNs. Our work can be divided in three parts. The first one concentrates on studying all WMSNs applications, components, topologies, constraints and limitations. Adding to this stereo vision disparity map calculations methods in order to choose the best method(s) to make a 3d reconstruction on WMSNs with low cost in terms of complexity and power consumption. In the second part, we experiment and simulate different disparity map calculations on a couple of nodes by changing scenarios (indoor and outdoor), coverage distances, angles, number of nodes and algorithms. In the third part, we propose a tree-based network model to compute accurate disparity maps on multi-layer camera sensor nodes that meets the server needs to make a 3d scene reconstruction of the scene or object of interest. The results are acceptable and ensure the proof of the concept to use disparity maps in the context of WMSNs.

Three-dimensional information is often used in robotics and 3D-mapping. There exist several ways to obtain a three-dimensional map. However, the time of flight used in the laser scanners or the structured light utilized by Kinect-like sensors sometimes are not sufficient. In this thesis, we investigate two CNN based stereo matching methods for obtaining 3D-information from a grayscaled pair of rectified images.While the state-of-the-art stereo matching method utilize a Siamese architecture, in this project a two-channel and a two stream network are trained in an attempt to outperform the state-of-the-art. A set of experiments were performed to achieve optimal hyperparameters. By changing one parameter at the time, the networks with architectures mentioned above are trained. After a completed training the networks are evaluated with two criteria, the error rate, and the runtime.Due to time limitations, we were not able to find optimal learning parameters. However, by using settings from [17] we train a two-channel network that performed almost on the same level as the state-of-the-art. The error rate on the test data for our best architecture is 2.64% while the error rate for the state-of-the-art Siamese network is 2.62%. We were not able to achieve better performance than the state-of-the-art, but we believe that it is possible to reduce the error rate further. On the other hand, the state-of-the-art Siamese stereo matching network is more efficient and faster during the disparity estimation. Therefore, if the time efficiency is prioritized, the Siamese based network should be considered.

The demand for robots to work in environments together with humans is growing. This calls for new requirements on robots systems, such as the need to be perceived as responsive and accurate in human interactions. This thesis explores the possibility of using AI methods to predict the movement of a human and evaluating if that information can assist a robot with human interactions. The AI methods that were used is a Long Short Term Memory(LSTM) network and an artificial neural network(ANN). Both networks were trained on data from a motion capture dataset and on four different prediction times: 1/2, 1/4, 1/8 and a 1/16 second. The evaluation was performed directly on the dataset to determine the prediction error. The neural networks were also evaluated on a robotic arm in a simulated environment, to show if the prediction methods would be suitable for a real-life system. Both methods show promising results when comparing the prediction error. From the simulated system, it could be concluded that with the LSTM prediction the robotic arm would generally precede the actual position. The results indicate that the methods described in this thesis report could be used as a stepping stone for a human-robot interactive system.