A technique for real time object tracking in a mobile computing environment and its application to video see-through Augmented Reality (AR) has been designed, verified through simulation, and implemented and validated on a mobile computing device. Using position based visual position and orientation (POSE) methods and the Extended Kalman Filter (EKF), it is shown how this technique lends itself to be flexible to tracking multiple objects and multiple object models using a single monocular camera on different mobile computing devices. Using the monocular camera of the mobile computing device, feature points of the object(s) are located through image processing on the display. The relative position and orientation between the device and the object(s) is determined recursively by an EKF process. Once the relative position and orientation is determined for each object, three dimensional AR image(s) are rendered onto the display as if the device is looking at the virtual object(s) in the real world. This application and the framework presented could be used in the future to overlay additional informational onto displays in mobile computing devices. Example applications include robotic aided surgery where animations could be overlaid to assist the surgeon, in training applications that could aid in operation of equipment or in search and rescue operations where critical information such as floor plans and directions could be virtually placed onto the display. Current approaches in the field of real time object tracking are discussed along with the methods used for video see-through AR applications on mobile computing devices. The mathematical framework for the real time object tracking and video see-through AR rendering is discussed in detail along with some consideration to extension to the handling of multiple AR objects. A physical implementation for a mobile computing device is proposed detailing the algorithmic approach along with design decisions. The real time object tracking and video see-through AR system proposed is verified through simulation and details around the accuracy, robustness, constraints, and an extension to multiple object tracking are presented. The system is then validated using a ground truth measurement system and the accuracy, robustness, and its limitations are reviewed. A detailed validation analysis is also presented showing the feasibility of extending this approach to multiple objects. Finally conclusions from this research are presented based on the findings of this work and further areas of study are proposed.

Šiame darbe tiriami algoritminiai sprendimai mobiliam robotui, leidžiantys aptikti ieškomą objektą bei įvertinti jo poziciją ir orientaciją erdvėje. Atlikus šios srities technologijų analizę surasta įvairių realizacijai tinkamų metodų, tačiau bendro jų efektyvumo palyginimo trūko. Siekiant užpildyti šią spragą realizuota programinė ir techninė įranga, kuria atliktas labiausiai roboto sistemoms tinkamų metodų vertinimas. Algoritmų analizė susideda iš algoritmų tikslumo ir jų veikimo spartos vertinimo panaudojant tam paprastus bei efektyvius metodus. Darbe analizuojamas objektų orientacijos nustatymas iš Kinect kameros gylio duomenų pasitelkiant ICP algoritmą. Atliktas dviejų gylio sistemų spartos ir tikslumo tyrimas parodė, jog Kinect kamera spartos atžvilgiu yra efektyvesnis bei 2-5 kartus tikslesnis sprendimas nei įprastinė stereo kamerų sistema. Objektų aptikimo algoritmų efektyvumo eksperimentuose nustatytas maksimalus aptikimo tikslumas apie 90% bei pasiekta maksimali 15 kadrų/s veikimo sparta analizuojant standartinius VGA 640x480 raiškos vaizdus. Atliktas objektų pozicijos ir orientacijos nustatymo ICP metodo efektyvumo tyrimas parodė, jog vidutinė absoliutinė pozicijos ir orientacijos nustatymo paklaida yra atitinkamai apie 3.4cm bei apie 30 laipsnių, o veikimo sparta apie 2 kadrai/s. Tolesnis optimizavimas arba duomenų kiekio minimizavimas yra būtinas norint pasiekti geresnius veikimo rezultatus mobilioje ribotų resursų roboto sistemoje. Darbe taip pat buvo sėkmingai... [toliau žr. visą tekstą] / This work presents a performance analysis of the state-of-the-art computer vision algorithms for object detection and pose estimation. Initial field study showed that many algorithms for the given problem exist but still their combined comparison was lacking. In order to fill in the existing gap a software and hardware solution was created and the comparison of the most suitable methods for a robot system were done. The analysis consists of detector accuracy and runtime performance evaluation using simple and robust techniques. Object pose estimation via ICP algorithm and stereo vision Kinect depth sensor method was used in this work. A conducted two different stereo system analysis showed that Kinect achieves best runtime performance and its accuracy is 2-5 times more superior than a regular stereo setup. Object detection experiments showcased a maximum object detection accuracy of nearly 90% and speed of 15 fps for standard size VGA 640x480 resolution images. Accomplished object position and orientation estimation experiment using ICP method showed, that average absolute position and orientation detection error is respectively 3.4cm and 30 degrees while the runtime speed – 2 fps. Further optimization and data size minimization is necessary to achieve better efficiency on a resource limited mobile robot platform. The robot hardware system was also successfully implemented and tested in this work for object position and orientation detection.

Nonrigid deformation modeling and estimation from images is a technically challenging task due to its nonlinear, nonconvex and high-dimensional nature. Traditional optimization procedures often rely on good initializations and give locally optimal solutions. On the other hand, learning-based methods that directly model the relationship between deformed images and their parameters either cannot handle complicated forms of mapping, or suffer from the Nyquist Limit and the curse of dimensionality due to high degrees of freedom in the deformation space. In particular, to achieve a worst-case guarantee of ∈ error for a deformation with d degrees of freedom, the sample complexity required is O(1/∈d). In this thesis, a generative model for deformation is established and analyzed using a unified theoretical framework. Based on the framework, three algorithms, Data-Driven Descent, Top-down and Bottom-up Hierarchical Models, are designed and constructed to solve the generative model. Under Lipschitz conditions that rule out unsolvable cases (e.g., deformation of a blank image), all algorithms achieve globally optimal solutions to the specific generative model. The sample complexity of these methods is substantially lower than that of learning-based approaches, which are agnostic to deformation modeling. To achieve global optimality guarantees with lower sample complexity, the structureembedded in the deformation model is exploited. In particular, Data-driven Descentrelates two deformed images that are far away in the parameter space by compositionalstructures of deformation and reduce the sample complexity to O(Cd log 1/∈).Top-down Hierarchical Model factorizes the local deformation into patches once theglobal deformation has been estimated approximately and further reduce the samplecomplexity to O(Cd/1+C2 log 1/∈). Finally, the Bottom-up Hierarchical Model buildsrepresentations that are invariant to local deformation. With the representations, theglobal deformation can be estimated independently of local deformation, reducingthe sample complexity to O((C/∈)d0) (d0 ≪ d). From the analysis, this thesis showsthe connections between approaches that are traditionally considered to be of verydifferent nature. New theoretical conjectures on approaches like Deep Learning, arealso provided. practice, broad applications of the proposed approaches have also been demonstrated to estimate water distortion, air turbulence, cloth deformation and human pose with state-of-the-art results. Some approaches even achieve near real-time performance. Finally, application-dependent physics-based models are built with good performance in document rectification and scene depth recovery in turbulent media.

As robotic systems move from well-controlled settings to increasingly unstructured environments, they are required to operate in highly dynamic and cluttered scenarios. Finding an object, estimating its pose, and tracking its pose over time within such scenarios are challenging problems. Although various approaches have been developed to tackle these problems, the scope of objects addressed and the robustness of solutions remain limited. In this thesis, we target a robust object perception using visual sensory information, which spans from the traditional monocular camera to the more recently emerged RGB-D sensor, in unstructured environments. Toward this goal, we address four critical challenges to robust 6-DOF object pose estimation and tracking that current state-of-the-art approaches have, as yet, failed to solve. The first challenge is how to increase the scope of objects by allowing visual perception to handle both textured and textureless objects. A large number of 3D object models are widely available in online object model databases, and these object models provide significant prior information including geometric shapes and photometric appearances. We note that using both geometric and photometric attributes available from these models enables us to handle both textured and textureless objects. This thesis presents our efforts to broaden the spectrum of objects to be handled by combining geometric and photometric features. The second challenge is how to dependably estimate and track the pose of an object despite the clutter in backgrounds. Difficulties in object perception rise with the degree of clutter. Background clutter is likely to lead to false measurements, and false measurements tend to result in inaccurate pose estimates. To tackle significant clutter in backgrounds, we present two multiple pose hypotheses frameworks: a particle filtering framework for tracking and a voting framework for pose estimation. Handling of object discontinuities during tracking, such as severe occlusions, disappearances, and blurring, presents another important challenge. In an ideal scenario, a tracked object is visible throughout the entirety of tracking. However, when an object happens to be occluded by other objects or disappears due to the motions of the object or the camera, difficulties ensue. Because the continuous tracking of an object is critical to robotic manipulation, we propose to devise a method to measure tracking quality and to re-initialize tracking as necessary. The final challenge we address is performing these tasks within real-time constraints. Our particle filtering and voting frameworks, while time-consuming, are composed of repetitive, simple and independent computations. Inspired by that observation, we propose to run massively parallelized frameworks on a GPU for those robotic perception tasks which must operate within strict time constraints.

Imagine that a robot fetched this thesis for you from a book shelf. How doyou think the robot would have been programmed? One possibility is thatexperienced engineers had written low level descriptions of all imaginabletasks, including grasping a small book from this particular shelf. A secondoption would be that the robot tried to learn how to grasp books from yourshelf autonomously, resulting in hours of trial-and-error and several bookson the floor.In this thesis, we argue in favor of a third approach where you teach therobot how to grasp books from your shelf through grasping by demonstration.It is based on the idea of robots learning grasping actions by observinghumans performing them. This imposes minimum requirements on the humanteacher: no programming knowledge and, in this thesis, no need for specialsensory devices. It also maximizes the amount of sources from which therobot can learn: any video footage showing a task performed by a human couldpotentially be used in the learning process. And hopefully it reduces theamount of books that end up on the floor. This document explores the challenges involved in the creation of such asystem. First, the robot should be able to understand what the teacher isdoing with their hands. This means, it needs to estimate the pose of theteacher's hands by visually observing their in the absence of markers or anyother input devices which could interfere with the demonstration. Second,the robot should translate the human representation acquired in terms ofhand poses to its own embodiment. Since the kinematics of the robot arepotentially very different from the human one, defining a similarity measureapplicable to very different bodies becomes a challenge. Third, theexecution of the grasp should be continuously monitored to react toinaccuracies in the robot perception or changes in the grasping scenario.While visual data can help correcting the reaching movement to the object,tactile data enables accurate adaptation of the grasp itself, therebyadjusting the robot's internal model of the scene to reality. Finally,acquiring compact models of human grasping actions can help in bothperceiving human demonstrations more accurately and executing them in a morehuman-like manner. Moreover, modeling human grasps can provide us withinsights about what makes an artificial hand design anthropomorphic,assisting the design of new robotic manipulators and hand prostheses. All these modules try to solve particular subproblems of a grasping bydemonstration system. We hope the research on these subproblems performed inthis thesis will both bring us closer to our dream of a learning robot andcontribute to the multiple research fields where these subproblems arecoming from. / QC 20111125

Thesis (MScEng (Electrical and Electronic Engineering))--University of Stellenbosch, 2011. / ENGLISH ABSTRACT: This thesis presents the design and implementation of all the subsystems required to perform precision autonomous helicopter landings within a low-cost framework. To obtain high-accuracy state estimates during the landing phase a vision-based approach, with a downwards facing camera on the helicopter and a known landing target, was used. An e cient monocular-view pose estimation algorithm was developed to determine the helicopter's relative position and attitude during the landing phase. This algorithm was analysed and compared to existing algorithms in terms of sensitivity, robustness and runtime. An augmented kinematic state estimator was developed to combine measurements from low-cost GPS and inertial measurement units with the high accuracy measurements from the camera system. High-level guidance algorithms, capable of performing waypoint navigation and autonomous landings, were developed. A visual position and attitude measurement (VPAM) node was designed and built to perform the pose estimation and execute the associated algorithms. To increase the node's throughput, a compression scheme is used between the image sensor and the processor to reduce the amount of data that needs to be processed. This reduces processing requirements and allows the entire system to remain on-board with no reliance on radio links. The functionality of the VPAM node was con rmed through a number of practical tests. The node is able to provide measurements of su cient accuracy for the subsequent systems in the autonomous landing system. The functionality of the full system was con rmed in a software environment, as well as through testing using a visually augmented hardware-in-the-loop environment. / AFRIKAANSE OPSOMMING: Hierdie tesis beskryf die ontwikkeling van die substelsels wat vir akkurate outonome helikopter landings benodig word. 'n Onderliggende doel was om al die ontwikkeling binne 'n lae-koste raamwerk te voltooi. Hoe-akkuraatheid toestande word benodig om akkurate landings te verseker. Hierdie metings is verkry deur middel van 'n optiese stelsel, bestaande uit 'n kamera gemonteer op die helikopter en 'n bekende landingsteiken, te ontwikkel. 'n Doeltreffende mono-visie posisie-en-orientasie algoritme is ontwikkel om die helikopter se posisie en orientasie, relatief tot die landingsteiken, te bepaal. Hierdie algoritme is deeglik ondersoek en vergelyk met bestaande algoritmes in terme van sensitiwiteit, robuustheid en uitvoertyd. 'n Optimale kinematiese toestandswaarnemer, wat metings van GPS en inersiele sensore kombineer met die metings van die optiese stelsel, is ontwikkel en deur simulasie bevestig. Hoe-vlak leidingsalgoritmes is ontwikkel wat die helikopter in staat stel om punt-tot-punt navigasie en die landingsprosedure uit te voer. 'n Visuele posisie-en-orientasie meetnodus is ontwikkel om die mono-visie posisie-en orientasie algoritmes uit te voer. Om die deurset te verhoog is 'n saampersingsalgoritme gebruik wat die hoeveelheid data wat verwerk moet word, te verminder. Dit het die benodigde verwerkingskrag verminder, wat verseker het dat alle verwerking op aanboord stelsels kan geskied. Die meetnodus en mono-visie algoritmes is deur middel van praktiese toetse bevestig en is in staat om metings van voldoende akkuraatheid aan die outonome landingstelsel te verskaf. Die werking van die volledige stelsel is, deur simulasies in 'n sagteware en hardeware-indie- lus omgewing, bevestig.

Made available in DSpace on 2014-12-17T15:47:04Z (GMT). No. of bitstreams: 1 SeverinoPGN_TESE.pdf: 2322839 bytes, checksum: eda5c48fde7c920680bcb8d8be8d5d21 (MD5) Previous issue date: 2014-02-27 / The camera motion estimation represents one of the fundamental problems in Computer Vision and it may be solved by several methods. Preemptive RANSAC is one of them, which in spite of its robustness and speed possesses a lack of flexibility related to the requirements of applications and hardware platforms using it. In this work, we propose an improvement to the structure of Preemptive RANSAC in order to overcome such limitations and make it feasible to execute on devices with heterogeneous resources (specially low budget systems) under tighter time and accuracy constraints. We derived a function called BRUMA from Preemptive RANSAC, which is able to generalize several preemption schemes, allowing previously fixed parameters (block size and elimination factor) to be changed according the applications constraints. We also propose the Generalized Preemptive RANSAC method, which allows to determine the maximum number of hipotheses an algorithm may generate. The experiments performed show the superiority of our method in the expected scenarios. Moreover, additional experiments show that the multimethod hypotheses generation achieved more robust results related to the variability in the set of evaluated motion directions / A estima??o de pose/movimento de c?mera constitui um dos problemas fundamentais na vis?o computacional e pode ser resolvido por v?rios m?todos. Dentre estes m?todos se destaca o Preemptive RANSAC (RANSAC Preemptivo), que apesar da robustez e velocidade apresenta problemas de falta de flexibilidade em rela??o a requerimentos das aplica??es e plataformas computacionais utilizadas. Neste trabalho, propomos um aperfei?oamento da estrutura do Preemptive RANSAC para superar esta limita??o e viabilizar sua execu??o em dispositivos com recursos variados (enfatizando os de poucas capacidades) atendendo a requisitos de tempo e precis?o diversos. Derivamos do Preemptive RANSAC uma fun??o a que chamamos BRUMA, que ? capaz de generalizar v?rios esquemas de preemp??o e que permite que par?metros anteriormente fixos (tamanho de bloco e fator de elimina??o) sejam configurados de acordo com as restri??es da aplica??o. Propomos o m?todo Generalized Preemptive RANSAC (RANSAC Preemptivo Generalizado) que permite ainda alterar a quantidade m?xima de hip?teses a gerar. Os experimentos demonstraram superioridade de nossa proposta nos cen?rios esperados. Al?m disso, experimentos adicionais demonstram que a gera??o de hip?teses multim?todos produz resultados mais robustos em rela??o ? variabilidade nos tipos de movimento executados

[pt] A identificação facial em vídeo é uma aplicação de grande interesse na comunidade cientifica e na indústria de segurança, impulsionando a busca por técnicas mais robustas e eficientes. Atualmente, no âmbito de reconhecimento facial, as técnicas de identificação frontal são as com melhor taxa de acerto quando comparadas com outras técnicas não frontais. Esse trabalho tem como objetivo principal buscar métodos de avaliar imagens em vídeo em busca de pessoas (rostos), avaliando se a qualidade da imagem está dentro de uma faixa aceitável que permita um algoritmo de reconhecimento facial frontal identificar os indivíduos. Propõem-se maneiras de diminuir a carga de processamento para permitir a avaliação do máximo número de indivíduos numa imagem sem afetar o desempenho em tempo real. Isso é feito através de uma análise da maior parte das técnicas utilizadas nos últimos anos e do estado da arte, compilando toda a informação para ser aplicada em um projeto que utiliza os pontos fortes de cada uma e compense suas deficiências. O resultado é uma plataforma multithread. Para avaliação do desempenho foram realizados testes de carga computacional com o uso de um vídeo público disponibilizado na AVSS (Advanced Video and Signal based Surveillance). Os resultados mostram que a arquitetura promove um melhor uso dos recursos computacionais, permitindo um uso de uma gama maior de algoritmos em cada segmento que compõe a arquitetura, podendo ser selecionados segundo critérios de qualidade da imagem e ambiente onde o vídeo é capturado. / [en] Face recognition in video is an application of great interest in the scientific community and in the surveillance industry, boosting the search for efficient and robust techniques. Nowadays, in the facial recognition field, the frontal identification techniques are those with the best hit ratio when compared with others non-frontal techniques. This work has as main objective seek for methods to evaluate images in video to look for people (faces), assessing if the image quality is in an acceptable range that allows a facial recognition algorithm to identify the individuals. It s proposed ways to decrease the processing load to allow a maximum number of individuals assessed in an image without affecting the real time performance. This is reached through analysis of most the techniques used in the last years and the state-of-the-art, compiling all information to be applied in a project that uses the strengths of each one and offset its shortcomings. The outcome is a multithread platform. Performance evaluation was performed through computational load tests by using public videos available in AVSS ( Advanced Video and Signal based Surveillance). The outcomes show that the architecture makes a better use of the computational resources, allowing use of a wide range of algorithms in every segment of the architecture that can be selected according to quality image and video environment criteria.

Articulated Heavy Goods Vehicles (HGVs) are more efficient than conventional rigid lorries, but exhibit reduced low-speed manoeuvrability and high-speed stability. Technologies such as autonomous reversing and path-following trailer steering can mitigate this, but practical limitations of the available sensing technologies restrict their commercialisation potential. This dissertation describes the development of practical vision-based articulation angle and trailer off-tracking sensing for HGVs. Chapter 1 provides a background and literature review, covering important vehicle technologies, existing commercial and experimental sensors for articulation angle and off-tracking measurement, and relevant vision-based technologies. This is followed by an introduction to pertinent computer vision theory and terminology in Chapter 2. Chapter 3 describes the development and simulation-based assessment of an articulation angle sensing concept. It utilises a rear-facing camera mounted behind the truck or tractor, and one of two proposed image processing methods: template-matching and Parallel Tracking and Mapping (PTAM). The PTAM-based method was shown to be the more accurate and versatile method in full-scale vehicle tests. RMS measurement errors of 0.4-1.6° were observed in tests on a tractor semi-trailer (Chapter 4), and 0.8-2.4° in tests on a Nordic combination with two articulation points (Chapter 5). The system requires no truck-trailer communication links or artificial markers, and is compatible with multiple trailer shapes, but was found to have increasing errors at higher articulation angles. Chapter 6 describes the development and simulation-based assessment of a trailer off-tracking sensing concept, which utilises a trailer-mounted stereo camera pair and visual odometry. The concept was evaluated in full-scale tests on a tractor semi-trailer combination in which camera location and stereo baseline were varied, presented in Chapter 7. RMS measurement errors of 0.11-0.13 m were obtained in some tests, but a sensitivity to camera alignment was discovered in others which negatively affected results. A very stiff stereo camera mount with a sub-0.5 m baseline is suggested for future experiments. A summary of the main conclusions, a review of the objectives, and recommendations for future work are given in Chapter 8. Recommendations include further refinement of both sensors, an investigation into lighting sensitivity, and alternative applications of the sensors.

Today modern industry systems are almost fully automated. The high requirements regarding speed, flexibility, precision and reliability makes it in some cases very difficult to create. One of the most willingly researched solution to solve many processes without human influence is bin-picking. Bin picking is a very complex process which integrates devices such as: robotic grasping arm, vision system, collision avoidance algorithms and many others. This paper describes the creation of a vision system - the most important part of the whole bin-picking system. Authors propose a model-based solution for estimating a best pick-up candidate position and orientation. In this method database is created from 3D CAD model, compared with processed image from the 3D scanner. Paper widely describes database creation from 3D STL model, Sick IVP 3D scanner configuration and creation of the comparing algorithm based on autocorrelation function and morphological operators. The results shows that proposed solution is universal, time efficient, robust and gives opportunities for further work. / +4915782529118