View planning is an important part of achieving full robotic autonomy. The ability to incorporate the view of a robot as well as the ability of evaluating and choosing the best view are highly desired abilities for a wide variety of robotics applications. In this work we present a new iterative optimization scheme and evaluational method in order to choose the Next Best View in the framework of object modeling. As unmanned aerial vehicles (UAVs) become more and more common we take advantage of the additional degrees of freedom a UAV offers. We show, in simulation, that the proposed method is able to pick out views that are highly relevant in order to model the observed object in question. It is shown that with iterative optimization the resulting view is improved. Additional experiments where the method is deployed onto a real UAV show the real-world applicability of the method. / Planering av hur vyer för bildinhämtning skall väljas är en viktig del för att uppnå full robotautonomi. Att kunna infoga information från olika vyer samt att välja den bästa vyn är viktigt för många olika robottillämpningar. I det här arbetet presenterar vi en ny iterativ optimeringsbaserad metod för att välja bästa nästa vy inom ramen för objektmodellering. I takt med att flygande robotar, s.k. UAVer, blir allt vanligare kan vi utnyttja de ytterligare frihetsgraderna som ett UAV-system ger. Vi visar i simulering att den föreslagna metoden kan välja ut vyer som är relevanta för att modellera ett visst objekt och att vyerna förbättras genom vår iterativa optimering. Experiment på en verklig UAV visar att metoden är tillämpbar i praktiken.

<p> In this thesis the problem of object recognition/pose estimation using active sensing is investigated. It is assumed that multiple cameras acquire images from different view angles of an object belonging to a set of a priori known objects. The eigenspace method is used to process the sensory observations and produce an abstract measurement vector. This step is necessary to avoid the manipulation of the original sensor data, i.e. large images, that can render the sensor modelling and matching process practically infeasible.</p> <p> The eigenspace representation is known to have shortcomings in dealing with structured noise such as occlusion. To overcome this problem, models of occlusions and sensor noise have been incorporated into the probabilistic model of sensor/object to increase robustness with respect to such uncertainties. The active recognition algorithm has also been modified to consider the possibility of occlusion, as well as variation in the occlusion levels due to camera movements.</p> <p> A recursive Bayesian state estimation problem is formulated to model the observation uncertainties through a probabilistic scheme. This enables us to identify the object and estimate its pose by fusing the information obtained from individual cameras. To this end, an extensive training step is performed, providing the system with the sensor model required for the Bayesian estimation. In order to enhance the quality of the estimates and to reduce the number of images taken, we employ active real-time viewpoint planning strategies to position cameras. For that purpose, the positions of cameras are controlled based on two different statistical performance criteria, namely the Mutual Information (MI) and Cramér-Rao Lower Bound (CRLB).</p> <p> A multi-camera active vision system has been developed in order to implement the ideas proposed in this thesis. Comparative Monte Carlo experiments conducted with the two-camera system demonstrate the effectiveness of the proposed methods in object classification/pose estimation in the presence of structured noise. Different concepts introduced in this work, i.e., the multi-camera data fusion, the occlusion modelling, and the active camera movement, all improve the recognition process significantly. Specifically, these approaches all increase the recognition rate, decrease the number of steps taken before recognition is completed, and enhance robustness with respect to partial occlusion considerably.</p> / Thesis / Master of Applied Science (MASc)

The maintenance of infrastructure is critical to the well-being of society. This work focuses on a novel method for inspecting the exterior of dams using unmanned aerial vehicles (UAVs) in an automated fashion. The UAVs are equipped with optical sensors capturing still images. The resulting images are used to generate three-dimensional (3D) models using Structure from Motion (SfM) computer software. The SfM models are then used to inspect the exterior of the dam. As typical dam inspections entail completing a checklist of inspection items with varied degrees of precision (e.g. a concrete spillway may be finely inspected for cracking or joint deterioration while the general stability and water-tightness of a large embankment may be observed from a distance), a targeted inspection is also needed for the UAV method. In conjunction with the work presented in this thesis, a novel algorithm was developed which uses camera view planning across multiple proximity levels to generate a set of camera poses (positions and orientations) which can be collected in an autonomous UAV flight that facilitates generation of SfM models having tiered model quality for targeted inspection of infrastructure features. In this thesis, this novel algorithm and accompanying mobile application (referred to together as the novel advanced autonomous method) were field validated at Tibble Fork Dam, UT. The advanced autonomous method was compared to two other common image acquisition methods—basic autonomous and manual piloted—based on the SfM models produced from the collected image sets. The advanced autonomous method was found to produce models having tiered quality needed for efficient targeted inspection (25% and 50% higher resolution in medium and high priority target areas). The advanced autonomous method was found to produce models having on average 38% higher precise point accuracy (1.3cm) and 53% tighter surface reproducibility (for repeat inspections) (1.9cm) than basic autonomous and manual piloted image acquisition methods. The advanced autonomous method required on average 167% longer flight time and 38% fewer images than the other two methods, resulting in increased field time but decreased processing load. Additionally, viability of the advanced autonomous method for practical dam inspection was assessed through a case study inspection of Tibble Fork Dam using the collected SfM model and corresponding still images. The SfM model and corresponding images were found fully adequate for performing 94% of the inspection tasks and partially adequate for the remaining tasks. In consideration of this and other practical implementation factors such as time and safety, the method appears highly viable as an alternate to or supplement with traditional on-foot visual exterior inspection of dams such as Tibble Fork Dam. Suggestions for future work include adjustments to the optimization framework to improve field efficiency, development of a framework for cooperative inspection using UAV swarms, and development of a more automated workflow that would allow fully-remote dam inspections.

Die vorliegende Arbeit befasst sich mit der Erfassung von Oberflächen mittels Streifenprojektionsverfahren. Dabei wird ein Berechnungsmodell erarbeitet, welches den durch eine Aufnahme erfassten Bereich der Objektoberfläche berechnet und bewertet. Mithilfe einer optimalen Positionierung von Einzelaufnahmen ist es möglich, ein Objekt bei festgelegten Randbedingungen zeitsparend zu erfassen.

La numérisation 3D telle que pratiquée aujourd'hui repose essentiellement sur les connaissances de l'opérateur qui la réalise. La qualité des résultats reste très sensible à la procédure utilisée et par conséquent aux compétences de l'opérateur. Ainsi, la numérisation manuelle est très coûteuse en ressources humaines et matérielles et son résultat dépend fortement du niveau de technicité de l'opérateur. Les solutions de numérisation les plus avancées en milieu industriel sont basées sur une approche d'apprentissage nécessitant une adaptation manuelle pour chaque pièce. Ces systèmes sont donc semi-automatiques compte tenu de l'importance de la contribution humaine pour la planification des vues.Mon projet de thèse se focalise sur la définition d'un procédé de numérisation 3D automatique et intelligente. Ce procédé est présenté sous forme d'une séquence de processus qui sont la planification de vues, la planification de trajectoires, l'acquisition et les post-traitements des données acquises. L'originalité de notre démarche de numérisation est qu'elle est générique parce qu'elle n'est pas liée aux outils et méthodes utilisés pour la réalisation des tâches liées à chaque processus. Nous avons également développé trois méthodes de planification de vues pour la numérisation d'objets sans connaissance a priori de leurs formes. Ces méthodes garantissent une indépendance des résultats par rapport au savoir-faire de l'opérateur. L'originalité de ces approches est qu'elles sont applicables à tous types de scanners. Nous avons implanté ces méthodes sur une cellule de numérisation robotisée. Nos approches assurent une reconstruction progressive et intelligente d'un large panel d'objets de différentes classes de complexité en déplaçant efficacement le scanner / The manual 3D digitization process is expensive since it requires a highly trained technician who decides about the different views needed to acquire the object model. The quality of the final result strongly depends, in addition to the complexity of the object shape, on the selected viewpoints and thus on the human expertise. Nowadays, the most developed digitization strategies in industry are based on a teaching approach in which a human operator manually determines one set of poses for the ranging device. The main drawback of this methodology is the influence of the operator's expertise. Moreover, this technique does not fulfill the high level requirement of industrial applications which require reliable, repeatable, and fast programming routines.My thesis project focuses on the definition of a procedure for automatic and intelligent 3D digitization. This procedure is presented as a sequence of processes that are essentially the view planning, the motion planning, the acquisition and the post-processing of the acquired data. The advantage of our procedure is that it is generic since it is not performed for a specific scanning system. Moreover, it is not dependent on the methods used to perform the tasks associated with each elementary process. We also developed three view planning methods to generate a complete 3D model of unknown and complex objects that we implemented on a robotic cell. Our methods enable fast and complete 3D reconstruction while moving efficiently the scanner. Additionaly, our approaches are applicable to all kinds of range sensors.

Die vorliegende Arbeit befasst sich mit der Erfassung von Oberflächen mittels Streifenprojektionsverfahren. Dabei wird ein Berechnungsmodell erarbeitet, welches den durch eine Aufnahme erfassten Bereich der Objektoberfläche berechnet und bewertet. Mithilfe einer optimalen Positionierung von Einzelaufnahmen ist es möglich, ein Objekt bei festgelegten Randbedingungen zeitsparend zu erfassen.