An image-based technique for passive acquisition of three-dimensional (3-D) range data is proposed. The distance is extracted, in this technique, from the estimation of focus conditions on images produced through a monocular imaging system under natural illumination. The image taken from a 3-D object is generally out-of-focus (defocused). For each surface point, the severity of defocus on the image depends upon how far away the point is from the imaging system and how camera (optical) parameters are adjusted. Each setting of the parameters can be recorded physically, and associated in object-space with the inverse of a distance that corresponds to the position for the sharpest imaging under this setting. Therefore, for a given surface point the defocus severity is a function of such an inverse object-distance. It can be shown that this function is symmetrical to, and monotonic on both sides of, a point corresponding to the inverse distance of the surface point. To estimate the parameters of the function (one of which is the inverse distance of the surface point), 3~4 images need to be taken under different camera settings with known associated inverse distances in object-space, determined through a once-for-all calibration procedure. Defocus severity is evaluated from a calculation on the window image that corresponds to a small area around the surface point, and the inverse variance in the window is suggested in this technique for the best performance. The 3-D surface geometry is acquired by applying the algorithm, in parallel, to all surface points in the field of view. Various aspects of the technique are discussed and several algorithms are developed. The technique is implemented on an opto-digital imaging system and evaluated under different conditions. A number of objects are tested to demonstrate its performance. / Thesis / Master of Engineering (ME)

京都大学 / 新制・課程博士 / 博士(工学) / 甲第24593号 / 工博第5099号 / 新制||工||1976(附属図書館) / 京都大学大学院工学研究科都市社会工学専攻 / (主査)教授 角 哲也, 准教授 竹門 康弘, 准教授 Kantoush Sameh / 学位規則第4条第1項該当 / Doctor of Philosophy (Engineering) / Kyoto University / DFAM

Image-based 3D reconstruction has been employed in industrial metrology for micro measurements and quality control purposes. However, generating a highly-detailed and reliable 3D reconstruction of non-collaborative surfaces (textureless, shiny, and transparent) is still an open issue. This thesis presents various methodologies to successfully generate a highly-detailed and reliable 3D reconstruction of non-collaborative objects using the proposed photometric stereo image acquisition system. The first proposed method employs geometric construction to integrate photogrammetry and photometric stereo in order to overcome each technique's limitations and to leverage each technique's strengths in order to reconstruct an accurate and high-resolution topography of non-collaborative surfaces. This method uses accurate photogrammetric 3D measurements to rectify the global shape deviation of photometric stereo meanwhile uses photometric stereo to recover the high detailed topography of the object. The second method combines the high spatial frequencies of photometric stereo depth map with the low frequencies of photogrammetric depth map in frequency domain to produce accurate low frequencies while retaining high frequencies. For the third approach, we utilize light directionality to improve texture quality by leveraging shade and shadow phenomena using the proposed image-capturing system that employs several light sources for highlighting roughness and microstructures on the surface. And finally, we present two methods that effectively orient images by leveraging the low-contrast textures highlighted on object surfaces (roughness and 3D microstructures) using proper lighting system. Various objects with different surface characteristics including textureless, reflective, and transparent are used to evaluate different proposed approaches. To assess the accuracy of each approach, a comprehensive comparison between reference data and generated 3D points is provided.

This thesis explores the use of computer vision to facilitate three different processes of a quarry's operation. The first is the blasting process. This is where operators determine where to drill in order to execute an efficient and safe blast. Having an operator manually determine the drilling angles and positions can lead to inefficient and dangerous blasts. By using two cameras, oriented vertically, and separated by a fixed baseline, Structure from Motion techniques can be used to create a scaled 3D model of a bench. This can then be analyzed to provide operators with borehole locations and drilling angles in relation to fixed reference targets. The second process explored is the crushing process, where the rocks pass through different crushers that reduce the rocks into smaller sizes. The crushed rocks are then dropped onto a moving conveyor belt. The maximum dimension of the rocks exiting the crushers should not exceed size thresholds that are specific to each crusher. This thesis presents a 2D vision system capable of estimating the size distribution of the rocks by attempting to segment the rocks in each image. The size distribution, based on the maximum dimension of each rock, is estimated by finding the maximum dimension in the image in pixels and converting that to inches. The third process of the quarry operations explored is where the final product is piled up to form stockpiles. For inventory purposes, operators often carry out a manual estimation of the size of a the stockpile. This thesis presents a vision system capable of providing a more accurate estimate for the size of the stockpile by using Structure from Motion techniques to create a 3D reconstruction. User interaction helps to find the points that are relevant to the stockpile in the resulting point cloud, which are then used to estimate the volume. / Master of Science

Sample collection is a common task for mobile robots and there are a variety of manipulators available to perform this operation. This thesis presents a novel scoop sample collection system design which is able to both collect and contain a sample using the same hardware. To ease the operator burden during sampling the scoop system is paired with new semi-autonomous and fully autonomous collection techniques. These are derived from data provided by colored 3D point clouds produced via image-based 3D reconstructions. A custom robotic mobility platform, the Scoopbot, is introduced to perform completely automated imaging of the sampling area and also to pick up the desired sample. The Scoopbot is wirelessly controlled by a base station computer which runs software to create and analyze the 3D point cloud models. Relevant sample parameters, such as dimensions and volume, are calculated from the reconstruction and reported to the operator. During tests of the system in full (48 images) and fast (6-8 images) modes the Scoopbot was able to identify and retrieve a sample without any human intervention. Finally, a new building crack detection algorithm (CDA) is created to use the 3D point cloud outputs from image sets gathered by a mobile robot. The CDA was shown to successfully identify and color-code several cracks in a full-scale concrete building element. / Master of Science

Depth-Image-Based Rendering (DIBR) est une technologie fondamentale dans plusieurs applications liées à la 3D, telles que la vidéo en mode point de vue libre (FVV), la réalité virtuelle (VR) et la réalité augmentée (AR). Cependant, l'évaluation de la qualité des vues synthétisées par DIBR a également posé de nouveaux problèmes, car ce processus induit de nouveaux types de distorsions, qui sont intrinsèquement différentes des distorsions provoquées par le codage vidéo. Ce travail est destiné à mieux évaluer la qualité des vues synthétisées par DIBR en multimédia immersif. Au chapitre 2, nous proposons deux métriques complètements sans référence (NR). Le principe de la première métrique NR NIQSV consiste à utiliser plusieurs opérations morphologiques d’ouverture et de fermeture pour détecter et mesurer les distorsions, telles que les régions floues et l’effritement. Dans la deuxième métrique NR NIQSV+, nous améliorons NIQSV en ajoutant un détecteur de “black hole” et une détection “stretching”.Au chapitre 3, nous proposons deux métriques de référence complète pour traiter les distorsions géométriques à l'aide d'un masque de désocclusion et d'une méthode de correspondance de blocs multi-résolution. Au chapitre 4, nous présentons une nouvelle base de données d'images synthétisée par DIBR avec ses scores subjectifs associés. Ce travail se concentre sur les distorsions uniquement induites par différentes méthodes de synthèse de DIBR qui déterminent la qualité d’expérience (QoE) de ces applications liées à DIBR. En outre, nous effectuons également une analyse de référence des mesures d'évaluation de la qualité objective de pointe pour les vues synthétisées par DIBR sur cette base de données. Le chapitre 5 conclut les contributions de cette thèse et donne quelques orientations pour les travaux futurs. / Depth-Image-Based Rendering (DIBR) is a fundamental technology in several 3D-related applications, such as Free viewpoint video (FVV), Virtual Reality (VR) and Augmented Reality (AR). However, new challenges have also been brought in assessing the quality of DIBR-synthesized views since this process induces some new types of distortions, which are inherently different from the distortions caused by video coding. This work is dedicated to better evaluate the quality of DIBRsynthesized views in immersive multimedia. In chapter 2, we propose a completely No-reference (NR) metric. The principle of the first NR metrics NIQSV is to use a couple of opening and closing morphological operations to detect and measure the distortions, such as “blurry regions” and “crumbling”. In the second NR metric NIQSV+, we improve NIQSV by adding a “black hole” and a “stretching” detection. In chapter 3, we propose two Fullreference metrics to handle the geometric distortions by using a dis-occlusion mask and a multi-resolution block matching methods.In chapter 4, we present a new DIBR-synthesized image database with its associated subjective scores. This work focuses on the distortions only induced by different DIBR synthesis methods which determine the quality of experience (QoE) of these DIBR related applications. In addition, we also conduct a benchmark of the state-of-the-art objective quality assessment metrics for DIBR-synthesized views on this database. The chapter 5 concludes the contributions of this thesis and gives some directions of future work.

Cílem této práce je demonstrovat možnost renderování srsti přímo do existujících obrazů bez toho, aby bylo po uživateli požadováno překreslení všech pixelů nebo dodání kompletní 3D geometrie a osvětlení. Srst je přidána na povrch objektů pomocí extrakce jejich přibližného tvaru a světelných informací z obrazu a takto získaný objekt je poté přerenderován. Tento přístup je nový v tom, že vysokoúrovňové úpravy obrazu (jako např. přidání srsti), mohou úspěšně vést k vizuálně korektním výsledkům a to i přes omezení nepřesnou geometrií a světelnými podmínkami. Relativně velká množina technik použitých v této práci zahrnuje obrazy s velkým dynamickým rozsahem, metody extrakce 3D tvaru z obrazu, výsledky výzkumu vnímání tvaru a osvětlení a fotorealistické renderování. Hlavním cílem práce je potvrdit koncept popsaný výše. Hlavním implementačním jazykem bylo C++ s použitím knihoven wxWidgets, OpenGL a libTIFF. Renderování bylo realizováno v software 3Delight kompatibilním se standardem Renderman, za pomoci množiny shaderů implementovaných v nativním jazyce Rendermanu.

L'objectif principal de cette thèse est de développer des outils pour la reconstruction de l'environnement urbain à partir d'images. Les entrées typiques de notre travail est un ensemble d'images de façades, des empreintes au sol de bâtiments, et des modèles 3D reconstruits à partir d'images aériennes. Les principales étapes comprennent le calibrage des images,le recalage avec le modèle 3D, la récupération des informations de profondeur ainsi que la sémantique des façades.Pour atteindre cet objectif, nous utilisons des techniques du domaine de vision par ordinateur, reconnaissance de formes et de l'informatique graphique. Les contributions de notre approche sont présentés en deux parties.Dans la première partie, nous nous sommes concentrés sur des techniques de reconstruction multi-vues dans le but de récupérer automatiquement les informations de profondeur de façades à partir un ensemble des photographies non calibrées. Tout d'abord, nous utilisons la technique structure et mouvement pour calibrer automatiquement l'ensemble des photographies. Ensuite, nous proposons des techniques pour le recalage de la reconstruction avec un modèle 3D. Enfin, nous proposons des techniques de reconstruction 3d dense (stéréo multi-vues et voxel coloring) pour produire un maillage 3D texturé d'une scène d'un ensemble d'images calibrées.La deuxième partie est consacrée à la reconstruction à partir d'une seule vue et son objectif est de récupérer la structure sémantique d'une façade d'une image ortho-rectifiée. La nouveauté de cette approche est l'utilisation d'une grammaire stochastique décrivant un style architectural comme modèle pour la reconstruction de façades. nous combinons un ensemble de détecteurs image avec une méthode d'optimisation globale stochastique en utilisant l'algorithme Metropolis-Hastings. / The main goal of this thesis is to develop innovative and practicaltools for the reconstruction of buildings from images. The typical input to our workis a set of facade images, building footprints, and coarse 3d models reconstructedfrom aerial images. The main steps include the calibration of the photographs,the registration with the coarse 3d model, the recovery of depth and sematicinformation, and the refinement of the coarse 3d model.To achieve this goal, we use computer vision, pattern recognition and computergraphics techniques. Contributions in this approach are presented on two parts.In the first part, we focused on multiple view reconstruction techniques withthe aim to automatically recover the depth information of facades from a setof uncalibrated photographs. First, we use structure from motion techniques toautomatically calibrate the set of photographs. Then, we propose techniques for theregistration of the sparse reconstruction to a coarse 3d model. Finally, we proposean accelerated multi-view stereo and voxel coloring framework using graphicshardware to produce a textured 3d mesh of a scene from a set of calibrated images.The second part is dedicated to single view reconstruction and its aim is to recoverthe semantic structure of a facade from an ortho-rectified image. The novelty ofthis approach is the use of a stochastic grammar describing an architectural style asa model for facade reconstruction. we combine bottom-up detection with top-downproposals to optimize the facade structure using the Metropolis-Hastings algorithm.

Title: Interactive Preview Renderer for Complex Camera Models Author: Bohumír Zámečník Department: Department of Software and Computer Science Education Supervisor: Dr. Alexander Wilkie Supervisor's e-mail address: alexander@wilkie.at Abstract: An interactive renderer was implemented that allows users to preview the effects of imaging with lenses, such as depth of field, bokeh (defocus highlights) and tilt-shift lens configurations. It is based on a state-of-the-art method which com- bines the power of GPU rasterization and ray tracing. Many models and interactive visualizations were created. A non-interactive simulation of a complex geometrical lens model has been made which is able to produce optical aberrations. Also a prototype implementation of recent fast spreading filters is available. A thorough summary of the principles of optical image formation, lens models and depth of field rendering methods used in computer graphics is given along with a comparison of the approaches and new insights. New possibilities of representing the behavior of complex lenses are suggested, which could be employed to accelerate the rendering. Keywords: image synthesis, camera models, depth of field, GPU, image-based ray tracing

The Dynamic Positioning (DP) systems currently available employ a feedback controller based on the position and heading error, associated with a wind feed-forward control action to keep the vessel position. This last technology improves the eficiency of the DP by anticipating wind loads. However, there is no consolidated technology to feed the controller with wave loads information and the major issue is related to the wave measurement process. The present thesis aims at filling this technological gap by proposing an alternative approach for the measuring of waves near the vessel. Image-based measurement methods are noninvasive and can produce a spatial and temporal description of the surface, making them suitable for recovering the geometry of liquid surfaces. Nevertheless, these surface reconstruction methods, particularly those applying stereoscopic approaches, have dificulty in measuring waves produced in laboratory facilities, mainly because, in such conditions, the water surface is smooth, translucent and highly specular. Alternatively, intensity based image methods are capable of dealing with these surface characteristics, if employed under controlled conditions and if a suitable reflectance model is selected. In the present study, a well-known reflectance model is applied to recover the parameters of regular waves produced in an offshore basin. Firstly, an experimental setup is proposed, designed to grant appropriate conditions for the application of the reflectance model, even under conventional laboratory illumination. Later, the second set of experiments with a model scale DP vessel are presented, which applied the image-based method developed before as a wave feed-forward system. Three control methods are evaluated, namely: PID, PD (Proportional-Derivative) and the PD with the wave feed-forward. Results demonstrated that the presence of the wave feed-forward reduced the steady error of the PD controller. These are the first steps towards the practical use of the wave feed-forward, and several aspects remain pending. However, the promising results and discussions regarding the future steps define the contribution of this work. / Os sistemas atuais de Posicionamento Dinâmico (DP) empregam técnicas de controle baseadas na realimentação da posição e do aproamento, associadas à compensação antecipada das cargas de vento (wind feed-forward) para manter a posição da embarcação. Esta tecnologia melhora a experiência do DP, pois antecipa a ação do vento. Entretanto, não há tecnologia consolidada para a pré-compensação de forças ondas (wave feed-forward) e o maior desafio reside na medição desses agentes ambientais. A presente tese tem como objetivo preencher essa lacuna tecnológica propondo uma abordagem alternativa para a medição das ondas próxima ao casco. Métodos de medição baseados em imagem são não invasivos e produzem descrições espaciais e temporais da superfície analisada, tornando-os particularmente adequados à medição de superfícies líquidas. Entretanto, os métodos comumente empregados, principalmente aqueles baseados em imagens estéreo, são incapazes de medir ondas produzidas em ambiente de laboratório, pois, nestas condições, a superfície da água é lisa, apresenta transparência e comporta-se como um espelho. Por outro lado, métodos baseados na intensidade de luz são capazes de lidar com tais características, se aplicados em condições adequadas e quando empregam modelos apropriados. Neste trabalho, um modelo de reetância amplamente conhecido é empregado para extrair os parâmetros principais de ondas regulares produzidas em um tanque de provas offshore. Inicialmente, propõe-se um arranjo experimental que permita a aplicação do modelo de refetância construído, mesmo sob as condições de iluminação naturais do laboratório. Posteriormente, um segundo conjunto de experimentos com um modelo de embarcação DP é proposto, no qual se aplica o método baseado em imagem, testado anteriormente, como sistema de pré-compensação de forças de onda. Três métodos de controle são avaliados, quais sejam: PID (Proporcional-Integral-Derivativo), PD (Proporcional-Derivativo) e o PD-WFF (PD associado ao wave feed-forward). Os resultados demonstram que a presença pré-conpensação de forças de onda reduz o erro em regime do controlador PD. Estes são os primeiros passos em direção da incorporação da pré-compensação de forças de onda nos sistemas DP reais e diversos aspectos técnicos ainda estão pendentes. Entretanto, os resultados promissores e discussões acerca dos futuros passos da pesquisa definem a contribuição do presente trabalho no campo do Posicionamento Dinâmico.