Orientador: Prof. Dr. Celso Setsuo Kurashima / Dissertação (mestrado) - Universidade Federal do ABC, Programa de Pós-Graduação em Engenharia da Informação, 2016. / Os computadores pessoais possuem um papel importante na vida das pessoas, para auxílio em suas tarefas cotidianas como trabalho, entretenimento, e comunicação. Na comunicação, os computadores pessoais podem ser utilizados para chamadas de teleconferência, recebendo e transmitindo imagens e áudio entre longas distâncias. Para adicionar imersão na experiência de teleconferência, e permitir a escolha do ponto de visão por parte do usuário, uma teleconferência tridimensional com ponto de visão livre pode ser realizada, se modelos faciais tridimensionais forem criados a partir de imagens. Essa dissertação apresenta a investigação de métodos de geração de Malha Poligonal a partir de informações 3D extraídas de imagens, para aplicação em sistemas de teleconferências imersivas com ponto de visão livre. Um método de geração de Malha Poligonal chamado Quad Grid foi proposto para permitir a modelagem facial 3D em tempo real. As malhas poligonais das faces foram analisadas em experimentos objetivos e subjetivos. Os experimentos mostraram que o método Quad Grid teve resultado satisfatório e é um método rápido para a reconstrução facial 3D com malha poligonal. O software prova de conceito dessa dissertação resultou em um sistema de teleconferência 3D com ponto de visão livre. / The personal computers have an important role in the people lives to help in their daily tasks such as work, entertainment, and communication. With regard to the communication, the personal computer may be used for conference calls, receiving and transmitting images and audio between long distances. To add immersion inside the teleconferencing experience and allow the choice of viewing point by the user, a three-dimensional teleconference with free view point can be performed if three-dimensional facial models are created from images. This paper presents the research methods of generating polygonal mesh from 3D information extracted from images, for use in immersive conference calls systems with free view point. A method of generating polygonal mesh called Quad Grid has been proposed to allow the 3D face modeling in real time. The polygonal mesh of the faces were analyzed in objective and subjective experiments. The experiments showed that the Quad Grid method was satisfactory and is a fast method for 3D facial reconstruction with polygonal mesh. The proof of concept software of this Master Thesis resulted in a 3D teleconferencing system with free view point.

abstract: Using stereo vision for 3D reconstruction and depth estimation has become a popular and promising research area as it has a simple setup with passive cameras and relatively efficient processing procedure. The work in this dissertation focuses on locally adaptive stereo vision methods and applications to different imaging setups and image scenes. Solder ball height and substrate coplanarity inspection is essential to the detection of potential connectivity issues in semi-conductor units. Current ball height and substrate coplanarity inspection tools are expensive and slow, which makes them difficult to use in a real-time manufacturing setting. In this dissertation, an automatic, stereo vision based, in-line ball height and coplanarity inspection method is presented. The proposed method includes an imaging setup together with a computer vision algorithm for reliable, in-line ball height measurement. The imaging setup and calibration, ball height estimation and substrate coplanarity calculation are presented with novel stereo vision methods. The results of the proposed method are evaluated in a measurement capability analysis (MCA) procedure and compared with the ground-truth obtained by an existing laser scanning tool and an existing confocal inspection tool. The proposed system outperforms existing inspection tools in terms of accuracy and stability. In a rectified stereo vision system, stereo matching methods can be categorized into global methods and local methods. Local stereo methods are more suitable for real-time processing purposes with competitive accuracy as compared with global methods. This work proposes a stereo matching method based on sparse locally adaptive cost aggregation. In order to reduce outlier disparity values that correspond to mis-matches, a novel sparse disparity subset selection method is proposed by assigning a significance status to candidate disparity values, and selecting the significant disparity values adaptively. An adaptive guided filtering method using the disparity subset for refined cost aggregation and disparity calculation is demonstrated. The proposed stereo matching algorithm is tested on the Middlebury and the KITTI stereo evaluation benchmark images. A performance analysis of the proposed method in terms of the I0 norm of the disparity subset is presented to demonstrate the achieved efficiency and accuracy. / Dissertation/Thesis / Doctoral Dissertation Electrical Engineering 2017

La thèse s'inscrit dans le domaine de la vision par ordinateur. Il s'agit, dans un environnement intérieur inconnu, partiellement connu ou connu de trouver la position et l'orientation d'une camera mobile en temps réel à partir d'une séquence vidéo prise par cette même camera. Le sujet implique également la reconstruction 3D de l'environnement. Les algorithmes de vision seront implémentés et testés sur des plateformes massivement parallèles. Processing the video sequence of a indoor camera in motion we have to find the position and angle of the camera in real time. We will use a single prime lens camera. It may involve an unknown, partially known or well known environment. A big part of the computation is the 3D reconstruction of the scene. The algorithms used to locate the camera will be implemented and tested on GPU. / In this thesis, we explore the problem of modeling an unknown environment using monocular vision for localization applications. We focus in modeling dynamic indoor environments. Many objects in indoor environments are likely to be moved. These movements significantly affect the structure and appearance of the environment and disrupt the existing methods of visual localization. We present in this work a new approach for modeling the environment and its evolution with time. We define explicitly the scene as a static structure and a set of dynamic objects. The object is defined as a rigid entity that a user can take, move and that is visually detectable. First, we show how to automatically discover new objects in a dynamic environment. Existing methods of visual localization simply ignore the inconsistencies due to changes in the scene. We aim to analyze these changes to extract additional information. Without any prior knowledge, an object is a set of points with coherent motion relative to the static structure of the scene. We combine two methods of visual localization to compare various explorations in the same environment taken at different time. The comparison enables to detect objects that have moved between the two shots. For each object, a geometric model and an appearance model are learned. Moreover, we extend the scene model while updating the metrical map and the topological map of the static structure of the environment. Object discovery using motion is based on a new algorithm of multiple structures detection in an image pair. Given a set of correspondences between two views, the method based on RANSAC extracts the different structures corresponding to different model parameterizations seen in the data. The method is applied to homography estimation to detect planar structures and to fundamental matrix estimation to detect structures that have been shifted one from another. Our approach for dynamic scene modeling is applied in a new formulation of place recognition to take into account the presence of dynamic objects in the environment. The model of the place consists in an appearance model of the static structure observed in that place. An object database is learned from previous observations in the environment with the method of object discovery using motion. The place recognition we propose detects the dynamic objects seen in the place and rejects the false detection due to these objects. The different methods described in this dissertation are tested on synthetic and real data. Qualitative and quantitative results are presented throughout the dissertation.

Made available in DSpace on 2015-05-08T14:59:54Z (GMT). No. of bitstreams: 1 arquivototal.pdf: 6740318 bytes, checksum: 039aeeeb7ab617f3306f6110a5f307b0 (MD5) Previous issue date: 2012-07-10 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES / The massive use of mills and hydrocyclones operating in closed circuit, in several great relevance socio-economic industrial applications, checks special importance to the problem of the development of processes and mathematical models returned to the optimization of the circuit. This work presents an investigation about about techniques for reducing the prediction error (approximation by spheres) to increase accuracy in non-contact measurements of particle size of minerals in order to increase the effciency of classiffcation by hydrocyclones. Computational methods for counting and measuring particles were developed and implemented, in order to compare the computational vision and particle size separation sieving methods. The proposed method also deals with 3D reconstruction of the particles based on the measurement of their heights. To this goal, a stack of images with variable focus is captured. From this stack calculate the average particle size (estimated from the z axis), display the pictures of the stack through the focus index and the image height. An estimate of the average height of the particles is done and than approximate the shape of the particles (spheroids) and comparison with the results approximating a sphere. Measuring instruments have been adapted for the acquisition of images and also for estimating the height of the particles of the samples. / A utilização massiva de moinhos e hidrociclones operando em circuito fechado, em diversas aplicações industriais de grande relevância sócioeconômica, confere especial importância ao problema do desenvolvimento de processos e modelos matemáticos voltados à otimização do circuito de cominuição-classificação. Este trabalho apresenta uma investigação sobre técnicas para redução do erro de predição (aproximação por esferas) em medições sem contato visando aumentar a exatidão na granulometria de minérios com o intuito de aumentar a eficiência de classificação por hidrociclonagem. Métodos computacionais para contagem e mensuração de partículas foram desenvolvidos e implementados, com vistas a comparar a visão computacional ao método de separação de partículas por peneiramento. O método proposto trata, também, da reconstru ção 3D das partículas baseado na medida de suas alturas. Para tanto, uma pilha de imagens com foco variável é capturada. A partir desta pilha calcula-se a altura mé- dia das partículas (estimativa do eixo Z), exibem-se as imagens da pilha completamente em foco e a imagem de índices de alturas. De posse da estimativa da altura média das partículas é feita a aproximação da forma das partículas (esferóides) e a comparação com os resultados da aproximação por esferas. Instrumentos de medição foram adaptados para aquisição das imagens e também para estimar a altura das partículas das amostras.

INTRODUÇÃO: A presença e extensão da perda neuronal no encéfalo dos idosos ainda é controversa. A substância negra (SN) é uma região localizada no mesencéfalo e caracterizada macroscopicamente pela coloração escura, devido à presença neuromelanina. O envelhecimento da SN ainda é um mistério, mas existem grandes quantidades de estudos relativos às modificações sofridas por essa região com o passar dos anos. Alguns estudos detectaram perda neuronal na SN durante o envelhecimento, enquanto outros afirmam que não há alteração significativa durante esse processo. OBJETIVO: Caracterizar as alterações que ocorrem na SN durante o envelhecimento humano, em indivíduos sem sintomas da doença de Parkinson, do ponto de vista citoarquitetônico em 3D. CASUÍSTICA E METODOLOGIA:O presente projeto faz parte do Projeto Envelhecimento Cerebral (PEC) do Grupo de Estudos em Envelhecimento Cerebral (GEEC) da Faculdade de Medicina da Universidade de São Paulo (FMUSP). Foram utilizados 18 casos, classificados em quatro faixas etárias. Quinze casos foram analisados estereológicamente e recontruídos tridimensionalmente. Para estes, foram determinados número de neurônios, volume da SN e densidade neuronal e submetidos à testes estatísticos não paramétricos. Três casos foram imunocorados com quatro diferentes anticorpos: anti tirosina-hidroxilase (TH), anti proteína tau hiperfosforilada, anti proteína alfa-sinucleína e anti calbindina D28k (Calb) e analisados qualitativamente. RESULTADOS: As lâminas imunocoradas com anticorpo anti -sinucleína e anti-tau, foram negativas. As lâminas imunocoradas com anticorpo anti- TH e anti-Calb D28k foram positivas. Dos quinze casos analisados estereologicamente, Para o número de neurônios, encontrou-se: mediana de 504.575 células, valor mínimo de 348.662 células e valor máximo de 672.172 células. Para o volume, encontrou-se mediana de 190,8mm3; valor mínimo de 134,1 mm3 e valor máximo de 267,3mm3. Para a densidade total média, encontrou-se a mediana de 2.517,4 cel/mm3, valor mínimo de 1.603,1 cel/mm3 e valor máximo de 5.015,7 cel/mm3. Realizou-se o teste de correlação de Spearman para as três variáveis neuronais, correlacionando hemisfério direito e esquerdo, encontrou-se uma correlação moderada para número de neurônios e volume, e correlação forte para densidade. Com o mesmo teste, realizou-se análise das variáveis neuronais em função da idade, e não houve alteração estatisticamente significativa. Apenas uma tendência à diminuição no volume total da SN em relação à idade. Também verificou-se a relação entre as variáveis neuronais, por gênero em função da idade e não houve alteração estatisticamente significativa. Nas reconstruções tridimensionais, verificou-se uma grande variabilidade interpessoal entre as 15 SN reconstruídas. DISCUSSÃO: Diversos estudos estereológicos foram feitos, mas poucos deles com análise estereológica sem viés e a maior parte deles demonstra diminuição no número de neurônios, discordando dos achados atuais. A reconstrução tridimensional também não descreve alterações decorrentes do envelhecimento, mas aparentemente modificações interpessoais. CONCLUSÃO: As características da substância negra se mantém ao longo dos anos, porém, ainda devem ser muito mais estudadas / INTRODUTION: Some aspects about maco and micro braischanges are suffering modifications along the years. The presence and extension of neuronal lost in elderly still a big and polemic issue. The SUBSTANTIA NIGRA (SN) is located at the midbrain and characterized by the dark color due to the pigmented neurons, which contain neuromelanin. The SNs aging is still a mystery, nevertheless, there are great researches related to these changes suffered in this region by the years. Some of the researches noticed that neuronal lost in SN during the aging, while others claim that there are no significant modifications through this process.OBJECTIVE: Describing SN changes during the human aging in individuals with no Parkinsons disease symptoms, from the 3D cytoarchitectonic point of view. METHODS: This project is part of the Brain Aging Project of Brain Aging Study Group, in University of São Paulo Medical School. Eighteen cases had been analyzed and classified into four ages groups. Fifteen cases had been analyzed with stereological methos and 3D reconstructed. During the stereological analysis, were determinate the neuronal number, SN volume, neuronal density and submitted to nonparametric statistics test. Three cases had been immunostained with 4 different antibodies anti tyrosine-hydroxylase (TH), anti hyperphosphorylated tau (anti-tau), anti alfa-synuclein (alpha-syn) e anti calbindin D28k (Calb) and qualitatively analyzed. RESULTS: The immunostained slices with the anti alpha-syn and anti-tau antibodies were negative. The immunostained slices with anti TH and anti calb D28K antibodies were positive. From the 15 cases estereologically analyzed, 53,3% were female and 46,7% male. Had been analyzed neuron numbers, SN volume and neuronal density. For the neural numbers were found median of 504.575 cells, minimum value of 348.662 cells and maximum value of 672.172 cells. For the volume were found median of 190,8mm³; minimum value of 134,1mm³ and maximum value of 267,3mm³. For the total density, was found median of de 2.517,4 cel/mm³, minimum value of 1603,1 cel/mm³ , maximum value of 5015,7 cel/mm³. The Spearmans correlation test had been done to the three neuronal variable correlating the right and left hemispheres and was a moderated correlation between neuron numbers and neuron volume and strong correlation to density. Using the same test, the rate neuron by the age analysis had been concluded and there was no significant change.There was noticed just one trend to the reduction in the SN total volume related to aging. It was also checked the relation between the neurons variety by gender considering aging and there was no significant changes. In the 3D reconstruction, was possible to notice a great interpersonal variability between the 15 SN rebuilt. DISCUSSION: Many stereological researches had been done, however, a few of them used the unbiased stereological analysis and most of them show a reduction in the neuron number, disagreeing from the current conclusions. The 3D reconstruction also dont describe changes as aging result, but apparently interpersonal changes. CONCLUSION: The Substantia Nigra maintain the same characteristics along the years, however, it must be much more explored

Ossos, estruturas essenciais para a proteção de órgãos internos, estrutura corporal e suporte mecânico nos vertebrados, possuem uma complexa rede de canais (canais de Volkmann e Havers) responsáveis por nutrir as células do tecido. Entretanto a falta de estudos quantitativos leva a uma carência de medidas e parâmetros para a caracterização dessas estruturas. Utilizando computação gráfica, técnicas de processamento de imagens, e redes complexas descreveremos a obtenção, reconstrução, representação, e análise dessas redes de canais. Para isso, duas falanges distais, uma de um galo e uma de uma galinha, passaram por um processo de corte histológico, as lâminas resultantes foram fotografadas e as imagens tratadas até serem reconstruídas em 3D. Os volumes foram convertidos em redes complexas, o que permitiu o uso de métodos de análise consagrados pela literatura. As redes foram comparadas entre si e com a rede do trabalho desenvolvido por Matheus P. Viana et al. (1–3) usando análise de grau, posicionamento dos nós, detecção de comunidades, e ataques (em cascata e aleatórios). Três resultados se destacam: 1) as redes apresentam diviões predominantemente dicotômica dos canais; 2) as redes apresentam uma alta modularidade, indicando que áreas específicas desempenham funções específicas; e 3) as redes são particularmente resistentes a ataques em cascata. / Bones are essential for the protection of internal organs, for body structure, and for mechanical support in vertebrates, and present a complex network of channels (Havers and Volkmann channels) required to nourish tissue cells. However, the lack of quantitative studies leads to scarce parameters and measures to characterize these structures. By using computational graphic, image processing, and complex networks we will describe the acquisitation, reconstruction, representation, and analysis of these channel networks. Two distal phalanges (one from a hen and one from a rooster) were submitted to hystological section processing; the resulting slices were photographed and the images were treated before 3D reconstruction. The volumes were converted into complex networks which allow us to use methods of analysis widely accepted in literature. Networks were compared with each other and with the network obtained in the study by Viana et al. (1–3) using degree analysis, node positioning, community detection, and random and systematic attacks. Three results stand out: (i) the networks show a predominantly dichotomic division of channels; (ii) the networks show high modularity, indicating that specific areas perform specific functions; and (iii) the networks are particularly resistant to cascate attacks.

In this thesis we study the problem of multi-session dense rgb-d slam for 3D reconstruc- tion. Multi-session reconstruction can allow users to capture parts of an object that could not easily be captured in one session, due for instance to poor accessibility or user mistakes. We first present a thorough overview of single-session dense rgb-d slam and describe the multi-session problem as a loosening of the incremental camera movement and static scene assumptions commonly held in the single-session case. We then implement and evaluate sev- eral variations on a system for doing two-session reconstruction as an extension to a single- session dense rgb-d slam system. The extension from one to several sessions is divided into registering separate sessions into a single reference frame, re-optimizing the camera trajectories, and fusing together the data to generate a final 3D model. Registration is done by matching reconstructed models from the separate sessions using one of two adaptations on a 3D object detection pipeline. The registration pipelines are evaluated with many different sub-steps on a challenging dataset and it is found that robust registration can be achieved using the proposed methods on scenes without degenerate shape symmetry. In particular we find that using plane matches between two sessions as constraints for as much as possible of the registration pipeline improves results. Several different strategies for re-optimizing camera trajectories using data from both ses- sions are implemented and evaluated. The re-optimization strategies are based on re- tracking the camera poses from all sessions together, and then optionally optimizing over the full problem as represented on a pose-graph. The camera tracking is done by incrementally building and tracking against a tsdf volume, from which a final 3D mesh model is extracted. The whole system is qualitatively evaluated against a realistic dataset for multi-session re- construction. It is concluded that the overall approach is successful in reconstructing objects from several sessions, but that other fine grained registration methods would be required in order to achieve multi-session reconstructions that are indistinguishable from singe-session results in terms of reconstruction quality.

Les techniques de reconstruction 3d sont devenues incontournables pour des applicationstelles que la caractérisation et l'analyse de surfaces. Les travaux réalisés au coursde cette thèse ont pour objectif d'améliorer la qualité des reconstructions 3d par stéréophotométrie.Cette méthode repose sur deux principes, l'inversion d'un modèle d'interactionlumière/matière (BRDF) et la configuration d'un système d'éclairage et de prises de vues.Pour des surfaces diffuses, la stéréo-photométrie est réalisée à partir d'un minimum detrois images acquises d'un point de vue fixe pour des directions d'éclairages différentes.Son avantage est d'extraire simultanément les propriétés géométriques et colorimétriquesdes surfaces analysées même en cas de forte rugosité. Néanmoins, son application exige laformulation de plusieurs hypothèses qui sont difficilement respectables dans un contexteréel. Ceci génère des erreurs significatives dans les reconstructions. Pour les réduire, nousproposons différentes contributions qui s'articulent autour de la prise en compte globale de lachaine d'acquisition. Les apports de nos travaux se situent aux niveaux de la caractérisationet de la modélisation du système d'éclairage, du capteur d'acquisition et de l'améliorationde la qualité des images. Nous nous sommes aussi intéressés à l'optimisation des protocolesde prises de vues dans le cas de spécularité surfacique ou d'ombrage dus à la présence derugosité. Les résultats obtenus montrent que la prise en compte de ces caractéristiques dansl'inversion d'un modèle de BRDF permet une nette amélioration des reconstructions et offrela possibilité de réduire la taille des systèmes d'acquisition. / Tridimensional reconstruction method has become essential for applications suchas the characterization and analysis of surfaces. In this thesis, aims are to increase the qualityof 3d reconstructions by photometric stereo. This method is based on two principles, reversinglight-matter interaction model and configuration of a lighting system. With diffuse surfaces,the photometric stereo use three captured images from a fixed point of view for differentillumination directions. Its main advantage is to extract the color and geometric propertiesfor the textured rough surfaces. However, its application requires to make assumptions thatare not credible in real cases. This problem generates significant errors in the reconstructions.To reduce them, we offer various solutions around the overall consideration of the acquisitionchain. Our contribution focuses on the characterization and modeling of the lighting system,the acquisition sensor and improved image quality. We are also interested to optimize acquisitionprotocol in the case of specular surface or shading due to the surface geometry. Ourresults show that the inclusion of these features in the inversion of a BRDF model allowsan improvement of 3d reconstructions as well as the possibility of reducing the size of theacquisition systems.

Ce mémoire présente une nouvelle méthode pour l’acquisition 3D basée sur la lumière structurée. Cette méthode unifie les techniques de depth from focus (DFF) et depth from defocus (DFD) en utilisant une projection dynamique (dé)focalisée. Avec cette approche, le système d’acquisition d’images est construit de manière à conserver la totalité de l’objet nette sur toutes les images. Ainsi, seuls les motifs projetés sont soumis aux déformations de défocalisation en fonction de la profondeur de l’objet. Quand les motifs projetés ne sont pas focalisés, leurs Point Spread Function (PSF) sont assimilées à une distribution gaussienne. La profondeur finale est calculée en utilisant la relation entre les PSF de différents niveaux de flous et les variations de la profondeur de l’objet. Notre nouvelle estimation de la profondeur peut être utilisée indépendamment. Elle ne souffre pas de problèmes d’occultation ou de mise en correspondance. De plus, elle gère les surfaces sans texture et semi-réfléchissante. Les résultats expérimentaux sur des objets réels démontrent l’efficacité de notre approche, qui offre une estimation de la profondeur fiable et un temps de calcul réduit. La méthode utilise moins d’images que les approches DFF et contrairement aux approches DFD, elle assure que le PSF est localement unique / This paper presents a novel 3D recovery method based on structured light. This method unifies depth from focus (DFF) and depth from defocus (DFD) techniques with the use of a dynamic (de)focused projection. With this approach, the image acquisition system is specifically constructed to keep a whole object sharp in all of the captured images. Therefore, only the projected patterns experience different defocused deformations according to the object’s depths. When the projected patterns are out of focus, their Point Spread Function (PSF) is assumed to follow a Gaussian distribution. The final depth is computed by the analysis of the relationship between the sets of PSFs obtained from different blurs and the variation of the object’s depths. Our new depth estimation can be employed as a stand-alone strategy. It has no problem with occlusion and correspondence issues. Moreover, it handles textureless and partially reflective surfaces. The experimental results on real objects demonstrate the effective performance of our approach, providing reliable depth estimation and competitive time consumption. It uses fewer input images than DFF, and unlike DFD, it ensures that the PSF is locally unique.

The ability to automatically estimate the volume of timber is becoming increasingly important within the timber industry. The large number of timber trucks arriving each day at Swedish timber terminals fortifies the need for a volume estimation performed in real-time and on-the-go as the trucks arrive. This thesis investigates if a volumetric integration of disparity maps acquired from a Multi-View Stereo (MVS) system is a suitable approach for automatic volume estimation of timber loads. As real-time execution is preferred, efforts were made to provide a scalable method. The proposed method was quantitatively evaluated on datasets containing two geometric objects of known volume. A qualitative comparison to manual volume estimates of timber loads was also made on datasets recorded at a Swedish timber terminal. The proposed method is shown to be both accurate and precise under specific circumstances. However, robustness is poor to varying weather conditions, although a more thorough evaluation of this aspect needs to be performed. The method is also parallelizable, which means that future efforts can be made to significantly decrease execution time.