This research examines an approach for capturing 3D surface and structural data of moving articulated creatures. Given the task of non-invasively and automatically capturing such data, a methodology and the associated experiments are presented, that apply to multiview videos of the subjects motion. Our thesis states: A functional structure and the timevarying surface of an articulated creature subject are contained in a sequence of its 3D data. A functional structure is one example of the possible arrangements of internal mechanisms (kinematic joints, springs, etc.) that is capable of performing the motions observed in the input data. Volumetric structures are frequently used as shape descriptors for 3D data. The capture of such data is being facilitated by developments in multi-view video and range scanning, extending to subjects that are alive and moving. In this research, we examine vision-based modeling and the related representation of moving articulated creatures using Spines. We define a Spine as a branching axial structure representing the shape and topology of a 3D objects limbs, and capturing the limbs correspondence and motion over time. The Spine concept builds on skeletal representations often used to describe the internal structure of an articulated object and the significant protrusions. Our representation of a Spine provides for enhancements over a 3D skeleton. These enhancements form temporally consistent limb hierarchies that contain correspondence information about real motion data. We present a practical implementation that approximates a Spines joint probability function to reconstruct Spines for synthetic and real subjects that move. In general, our approach combines the objectives of generalized cylinders, 3D scanning, and markerless motion capture to generate baseline models from real puppets, animals, and human subjects.

Técnicas de modelagem baseada em imagens têm recebido considerável atenção da comunidade de visualização computacional devido ao potencial de criar cenas realistas a partir de um pequeno conjunto de imagens bi-dimensionais. Entretanto, a qualidade dos modelos gerados pelas ferramentas atualmente disponíveis é extremamente dependente de entradas fornecidas pelo usuário. Este trabalho propõe a execução do projeto de uma ferramenta de auxílio para sistemas de modelagem baseada em imagens que utiliza o conceito de contornos ativos para aumentar o grau de automação do processo de localização do contorno do objeto presente na fotografia, que servirá de guia para a posterior localização dos vértices desse objeto. Através desta abordagem, figuras geométricas mais simples, como pirâmides e hexaedros, puderam ser reconstruídas após a recuperação das coordenadas de seus vértices / Image Based Modelling techniques has received considerable attention from the computer vision community due to the potential to create realistic scenes from some bi-dimensional images. However, the model?s quality generated by the tools available nowadays is extremely dependent on entries provided by the user. This work proposes the execution of a help tool project for image based modelling systems that uses the active contours concept to increase the process automation degree of locating the contour of an object in the image, which will guide the vertex location process of this object. Through this approach, simple geometric figures, as pyramids and squares, could be reconstructed after the vertex coordinates recuperation

Técnicas de modelagem baseada em imagens têm recebido considerável atenção da comunidade de visualização computacional devido ao potencial de criar cenas realistas a partir de um pequeno conjunto de imagens bi-dimensionais. Entretanto, a qualidade dos modelos gerados pelas ferramentas atualmente disponíveis é extremamente dependente de entradas fornecidas pelo usuário. Este trabalho propõe a execução do projeto de uma ferramenta de auxílio para sistemas de modelagem baseada em imagens que utiliza o conceito de contornos ativos para aumentar o grau de automação do processo de localização do contorno do objeto presente na fotografia, que servirá de guia para a posterior localização dos vértices desse objeto. Através desta abordagem, figuras geométricas mais simples, como pirâmides e hexaedros, puderam ser reconstruídas após a recuperação das coordenadas de seus vértices / Image Based Modelling techniques has received considerable attention from the computer vision community due to the potential to create realistic scenes from some bi-dimensional images. However, the model?s quality generated by the tools available nowadays is extremely dependent on entries provided by the user. This work proposes the execution of a help tool project for image based modelling systems that uses the active contours concept to increase the process automation degree of locating the contour of an object in the image, which will guide the vertex location process of this object. Through this approach, simple geometric figures, as pyramids and squares, could be reconstructed after the vertex coordinates recuperation

Cardiovascular modeling has the capability to provide valuable information allowing clinicians to better classify patients and aid in surgical planning. Modeling is advantageous for being non-invasive, and also allows for quantification of values not easily obtained from physical measurements. Hemodynamics are heavily dependent on vessel geometry, which varies greatly from patient to patient. For this reason, clinically relevant approaches must perform these simulations on patient-specific geometry. Geometry is acquired from various imaging modalities, including magnetic resonance imaging, computed tomography, and ultrasound. The typical approach for generating a computational model requires construction of a triangulated surface mesh for use with finite volume or finite element solvers. Surface mesh construction can result in a loss of anatomical features and often requires a skilled user to execute manual steps in 3rd party software. An alternative to this method is to use a Cartesian grid solver to conduct the fluid simulation. Cartesian grid solvers do not require a surface mesh. They can use the implicit geometry representation created during the image segmentation process, but they are constrained to a cuboidal domain. Since patient-specific geometry usually deviate from the orthogonal directions of a cuboidal domain, flow extensions are often implemented. Flow extensions are created via a skilled user and 3rd party software, rendering the Cartesian grid solver approach no more clinically useful than the triangulated surface mesh approach. This work presents an alternative to flow extensions by developing a method of applying vessel inlet and outlet boundary conditions to regions inside the Cartesian domain.

This thesis is devoted to 2D and 3D modeling of urban environments using structured representations and grammars. Our approach introduces a semantic representation for buildings that encodes expected architectural constraints and is able to derive complex instances using fairly simple grammars. Furthermore, we propose two novel inference algorithms to parse images using such grammars. To this end, a steepest ascent hill climbing concept is considered to derive the grammar and the corresponding parameters from a single facade view. It combines the grammar constraints with the expected visual properties of the different architectural elements. Towards addressing more complex scenarios and incorporating 3D information, a second inference strategy based on evolutionary computational algorithms is adopted to optimize a two-component objective function introducing depth cues. The proposed framework was evaluated qualitatively and quantitatively on a benchmark of annotated facades, demonstrating robustness to challenging situations. Substantial improvement due to the strong grammatical context was shown in comparison to the performance of the same appearance models coupled with local priors. Therefore, our approach provides powerful techniques in response to increasing demand on large scale 3D modeling of real environments through compact, structured and semantic representations, while opening new perspectives for image understanding

Abstract Automatic three-dimensional scene reconstruction from multiple images is a central problem in geometric computer vision. This thesis considers topics that are related to this problem area. New models and methods are presented for various tasks in such specific domains as camera calibration, image-based modeling and image matching. In particular, the main themes of the thesis are geometric camera calibration and quasi-dense image matching. In addition, a topic related to the estimation of two-view geometric relations is studied, namely, the computation of a planar homography from corresponding conics. Further, as an example of a reconstruction system, a structure-from-motion approach is presented for modeling sewer pipes from video sequences. In geometric camera calibration, the thesis concentrates on central cameras. A generic camera model and a plane-based camera calibration method are presented. The experiments with various real cameras show that the proposed calibration approach is applicable for conventional perspective cameras as well as for many omnidirectional cameras, such as fish-eye lens cameras. In addition, a method is presented for the self-calibration of radially symmetric central cameras from two-view point correspondences. In image matching, the thesis proposes a method for obtaining quasi-dense pixel matches between two wide baseline images. The method extends the match propagation algorithm to the wide baseline setting by using an affine model for the local geometric transformations between the images. Further, two adaptive propagation strategies are presented, where local texture properties are used for adjusting the local transformation estimates during the propagation. These extensions make the quasi-dense approach applicable for both rigid and non-rigid wide baseline matching. In this thesis, quasi-dense matching is additionally applied for piecewise image registration problems which are encountered in specific object recognition and motion segmentation. The proposed object recognition approach is based on grouping the quasi-dense matches between the model and test images into geometrically consistent groups, which are supposed to represent individual objects, whereafter the number and quality of grouped matches are used as recognition criteria. Finally, the proposed approach for dense two-view motion segmentation is built on a layer-based segmentation framework which utilizes grouped quasi-dense matches for initializing the motion layers, and is applicable under wide baseline conditions.

In this work, we have developed user interactive capabilities that allow us to perform segmentation and manipulation of patient-specific geometries required for Computational Fluid Dynamics (CFD) studies, entirely in image domain and within a single platform of ‘IAFEMesh'. Within this toolkit we have added commonly required manipulation capabilities for performing CFD on segmented objects by utilizing libraries like ITK, VTK and KWWidgets. With the advent of these capabilities we can now manipulate a single patient specific image into a set of possible cases we seek to study; which is difficult to do in commercially available software like VMTK, Slicer, MITK etc. due to their limited manipulation capabilities. Levelset representation of the manipulated geometries can be simulated in our flow solver (SCIMITAR-3D) without creating any surface or volumetric mesh. This image-levelset-flow framework offers few advantages. 1) We don't need to deal with the problems associated with mesh quality, edge connectivity related to mesh models, 2) and manipulations like boolean operation result in smooth, physically realizable entities which is challanging in mesh domain. We have validated our image-levelset-flow setup with the known results from previous studies. We have modified the algorithm by Krissian et al. and implemented it for the segmentation of Type-A aortic dissection. Finally, we implemented these capabilities to study the hemodynamics in Type-A aortic dissection. Our image based framework is a first of its kind and the hemodynamic study of Type-A dissection too is first study onto the best of our knowledge.

L'objectif principal de cette thèse est de développer des outils pour la reconstruction de l'environnement urbain à partir d'images. Les entrées typiques de notre travail est un ensemble d'images de façades, des empreintes au sol de bâtiments, et des modèles 3D reconstruits à partir d'images aériennes. Les principales étapes comprennent le calibrage des images,le recalage avec le modèle 3D, la récupération des informations de profondeur ainsi que la sémantique des façades.Pour atteindre cet objectif, nous utilisons des techniques du domaine de vision par ordinateur, reconnaissance de formes et de l'informatique graphique. Les contributions de notre approche sont présentés en deux parties.Dans la première partie, nous nous sommes concentrés sur des techniques de reconstruction multi-vues dans le but de récupérer automatiquement les informations de profondeur de façades à partir un ensemble des photographies non calibrées. Tout d'abord, nous utilisons la technique structure et mouvement pour calibrer automatiquement l'ensemble des photographies. Ensuite, nous proposons des techniques pour le recalage de la reconstruction avec un modèle 3D. Enfin, nous proposons des techniques de reconstruction 3d dense (stéréo multi-vues et voxel coloring) pour produire un maillage 3D texturé d'une scène d'un ensemble d'images calibrées.La deuxième partie est consacrée à la reconstruction à partir d'une seule vue et son objectif est de récupérer la structure sémantique d'une façade d'une image ortho-rectifiée. La nouveauté de cette approche est l'utilisation d'une grammaire stochastique décrivant un style architectural comme modèle pour la reconstruction de façades. nous combinons un ensemble de détecteurs image avec une méthode d'optimisation globale stochastique en utilisant l'algorithme Metropolis-Hastings. / The main goal of this thesis is to develop innovative and practicaltools for the reconstruction of buildings from images. The typical input to our workis a set of facade images, building footprints, and coarse 3d models reconstructedfrom aerial images. The main steps include the calibration of the photographs,the registration with the coarse 3d model, the recovery of depth and sematicinformation, and the refinement of the coarse 3d model.To achieve this goal, we use computer vision, pattern recognition and computergraphics techniques. Contributions in this approach are presented on two parts.In the first part, we focused on multiple view reconstruction techniques withthe aim to automatically recover the depth information of facades from a setof uncalibrated photographs. First, we use structure from motion techniques toautomatically calibrate the set of photographs. Then, we propose techniques for theregistration of the sparse reconstruction to a coarse 3d model. Finally, we proposean accelerated multi-view stereo and voxel coloring framework using graphicshardware to produce a textured 3d mesh of a scene from a set of calibrated images.The second part is dedicated to single view reconstruction and its aim is to recoverthe semantic structure of a facade from an ortho-rectified image. The novelty ofthis approach is the use of a stochastic grammar describing an architectural style asa model for facade reconstruction. we combine bottom-up detection with top-downproposals to optimize the facade structure using the Metropolis-Hastings algorithm.

Patient-specific multi-physics simulations have the potential to improve the diagnosis, treatment, and scientific inquiry of heart valve dynamics. It has been shown that the flow characteristics within the left ventricle are important to correctly capture the aortic and mitral valve motion and corresponding fluid dynamics, motivating the use of patient-specific imaging to describe the aortic and mitral valve geometries as well as the motion of the left ventricle (LV). The LV position can be captured at several time points in the cardiac cycle, such that its motion can be prescribed a priori as a Dirichlet boundary condition during a simulation. Valve leaflet motion, however, should be computed from soft-tissue models and incorporated using fully-coupled Fluid Structure Interaction (FSI) algorithms. While FSI simulations have in part or wholly been achieved by multiple groups, to date, no high-throughput models have been developed, which are needed for use in a clinical environment. This project seeks to enable patient-derived moving LV boundary conditions, and has been developed for use with a previously developed immersed boundary, fixed Cartesian grid FSI framework. One challenge in specifying LV motion from medical images stems from the low temporal resolution available. Typical imaging modalities contain only tens of images during the cardiac cycle to describe the change in position of the left ventricle. This temporal resolution is significantly lower than the time resolution needed to capture fluid dynamics of a highly deforming heart valve, and thus an approach to describe intermediate positions of the LV is necessary. Here, we propose a primarily Eulerian means of representing LV displacement. This is a natural extension, since an Eulerian framework is employed in the CFD model to describe the large displacement of the heart valve leaflets. This approach to using Eulerian interface representation is accomplished by applying “morphing” techniques commonly used in the field of computer graphics. For the approach developed in the current work, morphing is adapted to the unique characteristics of a Cartesian grid flow solver which presents challenges of adaptive mesh refinement, narrow band approach, parallel domain decomposition, and the need to supply a local surface velocity to the flow solver that describes both normal and tangential motion. This is accomplished by first generating a skeleton from the Eulerian interface representation, and deforming the skeleton between image frames to determine bulk displacement. After supplying bulk displacement, local displacement is determined using the Eulerian fields. The skeletons are also utilized to automate the simulation setup to track the locations upstream and downstream where the system inflow/outflow boundary conditions are to be applied, which in the current approach, are not limited to Cartesian domain boundaries.

La présente thèse est consacrée à la modélisation 2D et 3D d’environnements urbains à l’aide de représentations structurées et de grammaires de formes. Notre approche consiste à introduire une représentation sémantique de bâtiments, qui encode les contraintes architecturales attendues, et qui soit capable de traiter des exemples complexes en utilisant des grammaires très simples. En outre, nous proposons deux nouveaux algorithmes d’inférence permettant l’analyse grammaticale d’images en utilisant ces grammaires. En premier lieu, un algorithme dit de hill climbing permet d’extraire les règles de grammaire et les paramètres correspondants à partir d’une vue unique d’une façade. Ce concept combine astucieusement les contraintes grammaticales et les propriétés visuelles attendues pour les différents éléments architecturaux. Cependant, afin de pouvoir traiter de cas plus complexes et également d’incorporer de l’information 3D, une deuxième stratégie d’inférence basée sur des algorithmes évolutionnaires a été adoptée pour optimiser un fonction à deux objectifs qui introduit notamment des notions de profondeur. Le système proposé a été évalué tant qualitativement que quantitativement sur un panel de façades de référence toute munies d’annotations, démontrant ainsi sa robustesse face à des situations d’abords difficiles. Grâce à la force du contexte grammatical, des améliorations substantielles ont été démontrées par rapport aux performances des mêmes modèles couplés à des a priori uniquement locaux. Par conséquent, notre approche fournit des outils puissants permettant de faire face à la demande croissante en modélisation 3D d’environnements réels à large échelle, grâce à des représentations sémantiques compactes et structurées. Ce travail ouvre par ailleurs un vaste champ de perspectives pour le domaine de l’interprétation d’images / This thesis is devoted to 2D and 3D modeling of urban environments using structured representations and grammars. Our approach introduces a semantic representation for buildings that encodes expected architectural constraints and is able to derive complex instances using fairly simple grammars. Furthermore, we propose two novel inference algorithms to parse images using such grammars. To this end, a steepest ascent hill climbing concept is considered to derive the grammar and the corresponding parameters from a single facade view. It combines the grammar constraints with the expected visual properties of the different architectural elements. Towards addressing more complex scenarios and incorporating 3D information, a second inference strategy based on evolutionary computational algorithms is adopted to optimize a two-component objective function introducing depth cues. The proposed framework was evaluated qualitatively and quantitatively on a benchmark of annotated facades, demonstrating robustness to challenging situations. Substantial improvement due to the strong grammatical context was shown in comparison to the performance of the same appearance models coupled with local priors. Therefore, our approach provides powerful techniques in response to increasing demand on large scale 3D modeling of real environments through compact, structured and semantic representations, while opening new perspectives for image understanding