The idea that a complex object can be decomposed into simpler parts is fundamental to 3D design, so it is clearly desirable that digital representations of 3D shapes incorporate this part information. While solid modeling techniques based on set-theoretic volumetric composition intrinsically support hierarchical part-based shape descriptions, organic objects such as a human vertebra are more efficiently represented by surface modeling techniques. And although a human observer will easily identify part decompositions in surface models, the homogenous graphs of connected points and edges used in surface representations do not readily support explicit part decompositions. In this thesis, I will develop a part-based representation for 3D surface models. In abstract mathematics, a surface part can be represented as a deformation of a Riemannian manifold. To create a practical implementation, it is necessary to define representations of the 3D part shape and the region on the target surface where the part is to be placed. To represent the part region I will develop the Discrete Exponential Map (DEM), an algorithm which approximates the intrinsic normal coordinates on manifolds. To support arbitrary part shapes I will develop the COILS surface deformation, a robust geometric differential representation of point-sampled surfaces. Based on this part definition, I will then propose the Surface Tree, which makes possible the representation of complex shapes via a procedural, hierarchical composition of surface parts, analogous to the trees used in solid modeling. A major theme throughout the thesis is that part-based approaches have the potential to make surface design interfaces significantly more efficient and expressive. To explore this question and demonstrate the utility of my technical contributions, I present three novel modeling tools: an interactive texture design interface, a drag-and-drop mesh composition tool, and a sketch-based Surface Tree modeling environment. In addition to comparative algorithmic evaluations, and a consideration of representational capabilities, I have evaluated this body of work by publicly distributing my modeling tools. I will close the thesis with a discussion of the extensive feedback provided by users of my drag-and-drop mesh composition tool, called meshmixer. This feedback suggests that part-based approaches have significant benefits for surface modeling.

The idea that a complex object can be decomposed into simpler parts is fundamental to 3D design, so it is clearly desirable that digital representations of 3D shapes incorporate this part information. While solid modeling techniques based on set-theoretic volumetric composition intrinsically support hierarchical part-based shape descriptions, organic objects such as a human vertebra are more efficiently represented by surface modeling techniques. And although a human observer will easily identify part decompositions in surface models, the homogenous graphs of connected points and edges used in surface representations do not readily support explicit part decompositions. In this thesis, I will develop a part-based representation for 3D surface models. In abstract mathematics, a surface part can be represented as a deformation of a Riemannian manifold. To create a practical implementation, it is necessary to define representations of the 3D part shape and the region on the target surface where the part is to be placed. To represent the part region I will develop the Discrete Exponential Map (DEM), an algorithm which approximates the intrinsic normal coordinates on manifolds. To support arbitrary part shapes I will develop the COILS surface deformation, a robust geometric differential representation of point-sampled surfaces. Based on this part definition, I will then propose the Surface Tree, which makes possible the representation of complex shapes via a procedural, hierarchical composition of surface parts, analogous to the trees used in solid modeling. A major theme throughout the thesis is that part-based approaches have the potential to make surface design interfaces significantly more efficient and expressive. To explore this question and demonstrate the utility of my technical contributions, I present three novel modeling tools: an interactive texture design interface, a drag-and-drop mesh composition tool, and a sketch-based Surface Tree modeling environment. In addition to comparative algorithmic evaluations, and a consideration of representational capabilities, I have evaluated this body of work by publicly distributing my modeling tools. I will close the thesis with a discussion of the extensive feedback provided by users of my drag-and-drop mesh composition tool, called meshmixer. This feedback suggests that part-based approaches have significant benefits for surface modeling.

The plant leaf is one of the most challenging natural objects to be realistically depicted by computer graphics due to its complex morphological and optical characteristics. Although many studies have been done on plant modeling, previous research on leaf modeling required for close-up realistic plant images is very rare. In this thesis, a novel method for modeling of the leaf shape based on the leaf venation is presented. As the first step of the method, the leaf domain is defined by the enclosure of the leaf boundary. Second, the leaf venation is interactively modeled as a hierarchical skeleton based on the actual leaf image. Third, the leaf domain is triangulated with the skeleton as constraints. The skeleton is articulated with nodes on the skeleton. Fourth, the skeleton is interactively transformed to a specific shape. A user can manipulate the skeleton using two methods which are complementary to each other: one controls individual joints on the skeleton while the other controls the skeleton through an intermediate spline curve. Finally, the leaf blade shape is deformed to conform to the skeleton by interpolation. An interactive modeler was developed to help a user to model a leaf shape interactively and several leaves were modeled by the interactive modeler. The ray-traced rendering images demonstrate that the proposed method is effective in the leaf shape modeling.

The plant leaf is one of the most challenging natural objects to be realistically depicted by computer graphics due to its complex morphological and optical characteristics. Although many studies have been done on plant modeling, previous research on leaf modeling required for close-up realistic plant images is very rare. In this thesis, a novel method for modeling of the leaf shape based on the leaf venation is presented. As the first step of the method, the leaf domain is defined by the enclosure of the leaf boundary. Second, the leaf venation is interactively modeled as a hierarchical skeleton based on the actual leaf image. Third, the leaf domain is triangulated with the skeleton as constraints. The skeleton is articulated with nodes on the skeleton. Fourth, the skeleton is interactively transformed to a specific shape. A user can manipulate the skeleton using two methods which are complementary to each other: one controls individual joints on the skeleton while the other controls the skeleton through an intermediate spline curve. Finally, the leaf blade shape is deformed to conform to the skeleton by interpolation. An interactive modeler was developed to help a user to model a leaf shape interactively and several leaves were modeled by the interactive modeler. The ray-traced rendering images demonstrate that the proposed method is effective in the leaf shape modeling.

<p>The dissertation focuses on solving some important theoretical and methodological problems associated with Bayesian modeling of infinite dimensional `objects', popularly called nonparametric Bayes. The term `infinite dimensional object' can refer to a density, a conditional density, a regression surface or even a manifold. Although Bayesian density estimation as well as function estimation are well-justified in the existing literature, there has been little or no theory justifying the estimation of more complex objects (e.g. conditional density, manifold, etc.). Part of this dissertation focuses on exploring the structure of the spaces on which the priors for conditional densities and manifolds are supported while studying how the posterior concentrates as increasing amounts of data are collected.</p><p>With the advent of new acquisition devices, there has been a need to model complex objects associated with complex data-types e.g. millions of genes affecting a bio-marker, 2D pixelated images, a cloud of points in the 3D space, etc. A significant portion of this dissertation has been devoted to developing adaptive nonparametric Bayes approaches for learning low-dimensional structures underlying higher-dimensional objects e.g. a high-dimensional regression function supported on a lower dimensional space, closed curves representing the boundaries of shapes in 2D images and closed surfaces located on or near the point cloud data. Characterizing the distribution of these objects has a tremendous impact in several application areas ranging from tumor tracking for targeted radiation therapy, to classifying cells in the brain, to model based methods for 3D animation and so on. </p><p> </p><p> The first three chapters are devoted to Bayesian nonparametric theory and modeling in unconstrained Euclidean spaces e.g. mean regression and density regression, the next two focus on Bayesian modeling of manifolds e.g. closed curves and surfaces, and the final one on nonparametric Bayes spatial point pattern data modeling when the sampling locations are informative of the outcomes.</p> / Dissertation

Quasi-articulated objects, such as human beings, are among the most commonly seen objects in our daily lives. Extensive research have been dedicated to 3D shape reconstruction and motion analysis for this type of objects for decades. A major motivation is their wide applications, such as in entertainment, surveillance and health care. Most of existing studies relied on one or more regular video cameras. In recent years, commodity depth sensors have become more and more widely available. The geometric measurements delivered by the depth sensors provide significantly valuable information for these tasks. In this dissertation, we propose three algorithms for monocular pose estimation and shape reconstruction of quasi-articulated objects using a single commodity depth sensor. These three algorithms achieve shape reconstruction with increasing levels of granularity and personalization. We then further develop a method for highly detailed shape reconstruction based on our pose estimation techniques. Our first algorithm takes advantage of a motion database acquired with an active marker-based motion capture system. This method combines pose detection through nearest neighbor search with pose refinement via non-rigid point cloud registration. It is capable of accommodating different body sizes and achieves more than twice higher accuracy compared to a previous state of the art on a publicly available dataset. The above algorithm performs frame by frame estimation and therefore is less prone to tracking failure. Nonetheless, it does not guarantee temporal consistent of the both the skeletal structure and the shape and could be problematic for some applications. To address this problem, we develop a real-time model-based approach for quasi-articulated pose and 3D shape estimation based on Iterative Closest Point (ICP) principal with several novel constraints that are critical for monocular scenario. In this algorithm, we further propose a novel method for automatic body size estimation that enables its capability to accommodate different subjects. Due to the local search nature, the ICP-based method could be trapped to local minima in the case of some complex and fast motions. To address this issue, we explore the potential of using statistical model for soft point correspondences association. Towards this end, we propose a unified framework based on Gaussian Mixture Model for joint pose and shape estimation of quasi-articulated objects. This method achieves state-of-the-art performance on various publicly available datasets. Based on our pose estimation techniques, we then develop a novel framework that achieves highly detailed shape reconstruction by only requiring the user to move naturally in front of a single depth sensor. Our experiments demonstrate reconstructed shapes with rich geometric details for various subjects with different apparels. Last but not the least, we explore the applicability of our method on two real-world applications. First of all, we combine our ICP-base method with cloth simulation techniques for Virtual Try-on. Our system delivers the first promising 3D-based virtual clothing system. Secondly, we explore the possibility to extend our pose estimation algorithms to assist physical therapist to identify their patients’ movement dysfunctions that are related to injuries. Our preliminary experiments have demonstrated promising results by comparison with the gold standard active marker-based commercial system. Throughout the dissertation, we develop various state-of-the-art algorithms for pose estimation and shape reconstruction of quasi-articulated objects by leveraging the geometric information from depth sensors. We also demonstrate their great potentials for different real-world applications.

Nous présentons dans cette thèse un nouveau modèle statistique de forme et l'utilisons pour la segmentation d'images avec a priori. Ce modèle est représenté par un champ de Markov. Les noeuds du graphe correspondent aux points de contrôle situés sur le contour de la forme géométrique, et les arêtes du graphe représentent les dépendances entre les points de contrôle. La structure du champ de Markov est déterminée à partir d'un ensemble de formes, en utilisant des techniques d'apprentissage de variétés et de groupement non-supervisé. Les contraintes entre les points sont assurées par l'estimation des fonctions de densité de probabilité des longueurs de cordes normalisées. Dans une deuxième étape, nous construisons un algorithme de segmentation qui intègre le modèle statistique de forme, et qui le relie à l'image grâce à un terme région, à travers l'utilisation de diagrammes de Voronoi. Dans cette approche, un contour de forme déformable évolue vers l'objet à segmenter. Nous formulons aussi un algorithme de segmentation basé sur des détecteurs de points d'intérêt, où le terme de régularisation est lié à l'apriori de forme. Dans ce cas, on cherche à faire correspondre le modèle aux meilleurs points candidats extraits de l'image par le détecteur. L'optimisation pour les deux algorithmes est faite en utilisant des méthodes récentes et efficaces. Nous validons notre approche à travers plusieurs jeux de données en 2D et en 3D, pour des applications de vision par ordinateur ainsi que l'analyse d'images médicales. / In this thesis, we introduce a new statistical shape model and use it for knowledge-based image segmentation. The model is represented by a Markov Random Field (MRF). The vertices of the graph correspond to landmarks lying on the shape boundary, whereas the edges of the graph encode the dependencies between the landmarks. The MRF structure is determined from a training set of shapes using manifold learning and unsupervised clustering techniques. The inter-point constraints are enforced using the learnedprobability distribution function of the normalized chord lengths.This model is used as a basis for knowledge-based segmentation. We adopt two approaches to incorporate the data support: one is based on landmark correspondences and the other one uses image region information. In the first case, correspondences between the model and the image are obtained through detectors and the optimal configuration is achieved through combination of detector responses and prior knowledge. The second approach consists of minimizing an energy that discriminates the object from the background while accounting for the shape prior. A Voronoi decomposition is used to express this objective function in a distributed manner using the landmarks of the model. Both algorithms are optimized using state-of-the art eficient optimization methods. We validate our approach on various 2D and 3D datasets of images, for computer vision applications as well as medical image analysis.

An automated procedure was developed to identify and extract confined poultry facilities from color 35-mm slide imagery collected by the United States Department of Agriculture/Farm Service Agency (USDA/FSA). The imagery is used by the USDA/FSA to monitor compliance with various farm support programs and to determine crop production acreage within a given county. The imagery is generally available for all counties within the state on an annual basis. The imagery, however, is not flown to rigid specifications as flight height, direction, and overlap can vary significantly. The USDA/FSA attempts to collect imagery with reasonably clear skies, as visual interpretations could be drastically impacted by cloudiness. The goal of this study was to develop procedures to effectively utilize this imagery base to identify and extract poultry facilities using automated techniques based on image processing and GIS. The procedure involved pre-screening the slides to determine coverage, geopositioning to USGS quadrangle base, color scanning to convert slide image to a digital format and archiving each data file with a naming convention that would allow rapid retrieval in later analysis. Image processing techniques were developed for identifying poultry facilities based on spectral characteristics. GIS tools were used to select poultry facilities from an array of features with similar spectral characteristics. A training data set was selected from which the spectral characteristics of poultry facilities were analyzed and compared with background conditions. Poultry facilities were found to have distinguishable characteristics. Descriptive statistics were used to define the range of spectral characteristics encompassing poultry facilities. Thresholding analyses were then utilized to eliminate all image features with spectral characteristics outside of this range. Additional analyses were made to remove noise in the spectral image due to the sun angle, line of sight of camera, variation in roof reflectance due to rust and/or aging, shading by trees, etc. A primary objective in these analyses was to enhance the spectral characteristics for the poultry facility while, at the same time, retaining physical characteristics, i.e. the spectral characteristic is represented by a single blue color with a high brightness value. The techniques developed to achieve a single blue color involved the use of Principal Component Analysis (PCA) on the red color band followed by RGB to Hue and RGB to Saturation analyses on the red and green color bands, respectively, from the resulting image. The features remaining from this series of analyses were converted into polygons (shape file) using ArcView GIS, which was then used to calculate the area and perimeter of each polygon. The parameters utilized to describe the shape of a poultry house included width, length, compactness, length-width ratio, and polygon centroid analysis. Poultry facilities were found to have an average width of approximately 12.6m with a low standard deviation indicating that the widths of all houses were very similar. The length of poultry facilities ranged from 63m to 261m with and average length of 149m. The compactness parameter, which also is related to length and width, ranged from 30 to 130 with a mean value of approximately 57. The shape parameters were used by ArcView GIS to identify polygons that represent poultry facilities. The order of selection was found to be compactness followed by length-width ratio and polygon centroid analysis. A data set that included thirty 35-mm slide images randomly selected from the Rockingham County data set, which contained over 2000 slides, was used to evaluate the automated procedure. The slides contained 182 poultry houses previously identified through manual procedures. Seven facilities were missed and 175 were correctly identified. Ninety-seven percent (97%) of existing poultry facilities were correctly identified which compares favorably with the 97 % accuracy resulted by manual procedures. . The manual procedure described by Mostaghimi, et. al.(1999) only gave the center coordinates for each poultry facility. The automated procedure not only gives the center coordinate for each poultry building but also gives estimates for geometric parameters area, length and width along with an estimate of the capacity of building (i.e. number of birds), and waste load generated by birds including nutrient and bacteria content. The nutrient and bacteria load generated by each poultry facility is important information for conducting TMDL studies currently being developed for impaired Virginia streams. The information is expected to be very helpful to consultants and state agencies conducting the studies. Agricultural support agencies such as USDA/NRCS and USDA/FSA, Extension Service, consultants, etc. will find the information very helpful in the development of implementation plans designed to meet TMDL target water quality goals. The data also should be useful to Water Authorities for selection of appropriate treatment of water supplies and to county and local government jurisdictions for developing policies to minimize the degradation of water supplies. / Ph. D.

In order to meet the needs of people with hearing loss today hearing aids are custom designed. Increasingly accurate 3-D scanning technology has contributed to the transition from conventional production scenarios to software based processes. Nonetheless, there is a tremendous amount of manual work involved to transform an input 3-D surface mesh of the outer ear into a final hearing aid shape. This manual work is often cumbersome and requires lots of experience which is why automatic solutions are of high practical relevance. This work is concerned with the recognition of 3-D surface meshes of ear implants. In particular we present a semantic part-labeling framework which significantly outperforms existing approaches for this task. We make at least three contributions which may also be found useful for other classes of 3-D meshes. Firstly, we validate the discriminative performance of several local descriptors and show that the majority of them performs poorly on our data except for 3-D shape contexts. The reason for this is that many local descriptor schemas are not rich enough to capture subtle variations in form of bends which is typical for organic shapes. Secondly, based on the observation that the left and the right outer ear of an individual look very similar we raised the question how similar the ear shapes among arbitrary individuals are? In this work, we define a notion of distance between ear shapes as building block of a non-parametric shape model of the ear to better handle the anatomical variability in ear implant labeling. Thirdly, we introduce a conditional random field model with a variety of label priors to facilitate the semantic part-labeling of 3-D meshes of ear implants. In particular we introduce the concept of a global parametric transition prior to enforce transition boundaries between adjacent object parts with an a priori known parametric form. In this way we were able to overcome the issue of inadequate geometric cues (e.g., ridges, bumps, concavities) as natural indicators for the presence of part boundaries. The last part of this work offers an outlook to possible extensions of our methods, in particular the development of 3-D descriptors that are fast to compute whilst at the same time rich enough to capture the characteristic differences between objects residing in the same class.

In order to meet the needs of people with hearing loss today hearing aids are custom designed. Increasingly accurate 3-D scanning technology has contributed to the transition from conventional production scenarios to software based processes. Nonetheless, there is a tremendous amount of manual work involved to transform an input 3-D surface mesh of the outer ear into a final hearing aid shape. This manual work is often cumbersome and requires lots of experience which is why automatic solutions are of high practical relevance. This work is concerned with the recognition of 3-D surface meshes of ear implants. In particular we present a semantic part-labeling framework which significantly outperforms existing approaches for this task. We make at least three contributions which may also be found useful for other classes of 3-D meshes. Firstly, we validate the discriminative performance of several local descriptors and show that the majority of them performs poorly on our data except for 3-D shape contexts. The reason for this is that many local descriptor schemas are not rich enough to capture subtle variations in form of bends which is typical for organic shapes. Secondly, based on the observation that the left and the right outer ear of an individual look very similar we raised the question how similar the ear shapes among arbitrary individuals are? In this work, we define a notion of distance between ear shapes as building block of a non-parametric shape model of the ear to better handle the anatomical variability in ear implant labeling. Thirdly, we introduce a conditional random field model with a variety of label priors to facilitate the semantic part-labeling of 3-D meshes of ear implants. In particular we introduce the concept of a global parametric transition prior to enforce transition boundaries between adjacent object parts with an a priori known parametric form. In this way we were able to overcome the issue of inadequate geometric cues (e.g., ridges, bumps, concavities) as natural indicators for the presence of part boundaries. The last part of this work offers an outlook to possible extensions of our methods, in particular the development of 3-D descriptors that are fast to compute whilst at the same time rich enough to capture the characteristic differences between objects residing in the same class.