The haptic exploration of 2-D images is a challenging problem in computer haptics. Research on the topic has primarily been focused on the exploration of maps and curves. This thesis describes the design and implementation of a system for the haptic exploration of photographs. The system builds on various research directions related to assistive technology, computer haptics, and image segmentation. An object-level segmentation hierarchy is generated from the source photograph to be rendered haptically as a contour image at multiple levels-of-detail. A tool for the authoring of object-level hierarchies was developed, as well as an innovative type of user interaction by region selection for accurate and efficient image segmentation. According to an objective benchmark measuring how the new method compares with other interactive image segmentation algorithms shows that our region selection interaction is a viable alternative to marker-based interaction. The hierarchy authoring tool combined with precise algorithms for image segmentation can build contour images of the quality necessary for the images to be understood by touch with our system. The system was evaluated with a user study of 24 sighted participants divided in different groups. The first part of the study had participants explore images using haptics and answer questions about them. The second part of the study asked the participants to identify images visually after haptic exploration. Results show that using a segmentation hierarchy supporting multiple levels-of-detail of the same image is beneficial to haptic exploration. As the system gains maturity, it is our goal to make it available to blind users.

The haptic exploration of 2-D images is a challenging problem in computer haptics. Research on the topic has primarily been focused on the exploration of maps and curves. This thesis describes the design and implementation of a system for the haptic exploration of photographs. The system builds on various research directions related to assistive technology, computer haptics, and image segmentation. An object-level segmentation hierarchy is generated from the source photograph to be rendered haptically as a contour image at multiple levels-of-detail. A tool for the authoring of object-level hierarchies was developed, as well as an innovative type of user interaction by region selection for accurate and efficient image segmentation. According to an objective benchmark measuring how the new method compares with other interactive image segmentation algorithms shows that our region selection interaction is a viable alternative to marker-based interaction. The hierarchy authoring tool combined with precise algorithms for image segmentation can build contour images of the quality necessary for the images to be understood by touch with our system. The system was evaluated with a user study of 24 sighted participants divided in different groups. The first part of the study had participants explore images using haptics and answer questions about them. The second part of the study asked the participants to identify images visually after haptic exploration. Results show that using a segmentation hierarchy supporting multiple levels-of-detail of the same image is beneficial to haptic exploration. As the system gains maturity, it is our goal to make it available to blind users.

The haptic exploration of 2-D images is a challenging problem in computer haptics. Research on the topic has primarily been focused on the exploration of maps and curves. This thesis describes the design and implementation of a system for the haptic exploration of photographs. The system builds on various research directions related to assistive technology, computer haptics, and image segmentation. An object-level segmentation hierarchy is generated from the source photograph to be rendered haptically as a contour image at multiple levels-of-detail. A tool for the authoring of object-level hierarchies was developed, as well as an innovative type of user interaction by region selection for accurate and efficient image segmentation. According to an objective benchmark measuring how the new method compares with other interactive image segmentation algorithms shows that our region selection interaction is a viable alternative to marker-based interaction. The hierarchy authoring tool combined with precise algorithms for image segmentation can build contour images of the quality necessary for the images to be understood by touch with our system. The system was evaluated with a user study of 24 sighted participants divided in different groups. The first part of the study had participants explore images using haptics and answer questions about them. The second part of the study asked the participants to identify images visually after haptic exploration. Results show that using a segmentation hierarchy supporting multiple levels-of-detail of the same image is beneficial to haptic exploration. As the system gains maturity, it is our goal to make it available to blind users.

The haptic exploration of 2-D images is a challenging problem in computer haptics. Research on the topic has primarily been focused on the exploration of maps and curves. This thesis describes the design and implementation of a system for the haptic exploration of photographs. The system builds on various research directions related to assistive technology, computer haptics, and image segmentation. An object-level segmentation hierarchy is generated from the source photograph to be rendered haptically as a contour image at multiple levels-of-detail. A tool for the authoring of object-level hierarchies was developed, as well as an innovative type of user interaction by region selection for accurate and efficient image segmentation. According to an objective benchmark measuring how the new method compares with other interactive image segmentation algorithms shows that our region selection interaction is a viable alternative to marker-based interaction. The hierarchy authoring tool combined with precise algorithms for image segmentation can build contour images of the quality necessary for the images to be understood by touch with our system. The system was evaluated with a user study of 24 sighted participants divided in different groups. The first part of the study had participants explore images using haptics and answer questions about them. The second part of the study asked the participants to identify images visually after haptic exploration. Results show that using a segmentation hierarchy supporting multiple levels-of-detail of the same image is beneficial to haptic exploration. As the system gains maturity, it is our goal to make it available to blind users.

This study assessed the utility of an object-oriented approach to deciduous and coniferous forest volume and above ground biomass estimation, based solely on small-footprint, multiple return lidar data. The study area is located in Appomattox Buckingham State Forest in the Piedmont physiographic province of Virginia, U.S.A, at 78°41’ W, 37°25’ N. Vegetation is composed of various coniferous, deciduous, and mixed forest stands. The eCognition segmentation algorithm was used to derive objects from a lidar-based canopy height model (CHM). New segment selection criteria, based on between- and within-segment CHM variance, and average field plot size, were developed. Horizontal point samples were used to measure in-field volume and biomass, for 2-class (deciduous-coniferous) and 3-class (deciduous-coniferous-mixed) forest schemes. Per-segment lidar distributional parameters, e.g., mean, range, and percentiles, were extracted from the lidar data and used as input to volume and biomass regression analysis. Discriminant classification was performed using lidar point height and CHM distributions. There was no evident difference between the two-class and three-class approaches, based on similar adjusted R2 values. Two-class forest definition was preferred due to its simplicity. Two-class adjusted R2 and root mean square error (RMSE) values for deciduous volume (0.59; 51.15 m3/ha) and biomass (0.58; 37.41 Mg/ha) were improvements over those found in another plot-based study for the same study area. Although coniferous RMSE values for volume (38.03 m3/ha) and biomass (17.15 Mg/ha) were comparable to published results, adjusted R2 values (0.66 and 0.59) were lower. This was attributed to more variability and a narrower range (6.94 - 350.93 m3/ha) in measured values. Classification accuracy for discriminant classification based on lidar point height distributions (89.2%) was a significant improvement over CHM-based classification (79%). A lack of modeling and classification differences between average segment sizes was attributed to the hierarchical nature of the segmentation algorithm. However, segment-based modeling was distinctly better than modeling based on existing forest stands, with values of 0.42 and 62.36 m3/ha (volume) and 0.46 and 41.18 Mg/ha (biomass) for adjusted R2 and RMSE, respectively. Modeling results and classification accuracies indicated that an object-oriented approach, based solely on lidar data, has potential for full-scale forest inventory applications. / Ph. D.

La segmentation hiérarchique est une méthode pour produire des partitions qui représentent une même image de manière de moins en moins fine. En même temps, elle sert d'entrée à la recherche d'une partition optimale, qui combine des extraits des diverses partitions en divers endroits. Le traitement hiérarchique des images est un domaine émergent en vision par ordinateur, et en particulier dans la communauté qui étudie les images hyperspectrales et les SIG, du fait de son capacité à structurer des données hyper-dimensionnelles. Le chapitre 1 porte sur les deux concepts fondamentaux de tresse et de treillis énergétique. La tresse est une notion plus riche que celle de hiérarchie de partitions, en ce qu'elle incorpore, en plus, des partitions qui ne sont pas emboîtées les unes dans les autres, tout en s'appuyant globalement sur une hiérarchie. Le treillis énergétique est une structure mixte qui regroupe une tresse avec une énergie, et permet d'y définir des éléments maximaux et minimaux. Lorsqu'on se donne une énergie, trouver la partition formée de classes de la tresse (ou de la hiérarchie) qui minimise cette énergie est un problème insoluble, de par sa complexité combinatoriale. Nous donnons les deux conditions de h-croissance et de croissance d'échelle, qui garantissent l'existence, l'unicité et la monotonie des solutions, et conduisent à un algorithme qui les détermine en deux passes de lecture des données. Le chapitre 2 reste dans le cadre précédent, mais étudie plus spécifiquement l'optimisation sous contrainte. Il débouche sur trois généralisations du modèle Lagrangien. Le chapitre 3 applique l'optimisation par treillis énergétique au cas de figure où l'énergie est introduite par une « vérité terrain », c'est à dire par un jeu de dessins manuel, que les partitions optimales doivent serrer au plus près. Enfin, le chapitre 4 passe des treillis énergétiques à ceux des courbes de Jordan dans le plan euclidien, qui définissent un modèle continu de segmentations hiérarchiques. Il permet entre autres de composer les hiérarchies avec diverses fonctions numériques / Hierarchical segmentation has been a model which both identifies with the construct of extracting a tree structured model of the image, while also interpreting it as an optimization problem of the optimal scale selection. Hierarchical processing is an emerging field of problems in computer vision and hyper-spectral image processing community, on account of its ability to structure high-dimensional data. Chapter 1 discusses two important concepts of Braids and Energetic lattices. Braids of partitions is a richer hierarchical partition model that provides multiple locally non-nested partitioning, while being globally a hierarchical partitioning of the space. The problem of optimization on hierarchies and further braids are non-tractable due the combinatorial nature of the problem. We provide conditions, of h-increasingness, scale-increasingness on the energy defined on partitions, to extract unique and monotonically ordered minimal partitions. Furthermore these conditions are found to be coherent with the Braid structure to perform constrained optimization on hierarchies, and more generally Braids. Chapter 2 demonstrates the Energetic lattice, and how it generalizes the Lagrangian formulation of the constrained optimization problem on hierarchies. Finally in Chapter 3 we apply the method of optimization using energetic lattices to the problem of extraction of segmentations from a hierarchy, that are proximal to a ground truth set. Chapter 4 we show how one moves from the energetic lattice on hierarchies and braids, to a numerical lattice of Jordan Curves which define a continous model of hierarchical segmentation. This model enables also to compose different functions and hierarchies

Depuis la fin des années 1990, les images optiques à très hautes résolutions spatiales issues de capteurs satellitaires sont de plus en plus accessibles par une vaste communauté d’utilisateurs. En particulier, différents systèmes satellitaires sont maintenant disponibles et produisent une quantité de données importante, utilisable pour l’observation de la Terre. En raison de cet important volume de données,les méthodes analytiques manuelles deviennent inadaptées pour un traitement efficace de ces données. Il devient donc crucial d’automatiser ces méthodes par des procédés informatiques, capables de traiter cette quantité de données hétérogènes.Dans le cadre de cette thèse, nos recherches se sont focalisées sur le développement de nouvelles approches basées régions (i.e., segmentation et classification) permettant l’extraction de plusieurs niveaux de connaissance et d’information à partir d’ensembles d’images à différentes résolutions spatiales. De telles images offrent en effet des vues différentes de la scène étudiée, ce qui peut permettre de faciliter l’extraction des objets d’intérêt. Ces derniers étant structurés sous la forme de hiérarchies d’objets complexes, nos travaux se sont naturellement tournés (1) vers l’utilisation d’approches de segmentation hiérarchique fournissant des ensembles de partitions de la scène à différents niveaux de détail et (2) vers l’intégration de connaissances de haut-niveau dans les processus de fouille de données. De manière plus générale, nous nous sommes intéressés à élaborer un outil informatique reposant sur une stratégie d’analyse descendante,similaire à celle d’un utilisateur, qui consiste à interpréter la scène en considérant, en premier lieu, les grandes zones composant les territoires (à partir des images aux résolutions les plus grossières) puis à affiner récursivement le niveau d’interprétation pour en extraire des zones plus spécialisées (à partir des images aux résolutions les plus fines).L’ensemble de ces travaux a été implanté dans une bibliothèque logicielle et validé dans le contexte de l’analyse d’environnements urbains à partir d’ensembles d’images multi résolutions. / In the field of remote sensing image analysis, the recognition of complex patterns from satellite images presents several challenges related to the size, the accuracy and the complexity of the considered data. Indeed, due tothe large amount of ground details provided by these images, the classical photo-interpretation approachesdo not provide satisfactory results. In this context, it is then relevant to develop new automatic tools adaptedto the extraction of complex patterns from such data.In this thesis, we have proposed new region-based approaches (i.e., segmentation and classification) enablingto extract different levels of information from sets of images at different spatial resolutions. Indeed, suchmultiresolution sets of images provide different (complementary) views on the represented objects of interestand can be used to make easier the extraction process of these objects. The main principle of the propose d'approach is to progressively extract and classify segments/objects of interest from the lowest to the highestresolution data, and then finally to determine complex patterns from VHSR images. This approach, inspired by the principle of photo-interpretation and human vision, merges hierarchical segmentation approaches withmultiresolution clustering strategies combined to the integration of high-level background knowledge.The proposed framework has been validated in the context of the urban mapping of complex objects.Experiments have been carried out on multiresolution sets of satellite images sensed over different cities. Theresults obtained have shown that the quality and the accuracy of the extracted patterns seem sufficient tofurther accurately perform both classification or object detection in an operational context.

Le but de ce travail est de fournir une méthode de reconstruction stéréoscopique capable d'estimer la topographie des catalyseurs à partir d'images MEB. Les méthodes stéréo standard ne permettent pas d'évaluer des reconstructions 3D de bonne qualité en raison de la surface homogène de ces échantillons. Bien que particulièrement prononcé sur nos catalyseurs, le manque de texture est un problème courant dans la reconstruction stéréo, et aucune solution idéale n'a encore été trouvée.Notre approche principale à ce problème est de combiner les méthodes stéréo existantes avec la segmentation hiérarchique des images de l’échantillon. En effet, la morphologie mathématique fournit des outils efficaces permettant de diviser une image en régions et sous-régions. Nous avons utilisé ces outils pour affiner et compléter les reconstructions 3D.La méthode que nous avons développé estime des reconstructions 3D moins bruitées et plus précises que les méthodes existantes. L'approche fournit également des informations supplémentaires: la segmentation finalement retenue ainsi que la carte indiquant l’orientation de chaque région sont des données intéressantes qui peuvent être utilisées pour affiner la compréhension des catalyseurs.Bien que le but de cette thèse soit très spécifique, l'approche proposée est généraliste. Elle a été notamment testée sur la base Middlebury et les résultats obtenus sont comparables et parfois meilleurs que les méthodes de pointe.L’approche pourrait aussi être étendue à d’autres cas d’utilisation. Tant que des données spatiales sont combinées avec une image, notre méthode TDSR peut être utilisée pour améliorer et compléter ces données spatiales. Les images RGBD et la segmentation sémantique sont quelques exemples d'applications potentielles. / The aim of this work is to provide a stereo reconstruction method able to estimate the topography of catalysts from SEM images. Standard stereo methods fail to evaluate adequate 3D reconstructions because of the homogeneous surface of these samples. Though particularly pronounced on our catalysts, the lack of texture is a common issue in stereo reconstruction, and no ideal solution has yet been found.Our main approach to this issue is to combine existing stereo methods with the hierarchical segmentation of the sample's images. Indeed, Mathematical Morphology provides efficient tools that divide an image into regions and subregions. We have used these tools to refine and complete the 3D reconstructions.The method we have developed estimates 3D reconstructions that are less noisy and more precise than state of the art methods. The approach also provides additional information: the final segmentation as well as the normal map are interesting data that can be used to refine the understanding of the catalysts.Though this thesis' purpose is very specific, the proposed approach is general.It has been notably used in the Middlebury database which contains images of in-door scenes, and obtained results were comparable and sometimes better than state of the art methods.It could also be extended to other uses. As long as spatial data is combined with an image, our TDSR method can be used to refine it. RGBD images and semantic segmentation are a few potential applications.

Cette thèse porte sur l'extraction de descripteurs hiérarchiques et multi-échelles d'images, en vue de leur interprétation, caractérisation et segmentation. Elle se décompose en deux parties.La première partie expose des éléments théoriques et méthodologiques sur l'obtention de classifications hiérarchiques des nœuds d'un graphe valué aux arêtes. Ces méthodes sont ensuite appliquées à des graphes représentant des images pour obtenir différentes méthodes de segmentation hiérarchique d'images. De plus, nous introduisons différentes façons de combiner des segmentations hiérarchiques. Nous proposons enfin une méthodologie pour structurer et étudier l'espace des hiérarchies que nous avons construites en utilisant la distance de Gromov-Hausdorff entre elles.La seconde partie explore plusieurs applications de ces descriptions hiérarchiques d'images. Nous exposons une méthode pour apprendre à extraire de ces hiérarchies une bonne segmentation de façon automatique, étant donnés un type d'images et un score de bonne segmentation. Nous proposons également des descripteurs d'images obtenus par mesure des distances inter-hiérarchies, et exposons leur efficacité sur des données réelles et simulées. Enfin, nous étendons les potentielles applications de ces hiérarchies en introduisant une technique permettant de prendre en compte toute information spatiale a priori durant leur construction. / This thesis deals with the extraction of hierarchical and multiscale descriptors on images, in order to interpret, characterize and segment them. It breaks down into two parts.The first part outlines a theoretical and methodological approach for obtaining hierarchical clusterings of the nodes of an edge-weighted graph. In addition, we introduce different approaches to combine hierarchical segmentations. These methods are then applied to graphs representing images and derive different hierarchical segmentation techniques. Finally, we propose a methodology for structuring and studying the space of hierarchies by using the Gromov-Hausdorff distance as a metric.The second part explores several applications of these hierarchical descriptions for images. We expose a method to learn how to automatically extract a segmentation of an image, given a type of images and a score of evaluation for a segmentation. We also propose image descriptors obtained by measuring inter-hierarchical distances, and expose their efficiency on real and simulated data. Finally, we extend the potential applications of these hierarchies by introducing a technique to take into account any spatial prior information during their construction.

Als Datengrundlage für die Erhebung von Flächennutzungsparametern, wie sie in geografischen Informationssystemen (GIS) abgelegt und verwaltet werden, dienen oft Bilddaten aus der Erdfernerkundung. Die zur Erkennung und Unterscheidung der Objekte notwendige hohe Pixelauflösung führt bei der Erfassung eines Zielgebiets wie beispielsweise einer Stadt zu enormen Datenmengen. Aus diesen Bilddaten gilt es, möglichst schnell und preiswert die für ein GIS notwendigen Informationen, wie Umrissvektoren und Objektattribute, zu extrahieren. Diese Arbeit ist ein Beitrag zur Automatisierung dieses Schritts mit besonderem Schwerpunkt auf der Gebäudeextraktion. Datengrundlage sind hochauflösende multispektrale Orthobilder und ein digitales Oberflächenmodell (DOM) der digitalen Luftbildkamera HRSC-A bzw. HRSC-AX zum Einsatz. Deswegen werden das Aufnahmeprinzip sowie die Datenverarbeitung der HRSC überblicksartig vorgestellt. Auf Basis dieser HRSC-Standarddatenprodukte wird ein Vorgehen zur Extraktion von Objekten entwickelt. In einer hierarchisch geordneten Abfolge an Segmentierungsschritten werden aus der Pixelinformation bedeutungstragende Einheiten extrahiert. Dieser Segmentierungsansatz lässt sich auf mehrere Objektkategorien, wie Straßen oder Ackerflächen, erweitern. So werden in der aktuellen Entwicklungsstufe neben Gebäuden auch Baumregionen detektiert. Anhand des Oberflächenmodells werden erhöhte Regionen erkannt. Dazu wird das DOM durch Berechnung eines Terrainmodells auf Grundhöhe normiert. Für erhöhte Objekte wird die Grundhöhe aus umliegenden Grundregionen abgeleitet. Die erhöhten Regionen werden anschließend in Bäume und Gebäude unterteilt. Dazu werden aus den Multispektraldaten Vegetationscharakteristika bestimmt und entsprechende Baumsegmente ermittelt. Die Gebäuderegionen resultieren aus einer Nachverarbeitung der verbleibenden Segmente. Um Gebäudekomplexe in einzelne Häuser aufzuteilen, wird ein gradientenbasierter Ansatz entwickelt. Anhand der für Brandmauern typischen Gradienteninformation werden Linienhypothesen zur Unterteilung der Gebäudesegmente generiert. Diese werden schrittweise anhand geometrischer und radiometrischer Kriterien auf ihre Plausibilität überprüft. Schließlich werden die ursprünglich aus dem DOM stammenden Konturen der Gebäudesegmente und deren Übereinstimung mit Bildkanten eines Orthobildes betrachtet. In einem adaptiven Ansatz wird das Konturpolygon durch die Gradienteninformation an angrenzende Bildkanten angepasst. Zur Umsetzung typischer Gebäudegeometrien wie rechter Winkel oder Parallelität werden innerhalb des Adaptionsprozesses entsprechende Nebenbedingungen formuliert. Die Extraktion erhöhter Objekte wie auch deren Unterteilung in Bäume und Gebäude erfolgt mit hoher Genauigkeit, z.B. liegen die Detektionsraten bei Gebäuden über 90%. Der neuartige Ansatz zur Unterteilung in einzelne Häuser ohne explizite Liniendetektion führt bereits in der vorgestellten Entwicklungsstufe zur Beschleunigung einer manuellen Interpretation. Die adaptive Verbesserung der Gebäudekontur führt zu gebäudetypischeren Umrissen ohne Beeinträchtigung der hohen Detektionsraten. / Remote sensing image data are often used as a basis for determining land use parameters, as they are stored and managed in geographic information systems (GIS). Covering a target area leads to an enormous amount of data due to the high pixel resolution required for recognizing and discriminating objects. To effectively derive GIS information like contour vectors or object attributes from these data, the extraction process has to be fast and cost-effective. This thesis is a contribution to the automization of this step with a focus on building extraction. High resolution multispectral ortho-images and a digital surface model (DSM), generated by the digital aerial camera HRSC-A or HRSC-AX, are used as data basis. Therefore, the HRSC imaging principle and data processing are summarized. Based on these HRSC standard data products, an object extraction scheme is developed. In a hierarchically ordered sequence of segmentation steps, meaningful units are extracted from pixel information. This segmentation approach is extendable to several object categories like streets or fields. Thus, tree regions, as well as buildings are detected in the current stage of implementation. Elevated regions are recognized using the digital surface model. For that purpose the DSM is normalized by calculating a terrain model. For elevated objects the terrain height is derived from surrounding ground regions. Subsequently, the elevated regions are separated into trees and buildings. Determining spectral characteristics of vegetation from the multispectral data leads to corresponding tree segments. The building regions result from post-processing the remaining segments. In order to split the building segments into single houses, a gradient based approach is developed. By means of the gradient information associated with firewalls, line hypotheses for subdividing the building segments are generated. Their plausibility is checked by gradually applying geometric and spectral criteria. Finally, the building contours, originally derived from the DSM, and their correspondence to image edges in an ortho-image, are considered. In an adaptive approach, the contour polygon is adjusted to neighboring image edges using the gradient information. Typical building geometries like right angles or parallelism are enforced by applying corresponding constraints in the adaption process. The extraction of elevated objects, as well as the separation into trees and buildings, is carried out with high accuracy, e.g. the building detection rates are over 90%. In the current development stage the novel approach for separating building segments into single houses without an explicit line detection already leads to a speeding-up of a manual interpretation. The adaptive improvement of building contours leads to building typical contours without affecting the high detection rates.