This thesis investigates two well defined problems in image segmentation, viz. interactive and semantic image segmentation. Interactive segmentation involves power assisting a user in cutting out objects from an image, whereas semantic segmentation involves partitioning pixels in an image into object categories. We investigate various models and energy formulations for both these problems in this thesis. In order to improve the performance of interactive systems, low level texture features are introduced as a replacement for the more commonly used RGB features. To quantify the improvement obtained by using these texture features, two annotated datasets of images are introduced (one consisting of natural images, and the other consisting of camouflaged objects). A significant improvement in performance is observed when using texture features for the case of monochrome images and images containing camouflaged objects. We also explore adding mid-level cues such as shape constraints into interactive segmentation by introducing the idea of geodesic star convexity, which extends the existing notion of a star convexity prior in two important ways: (i) It allows for multiple star centres as opposed to single stars in the original prior and (ii) It generalises the shape constraint by allowing for Geodesic paths as opposed to Euclidean rays. Global minima of our energy function can be obtained subject to these new constraints. We also introduce Geodesic Forests, which exploit the structure of shortest paths in implementing the extended constraints. These extensions to star convexity allow us to use such constraints in a practical segmentation system. This system is evaluated by means of a “robot user” to measure the amount of interaction required in a precise way, and it is shown that having shape constraints reduces user effort significantly compared to existing interactive systems. We also introduce a new and harder dataset which augments the existing GrabCut dataset with more realistic images and ground truth taken from the PASCAL VOC segmentation challenge. In the latter part of the thesis, we bring in object category level information in order to make the interactive segmentation tasks easier, and move towards fully automated semantic segmentation. An algorithm to automatically segment humans from cluttered images given their bounding boxes is presented. A top down segmentation of the human is obtained using classifiers trained to predict segmentation masks from local HOG descriptors. These masks are then combined with bottom up image information in a local GrabCut like procedure. This algorithm is later completely automated to segment humans without requiring a bounding box, and is quantitatively compared with other semantic segmentation methods. We also introduce a novel way to acquire large quantities of segmented training data relatively effortlessly using the Kinect. In the final part of this work, we explore various semantic segmentation methods based on learning using bottom up super-pixelisations. Different methods of combining multiple super-pixelisations are discussed and quantitatively evaluated on two segmentation datasets. We observe that simple combinations of independently trained classifiers on single super-pixelisations perform almost as good as complex methods based on jointly learning across multiple super-pixelisations. We also explore CRF based formulations for semantic segmentation, and introduce novel visual words based object boundary description in the energy formulation. The object appearance and boundary parameters are trained jointly using structured output learning methods, and the benefit of adding pairwise terms is quantified on two different datasets.

Stimuli outside classical receptive fields have been shown to exert significant influence over the activities of neurons in primary visual cortexWe propose that contextual influences are used for pre-attentive visual segmentation, in a new framework called segmentation without classification. This means that segmentation of an image into regions occurs without classification of features within a region or comparison of features between regions. This segmentation framework is simpler than previous computational approaches, making it implementable by V1 mechanisms, though higher leve l visual mechanisms are needed to refine its output. However, it easily handles a class of segmentation problems that are tricky in conventional methods. The cortex computes global region boundaries by detecting the breakdown of homogeneity or translation invariance in the input, using local intra-cortical interactions mediated by the horizontal connections. The difference between contextual influences near and far from region boundaries makes neural activities near region boundaries higher than elsewhere, making boundaries more salient for perceptual pop-out. This proposal is implemented in a biologically based model of V1, and demonstrated using examples of texture segmentation and figure-ground segregation. The model performs segmentation in exactly the same neural circuit that solves the dual problem of the enhancement of contours, as is suggested by experimental observations. Its behavior is compared with psychophysical and physiological data on segmentation, contour enhancement, and contextual influences. We discuss the implications of segmentation without classification and the predictions of our V1 model, and relate it to other phenomena such as asymmetry in visual search.

Useful information extraction from satellite images for the use of other higher level applications such as road network extraction and update, city planning etc. is a very important and active research area. It is seen that pixel-based techniques becomes insufficient for this task with increasing spatial resolution of satellite imaging sensors day by day. Therefore, the use of object-based techniques becomes indispensable and the segmentation method selection is very crucial for object-based techniques. In this thesis, various segmentation algorithms applied in remote sensing literature are presented and a segmentation process that is based on watersheds and multi-scale segmentation is proposed to use as the segmentation step of an object-based classifier. For every step of the proposed segmentation process, qualitative and quantitative comparisons with alternative approaches are done. The ones which provide best performance are incorporated into the proposed algorithm. Also, an unsupervised segmentation accuracy metric to determine all parameters of the algorithm is proposed. By this way, the proposed segmentation algorithm has become a fully automatic approach. Experiments that are done on a database formed with images taken from Google Earth&reg / software provide promising results.

In this study, an automatic video segmentation and classification system based on audio features has been presented. Video sequences are classified such as videos with &ldquo / speech&rdquo / , &ldquo / music&rdquo / , &ldquo / crowd&rdquo / and &ldquo / silence&rdquo / . The segments that do not belong to these regions are left as &ldquo / unclassified&rdquo / . For the silence segment detection, a simple threshold comparison method has been done on the short time energy feature of the embedded audio sequence. For the &ldquo / speech&rdquo / , &ldquo / music&rdquo / and &ldquo / crowd&rdquo / segment detection a multiclass classification scheme has been applied. For this purpose, three audio feature set have been formed, one of them is purely MPEG-7 audio features, other is the audio features that is used in [31] the last one is the combination of these two feature sets. For choosing the best feature a histogram comparison method has been used. Audio segmentation system was trained and tested with these feature sets. The evaluation results show that the Feature Set 3 that is the combination of other two feature sets gives better performance for the audio classification system. The output of the classification system is an XML file which contains MPEG-7 audio segment descriptors for the video sequence. An application scenario is given by combining the audio segmentation results with visual analysis results for getting audio-visual video segments.

Made available in DSpace on 2014-12-17T15:48:12Z (GMT). No. of bitstreams: 1 LucasMO.pdf: 1455032 bytes, checksum: 6bc4218b3d779cfc9915c6a2efda34f1 (MD5) Previous issue date: 2007-02-23 / Conselho Nacional de Desenvolvimento Cient?fico e Tecnol?gico / Image segmentation is the process of subdiving an image into constituent regions or objects that have similar features. In video segmentation, more than subdividing the frames in object that have similar features, there is a consistency requirement among segmentations of successive frames of the video. Fuzzy segmentation is a region growing technique that assigns to each element in an image (which may have been corrupted by noise and/or shading) a grade of membership between 0 and 1 to an object. In this work we present an application that uses a fuzzy segmentation algorithm to identify and select particles in micrographs and an extension of the algorithm to perform video segmentation. Here, we treat a video shot is treated as a three-dimensional volume with different z slices being occupied by different frames of the video shot. The volume is interactively segmented based on selected seed elements, that will determine the affinity functions based on their motion and color properties. The color information can be extracted from a specific color space or from three channels of a set of color models that are selected based on the correlation of the information from all channels. The motion information is provided into the form of dense optical flows maps. Finally, segmentation of real and synthetic videos and their application in a non-photorealistic rendering (NPR) toll are presented / Segmenta??o de imagens ? o processo que subdivide uma imagem em partes ou objetos de acordo com alguma caracter?stica comum. J? na segmenta??o de v?deos, al?m dos quadros serem divididos em fun??o de alguma caracter?stica, ? necess?rio obter uma coer?ncia temporal entre as segmenta??es de frames sucessivos do v?deo. A segmenta??o fuzzy ? uma t?cnica de segmenta??o por crescimento de regi?es que determina para cada elemento da imagem um grau de pertin?ncia (entre zero e um) indicando a confian?a de que esse elemento perten?a a um determinado objeto ou regi?o existente na imagem. O presente trabalho apresenta uma aplica??o do algoritmo de segmenta??o fuzzy de imagem, e a extens?o deste para segmentar v?deos coloridos. Nesse contexto, os v?deos s?o tratados como volumes 3D e o crescimento das regi?es ? realizado usando fun??es de afinidade que atribuem a cada pixel um valor entre zero e um para indicar o grau de pertin?ncia que esse pixel tem com os objetos segmentados. Para segmentar as seq??ncias foram utilizadas informa??es de movimento e de cor, sendo que essa ?ltima ? proveniente de um modelo de cor convencional, ou atrav?s de uma metodologia que utiliza a correla??o de Pearson para selecionar os melhores canais para realizar a segmenta??o. A informa??o de movimento foi extra?da atrav?s do c?lculo do fluxo ?ptico entre dois frames adjacentes. Por ?ltimo ? apresentada uma an?lise do comportamento do algoritmo na segmenta??o de seis v?deos e um exemplo de uma aplica??o que utiliza os mapas de segmenta??o para realizar renderiza??es que n?o sejam foto real?sticas

[pt] Exames de ultrassom possuem um importante papel na obstetrícia devido a seu baixo custo, baixo risco e sua capacidade de execução em tempo real. O advento da ultrassonografia tridimensional possibilitou o uso do volume fetal como medida biométrica para monitorar seu desenvolvimento. A quantificação do volume do feto requer um processo prévio de segmentação, que consiste na rotulação dos pixels pertencentes ao objeto de interesse em uma imagem digital. Não existe, entretanto, um método padrão para a realização da volumetria fetal e a maioria dos estudos conta com a realização de segmentações manuais, que demandam uma alta carga de trabalho repetitivo. A segmentação de imagens de ultrassom é particularmente desafiadora devido à presença de artefatos como o ruído speckle e sombras acústicas e ao fato de que o contraste entre as regiões de interesse é muitas vezes baixo. Neste trabalho, desenvolvemos e testamos um método semiautomático de segmentação do feto em exames de ultrassom 3D. Devido às dificuldades citadas, bons métodos de segmentação em imagens de ultrassom devem fazer uso de características esperadas das estruturas específicas a serem segmentadas. Esse pensamento guiou o desenvolvimento da nossa metodologia que, através uma sequência de passos simples, antingiu bons resultados quantitativos na tarefa de segmentação. / [en] Ultrasound exams have an important role in obstetrics due to its low cost, low risk and real-time capabilities. The advent of three-dimensional ultrasonography has made possible the use of the fetal volume as a biometric measurement to monitor its development. The quantification of the fetal volume requires a previous process of segmentation, which consists in the labelling of the pixels that belong to the object of interest in a digital image. There isn t, however, a standard methodology for fetal volumetry and most studies rely on manual segmentations. The segmentation of ultrasound images is particularly challenging due to the presence of artifacts as the speckle noise and acoustic shadows, and the fact that the contrast between regions of interest is commonly low. In this study, we have developed and tested a semiautomatic method of fetal segmentation in 3D ultrasound exams. Due to the aforementioned difficulties, good ultrasound segmentation methods need to make use of expected characteristics of the specific segmented structures. This thought has guided the development of our methodology that, through a sequence of simple steps, achieved good quantitative results in the segmentation task.

As we navigate the world, for example when driving a car from our home to the work place, we continuously perceive the 3D structure of our surroundings and intuitively recognise the objects we see. Such capabilities help us in our everyday lives and enable free and accurate movement even in completely unfamiliar places. We largely take these abilities for granted, but for robots, the task of understanding large outdoor scenes remains extremely challenging. In this thesis, I develop novel algorithms for (near) real-time dense 3D reconstruction and semantic segmentation of large-scale outdoor scenes from passive cameras. Motivated by "smart glasses" for partially sighted users, I show how such modeling can be integrated into an interactive augmented reality system which puts the user in the loop and allows her to physically interact with the world to learn personalized semantically segmented dense 3D models. In the next part, I show how sparse but very accurate 3D measurements can be incorporated directly into the dense depth estimation process and propose a probabilistic model for incremental dense scene reconstruction. To relax the assumption of a stereo camera, I address dense 3D reconstruction in its monocular form and show how the local model can be improved by joint optimization over depth and pose. The world around us is not stationary. However, reconstructing dynamically moving and potentially non-rigidly deforming texture-less objects typically require "contour correspondences" for shape-from-silhouettes. Hence, I propose a video segmentation model which encodes a single object instance as a closed curve, maintains correspondences across time and provide very accurate segmentation close to object boundaries. Finally, instead of evaluating the performance in an isolated setup (IoU scores) which does not measure the impact on decision-making, I show how semantic 3D reconstruction can be incorporated into standard Deep Q-learning to improve decision-making of agents navigating complex 3D environments.

L’objectif de cette thèse est de proposer un algorithme de segmentation et de suivi du myocarde basé sur le formalisme des ensembles de niveaux. Nous modélisons dans un premier temps le myocarde par un modèle géométrique (hyperquadriques) qui permet de représenter des formes asymétriques telles que le myocarde tout en évitant une étape d’apprentissage. Ce modèle est ensuite inclus dans le formalisme des ensembles de niveaux afin de servir de contrainte de forme lors de la segmentation simultanée de l’endocarde et de l’épicarde. Ce terme d’a priori de forme est couplé à un terme local d’attache aux données ainsi qu’à un terme évitant la fusion des deux contours. L’algorithme est validé sur 80 images en fin systole et en fin diastole segmentées par 3 cardiologues. Dans un deuxième temps, nous proposons de segmenter l’ensemble d’une séquence en utilisant l’information de mouvement. Dans ce but, nous faisons l’hypothèse de conservation des niveaux de la fonction implicite associée à l’ensemble de niveaux et l’exprimons comme une énergie dans un formalisme variationnel. Cette énergie est ensuite ajoutée à l’algorithme décrit précédemment pour la segmentation statique du myocarde afin de contraindre temporellement l’évolution du contour. L’algorithme est alors validé sur 20 séquences échocardiographiques (soit environ 1200 images) segmentées par 2 experts. / The aim of this work is to propose an algorithm to segment and track the myocardium using the level-set formalism. The myocardium is first approximated by a geometric model (hyperquadrics) which allows to handle asymetric shapes such as the myocardium while avoiding a learning step. This representation is then embedded into the level-set formalism as a shape prior for the joint segmentation of the endocardial and epicardial borders. This shape prior term is coupled with a local data attachment term and a thickness term that prevents both contours from merging. The algorithm is validated on a dataset of 80 images at end diastolic and end systolic phase with manual references from 3 cardiologists. In a second step, we propose to segment whole sequences using motion information. To this end, we apply a level conservation constraint on the implicit function associated to the level-set and express this contraint as an energy term in a variational framework. This energy is then added to the previously described algorithm in order to constrain the temporal evolution of the contour. Finally the algorithm is validated on 20 echocardiographic sequences with manual references of 2 experts (corresponding to approximately 1200 images).

Abstract : Malignant brain tumors are the second leading cause of cancer related deaths in children under 20. There are nearly 700,000 people in the U.S. living with a brain tumor and 17,000 people are likely to loose their lives due to primary malignant and central nervous system brain tumor every year. To identify whether a patient is diagnosed with brain tumor in a non-invasive way, an MRI scan of the brain is acquired followed by a manual examination of the scan by an expert who looks for lesions (i.e. cluster of cells which deviate from healthy tissue). For treatment purposes, the tumor and its sub-regions are outlined in a procedure known as brain tumor segmentation . Although brain tumor segmentation is primarily done manually, it is very time consuming and the segmentation is subject to variations both between observers and within the same observer. To address these issues, a number of automatic and semi-automatic methods have been proposed over the years to help physicians in the decision making process. Methods based on machine learning have been subjects of great interest in brain tumor segmentation. With the advent of deep learning methods and their success in many computer vision applications such as image classification, these methods have also started to gain popularity in medical image analysis. In this thesis, we explore different machine learning and deep learning methods applied to brain tumor segmentation. / Résumé: Les tumeurs malignes au cerveau sont la deuxième cause principale de décès chez les enfants de moins de 20 ans. Il y a près de 700 000 personnes aux États-Unis vivant avec une tumeur au cerveau, et 17 000 personnes sont chaque année à risque de perdre leur vie suite à une tumeur maligne primaire dans le système nerveu central. Pour identifier de façon non-invasive si un patient est atteint d'une tumeur au cerveau, une image IRM du cerveau est acquise et analysée à la main par un expert pour trouver des lésions (c.-à-d. un groupement de cellules qui diffère du tissu sain). Une tumeur et ses régions doivent être détectées à l'aide d'une segmentation pour aider son traitement. La segmentation de tumeur cérébrale et principalement faite à la main, c'est une procédure qui demande beaucoup de temps et les variations intra et inter expert pour un même cas varient beaucoup. Pour répondre à ces problèmes, il existe beaucoup de méthodes automatique et semi-automatique qui ont été proposés ces dernières années pour aider les praticiens à prendre des décisions. Les méthodes basées sur l'apprentissage automatique ont suscité un fort intérêt dans le domaine de la segmentation des tumeurs cérébrales. L'avènement des méthodes de Deep Learning et leurs succès dans maintes applications tels que la classification d'images a contribué à mettre de l'avant le Deep Learning dans l'analyse d'images médicales. Dans cette thèse, nous explorons diverses méthodes d'apprentissage automatique et de Deep Learning appliquées à la segmentation des tumeurs cérébrales.

La segmentation de l’axe antéro-postérieur en somites est une caractéristique majeure des Vertébrés. Ce processus est basé sur un oscillateur, l’« horloge de segmentation ». Cette thèse cherche à comprendre la dynamique de signalisation régulant ce processus. Nous avons étudié la régulation transcriptionnelle de Mesp2 et nous avons montré que Tbx6 contrôle son expression chez le poulet. Nous présentons également un système d’étude ex vivo présentant des oscillations stables du gène cyclique Lfng. Nous avons mis en évidence un effet de population régulant la génération de ces oscillations et reposant sur la voie Notch et des facteurs mécaniques que nous interprétons avec un modèle d’oscillateur excitable. De plus, nous avons démontré un effet dose-dépendant de la voie Fgf sur la différenciation cellulaire, remettant ainsi en question le modèle actuel de segmentation. Par ailleurs, ce système d’étude nous a permis d’identifier un rôle du taux de traduction dans le contrôle de la période de l’horloge. Enfin, nous présentons des travaux, où nous cherchons à reconstituer l’horloge de segmentation in vitro à partir de cellules souches murines différenciées. / The segmentation of the anteroposterior axis in somites is a major feature of Vertebrates. This process relies on an oscillator, the “segmentation clock”. The present thesis addresses the signaling dynamics regulating this process. We studied the transcriptional regulation of Mesp2 and showed that Tbx6 controls its expression in chicken. We established an ex vivo experimental system with stable oscillations of the cyclic gene Lfng. We demonstrated the existence of a population behavior that controls the generation of oscillations and involves the Notch pathway and mechanical factors. We interpreted these observations in the framework of an excitable oscillator. Moreover, we evidenced a dose-dependent effect of Fgf signaling on cell determination that challenges current models of segmentation. Furthermore, this experimental system has enabled us to identify a role of the translation rate on the clock period. Last, we showed ongoing work aiming to recapitulate the segmentation in vitro using differentiated mouse embryonic stem cells.