Le travail de thèse que nous présentons s’articule autour de l’utilisation de caméras motorisées à trois degrés de liberté, également appelées caméras PTZ. Ces caméras peuvent être pilotées suivant deux angles. L’angle de panorama (θ) permet une rotation autour d’un axe vertical et l’angle de tangage (ϕ) permet une rotation autour d’un axe horizontal. Si, théoriquement, ces caméras permettent donc une vue omnidirectionnelle, elles limitent le plus souvent la rotation suivant l’angle de panorama mais surtout suivant l’angle de tangage. En plus du pilotage des rotations, ces caméras permettent également de contrôler la distance focale permettant ainsi un degré de liberté supplémentaire. Par rapport à d’autres modèles, les caméras PTZ permettent de construire un panorama - représentation étendue d’une scène construite à partir d’une collection d’image - de très grande résolution. La première étape dans la construction d’un panorama est l’acquisition des différentes prises de vue. A cet effet, nous avons réalisé une étude théorique permettant une couverture optimale de la sphère à partir de surfaces rectangulaires en limitant les zones de recouvrement. Cette étude nous permet de calculer une trajectoire optimale de la caméra et de limiter le nombre de prises de vues nécessaires à la représentation de la scène. Nous proposons également différents traitements permettant d’améliorer sensiblement le rendu et de corriger la plupart des défauts liés à l’assemblage d’une collection d’images acquises avec des paramètres de prises de vue différents. Une part importante de notre travail a été consacrée au recalage automatique d’images en temps réel, c’est à dire que chaque étapes est effectuée en moins de 40ms pour permettre le traitement de 25 images par seconde. La technologie que nous avons développée permet d’obtenir un recalage particulièrement précis avec un temps d’exécution de l’ordre de 4ms (AMD1.8MHz). Enfin, nous proposons deux applications de suivi d’objets en mouvement directement issues de nos travaux de recherche. La première associe une caméra PTZ à un miroir sphérique. L’association de ces deux éléments permet de détecter tout objet en mouvement dans la scène puis de se focaliser sur l’un d’eux. Dans le cadre de cette application, nous proposons un algorithme de calibrage automatique de l’ensemble caméra et miroir. La deuxième application n’exploite que la caméra PTZ et permet la segmentation et le suivi des objets dans la scène pendant le mouvement de la caméra. Par rapport aux applications classiques de suivi de cible en mouvement avec une caméra PTZ, notre approche se différencie par le fait que réalisons une segmentation fine des objets permettant leur classification. / The thesis considers the of use motorized cameras with 3 degrees of freedom which are commonly called PTZ cameras. The orientation of such cameras is controlled according to two angles: the panorama angle (θ) describes the degree of rotation around on vertical axis and the tilt angle (ϕ) refers to rotation along a meridian line. Theoretically, these cameras can cover an omnidirectional field of vision of 4psr. Generally, the panorama angle and especially the tilt angle are limited for such cameras. In addition to control of the orientation of the camera, it is also possible to control focal distance, thus allowing an additional degree of freedom. Compared to other material, PTZ cameras thus allow one to build a panorama of very high resolution. A panorama is a wide representation of a scene built starting from a collection of images. The first stage in the construction of a panorama is the acquisition of the various images. To this end, we made a theoretical study to determine the optimal paving of the sphere with rectangular surfaces to minimize the number of zones of recovery. This study enables us to calculate an optimal trajectory of the camera and to limit the number of images necessary to the representation of the scene. We also propose various processing techniques which appreciably improve the rendering of the mosaic image and correct the majority of the defaults related to the assembly of a collection of images which were acquired with differing image capture parameters. A significant part of our work was used to the automatic image registration in real time, i.e. lower than 40ms. The technology that we developed makes it possible to obtain a particularly precise image registration with an computation time about 4ms (AMD1.8MHz). Our research leads directly to two proposed applications for the tracking of moving objects. The first involves the use of a PTZ camera and a spherical mirror. The combination of these two elements makes it possible to detect any motion object in the scene and to then to focus itself on one of them. Within the framework of this application, we propose an automatic algorithm of calibration of the system. The second application exploits only PTZ camera and allows the segmentation and the tracking of the objects in the scene during the movement of the camera. Compared to the traditional applications of motion detection with a PTZ camera, our approach is different by the fact that it compute a precise segmentation of the objects allowing their classification.

Master of Science / Department of Mechanical and Nuclear Engineering / Dale E. Schinstock, Chris Lewis / The Autonomous Vehicle Systems Lab specializes in using autonomous planes for remote sensing applications. By developing an inexpensive image acquisition platform and the algorithms to post process the data, remote sensing can be performed at a lower monetary cost with shorter lead times. This thesis presents one algorithm that has shown to be an effective alternative to the traditional Bundle Adjustment (BA) algorithm used for making composite images from many individual overlapping images. BA simultaneously estimates camera poses and visible feature locations from blocks of overlapping imagery, but is computationally expensive. The alternate algorithm (ABA) uses a cost function that does not explicitly include the feature locations. For photographic sets covering large areas, but having overlap only between adjacent photos, the search space and consequently the computational cost is significantly reduced when compared to typical BA. The usefulness of the algorithm is demonstrated by comparing a digital elevation model created through the ABA with LIDAR data.

Um mosaico é o conjunto de fotos de uma determinada área, recortadas e montadas técnica e artísticamente, de forma a dar a impressão de que todo o conjunto é uma única fotografia. No caso de fotografias aéreas, sua utilização soluciona o problema da necessidade de se retratar uma área de interesse mais extensa do que o campo de cobertura das lentes da câmera utilizada. O foco deste trabalho é a criação automática de mosaicos buscando encontrar a posição real de um conjunto de imagens imagens adquiridas em baixa altitude, de baixa escala, em relação à um Mapa de Base, de escala maior, realizando, assim, uma correlação entre imagens de escalas diferentes. Este problema é abordado por técnicas de análise multi-escala, mais precisamente, pela utilização de filtros de Gabor. A metodologia desenvolvida utiliza um banco de filtros de Gabor aplicado sobre uma imagem de referência de modo que, a partir da aplicação destes filtros sobre a mesma, seja possível gerar um processo automático de geração do mosaico para o restante do conjunto de imagens. Experimentos realizados utilizando o método proposto demonstram a eficácia do mesmo para imagens com texturas de orientação marcante, como o caso de imagens aéreas de plantação de eucaliptos / A mosaic is a set of pictures of a given area, technically and artistically cut and ?glued? together, giving the impression that the entire set resembles a single picture. For aerial photography, the use of mosaics solves the problem of imaging an area of interest whose dimension is much larger than that covered by the majority of the cameras available. This work focuses on the automatic creation of mosaics and aims to compute the real position of a set of images acquired at low altitudes (lower scale), in relation with a base map larger scale), by correlating images in different scales. Multi-scale analysis techniques, in particular, the Gabor filters, constitute an approach to this problem. The proposed methodology uses a bank of Gabor filters applied over a reference image in a way that an automatic process of mosaic generation, with the remaining set of images, could be carried out. Experiments have shown the efficiency of the proposed technique especially for images with clear textural orientation, for example, the case of aerial photographs of eucalyptus plantations

Cette thèse propose un algorithme de mosaïquage pour la construction d'images panoramiques des parois internes de la vessie. Le temps de construction de ces images correspondant aux parties intéressantes de la vessie doit être inférieur à la durée d'un examen clinique standard. La méthode de mosaïquage doit aussi être robuste vis-à-vis des variabilités inter-examens liées aux patients et aux instruments. Ces images panoramiques pourront être utilisées par le clinicien comme référence pour guider des examens ultérieurs, pour l'archivage des données et pour suivre l'évolution des lésions. La première étape de l'algorithme est le pré-traitement des images cystoscopiques consistant en l'atténuation des inhomogénéités d'illumination et du motif de fibres optiques visible dans les images acquises par un fibroscope. La deuxième étape est le recalage des images. La solution retenue consiste en la corrélation par les transformées de Fourier des images qui fournit des translations initiales à un algorithme itératif basé sur la différence d'intensité entre les images. Ce dernier délivre les paramètres de la transformation perspective reliant deux images successives de la séquence. Dans la troisième étape nous projetons les images dans un repère commun en utilisant des transformations globales calculées avec les résultats des recalages. Nous utilisons un moyennage pondéré des intensités des pixels pour atténuer les bords visibles lors de la projection. Les résultats quantitatifs obtenus avec un fantôme et des résultats qualitatifs calculés pour des séquences réelles montrent que notre approche automatique de mosaïquage est robuste et rapide (temps compatible avec la durée d'un examen cystoscopique clinique). Nos tests ont également prouvé que l'algorithme de recalage fonctionne pour des transformations géométriques plus grandes que celles rencontrées typiquement entre images d'une séquence vidéo (90% de recouvrement entre images consécutives pour ces dernières) / This work describes a mosaicing algorithm for constructing panoramic images of internal walls of the bladder. Time relating to the construction of panoramic images including the interesting parts must be shorter than that required by a standard cystoscopic examination. The mosaicing algorithm must be robust against lighting conditions, morphologic and texture variations relating to instruments and patient anatomy. These panoramic images could be used by a clinician for guiding further exams, storing non-redundant data and following-up evolution of lesions. The preprocessing of cystoscopic images is the first stage of the algorithm. Preprocessing consists of shading correction and fiber optics pattern attenuation occurring in fiberscope acquired images. The second stage is image registration. The chosen solution consist of cross-correlating images (using their Fourier transforms) in order to have initial translations for an iterative registration algorithm based on the sum of squared differences of images. In the third stage, images are projected in the coordinate system of the panoramic image using global transformations computed with matrices given by the iterative registration. We use a weighted average of pixel intensities to blend visible borders of images produced in the projection process. Numerical results obtained with a phantom and qualitative results obtained with real sequences show that our automatic approach is robust and allows for a fast construction of panoramic images in a period of time that is shorter than the duration of a clinical cystoscopic examination. Our experiments showed that the registration algorithm can handle geometric transformations that are larger than those existing typically in a video-sequence (90% of superposition between successive images in this case)

La microscopie confocale classique permet d'obtenir des images à haute réso- lution de cellules en culture ou dans un tissu biologique excisé. Cette technologie peut être adaptée aux applications in vivo grâce à l'utilisation de fibres optiques et d'optiques miniaturisées. A terme, la microscopie confocale fibrée devrait permettre aux médecins et biologistes de réaliser des biopsies optiques; c'est à dire un exa- men histologique, en temps réel, des tissus biologiques à l'intérieur d'un organisme vivant et directement au contact de la zone d'intérêt. Le but premier de cette thèse est de dépasser les limites matérielles de ces in- struments d'imagerie en développant des outils de recalage d'images spécifiques et innovants. En particulier, le propos de ce manuscrit est cadré par l'objectif de pro- poser, au travers d'outils de création de mosaïques d'images, des biopsies optiques à grand champ aux médecins. Cette application est considérée, dans cette thèse, comme un système, ou un circuit, qui prendrait en entrée un flot de données brutes et délivrerait en sortie des mosaïques d'images à grand champ. Nous détaillons les éléments critiques de ce système, en particulier la reconstruction d'images en temps réel, le recalage linéaire d'images et le recalage non linéaire, avant de présenter la structure du système complet. Les données brutes produites par la microscopie confocale fibrée sont difficiles à interpréter parce qu'elle sont modulées par la structure en nid d'abeille du réseau de fibres optiques et parce qu'elle sont entachées d'artefacts géométriques. Dans ce contexte, nous montrons qu'une reconstruction en temps réel des images peut être utilisée en pré-traitement afin de produire des séquences vidéos directement interprétables. Comme la microscopie confocale fibrée est une imagerie qui se fait au contact des tissus, le mouvement relatif du tissu par rapport à la sonde optique implique qu'il est parfois difficile d'obtenir de manière robuste certaines mesures quantitatives d'intérêt. Nous avons donc attaqué le problème du recalage linéaire, efficace et robuste de paires d'images. Nous montrons que des outils ré- cents provenant du domaine du contrôle robotique par la vision peuvent surpasser les solutions standards utilisées en analyse d'images biomédicales. L'adéquation de ces outils au problème du recalage linéaire d'images nous a amenés à revisiter le problème du recalage non-linéaire. En interprétant le recalage non-linéaire comme un problème d'optimisation sur un groupe de Lie, nous développons un algorithme rapide de recalage difféomorphe non-paramétrique d'images. En plus d'être dif- féomorphe, notre algorithme produit des résultats qui sont similaires à ceux de l'algorithme des démons de Thirion mais qui sont plus lisses et plus proche de la vérité. Finalement, nous obtenons une boîte à outils de reconstruction et de recalage d'images que nous utilisons pour proposer un algorithme robuste de création de mosaïques d'images qui permette de calculer un alignement globalement cohérent à partir de résultats locaux, de compenser les distorsions liées au mouvement et de retrouver les déformations non-rigides. Par ailleurs, notre algorithme de mosaïques d'images a récemment été incorporé dans un essai clinique multicentrique. Cet essai illustre l'intérêt clinique de nos outils dans le cadre spécifique de la surveillance de l'oesophage de Barrett.

Um mosaico é o conjunto de fotos de uma determinada área, recortadas e montadas técnica e artísticamente, de forma a dar a impressão de que todo o conjunto é uma única fotografia. No caso de fotografias aéreas, sua utilização soluciona o problema da necessidade de se retratar uma área de interesse mais extensa do que o campo de cobertura das lentes da câmera utilizada. O foco deste trabalho é a criação automática de mosaicos buscando encontrar a posição real de um conjunto de imagens imagens adquiridas em baixa altitude, de baixa escala, em relação à um Mapa de Base, de escala maior, realizando, assim, uma correlação entre imagens de escalas diferentes. Este problema é abordado por técnicas de análise multi-escala, mais precisamente, pela utilização de filtros de Gabor. A metodologia desenvolvida utiliza um banco de filtros de Gabor aplicado sobre uma imagem de referência de modo que, a partir da aplicação destes filtros sobre a mesma, seja possível gerar um processo automático de geração do mosaico para o restante do conjunto de imagens. Experimentos realizados utilizando o método proposto demonstram a eficácia do mesmo para imagens com texturas de orientação marcante, como o caso de imagens aéreas de plantação de eucaliptos / A mosaic is a set of pictures of a given area, technically and artistically cut and ?glued? together, giving the impression that the entire set resembles a single picture. For aerial photography, the use of mosaics solves the problem of imaging an area of interest whose dimension is much larger than that covered by the majority of the cameras available. This work focuses on the automatic creation of mosaics and aims to compute the real position of a set of images acquired at low altitudes (lower scale), in relation with a base map larger scale), by correlating images in different scales. Multi-scale analysis techniques, in particular, the Gabor filters, constitute an approach to this problem. The proposed methodology uses a bank of Gabor filters applied over a reference image in a way that an automatic process of mosaic generation, with the remaining set of images, could be carried out. Experiments have shown the efficiency of the proposed technique especially for images with clear textural orientation, for example, the case of aerial photographs of eucalyptus plantations

This is a study exploring how robust one feature descriptors, scale invariant feature transform (SIFT), and one feature detector, feature accelerated segmentation test (FAST), are in terms of handling fetoscopic acquired data when mosaicing. Today’s treatment of severe Twin-to-Twin Transfusion Syndrome at Karolinska University Hospital is fetoscopic guided laser occlusion of chorioangiopagous vessels (FLOC) where intersecting blood vessels causing a transfusion (Anastomoses) in between the fetuses are occluded. These blood vessels are located somewhere on the placenta. The fetoscopy includes navigation of a relatively large area where the field of view (FOV) is limited. The limited FOV during the fetoscopy makes it cumbersome to navigate and identify intersected blood vessels. The motivation of this study is to explore ways of dealing with the complications during FLOC by mosaicing an overview of the placenta that can be used as an assisting map to make the procedure safer by improving navigation of the fetoscope and identification of blood vessels during FLOC. In this study, the steps of mosaicing are defined based on mosaicing frameworks to explore how these methods perform in terms of being able to mosaic a map of the placenta. The methods have been tested on non-fetoscopic acquired data as well as fetoscopic acquired data to create a relative measure in between the two. Three tests on non- fetoscopic data were performed to explore how well the methods handled mosaicing of data with distinctive characteristics. The same methods were then tested on unprocessed fetoscopic data before being tested on preprocessed fetoscopic data to see if the results were affected by external preprocessing. The results showed that there were differences in between the methods. SIFT and FAST showed that they have potential of mosaicing non-fetoscopic data of varying extent. SIFT gave an impression of being more robust during all of the tests. SIFT especially performed better during the tests on data with few potential keypoints which is an advantage when speaking of fetoscopic acquired data. SIFT also managed to mosaic a larger area than FAST when mosaicing preprocessed fetoscopic data. Preprocessing the data improved the mosaicing when using SIFT but further improvements are needed. / Denna studie utforskar hur robust en intressepunktsbeskrivare, scale invariant feature transform (SIFT), och enintressepunktsdetektor, feature accelerated segmentation test (FAST), hanterar digitala bilder insamlade av ett fetoskop med syfte att sy ihop dessa till en översiktskarta. Dagens behandling av tvillingtransfusionsyndrom vid Karolinska Universitetssjukhuset är fetoscopic guided laser occlusion of chorioangiopagous vessels (FLOC). Under denna fetoskopi bränner man och därmed blockerar korsande blodkärl som orsakar en transfusion (anastomoses) och obalans i blodomloppet mellan två tvillingfoster. Dessa blodkärl är lokaliserade på placentan. Fetoskopin omfattar navigering av en relativt stor area med ett begränsat synfält. Det begränsade synfältet under FLOC gör det svårt att orientera fetoskopet och identifiera korsande blodkärl som orsakar transfusionen. Syftet med studien är att utforska ett sätt att hantera komplikationerna med FLOC igenom att utforska sätt att skapa en översiktskarta av placentan under FLOC. Denna översiktskarta kan nyttjas under FLOC och därmed göra proceduren säkrare igenom att kartan underlättar orienteringen av fetoskopet och identifiering av orsakande blodkärl. I denna studie är stegen för att skapa en översiktskarta baserade på olika datorseende ramverk för att se hur dessa tillvägagångssätt presterar när det gäller att skapa en översiktskarta. Metoderna för att skapa en översiktskarta har testats på data insamlad med webkamera och data insamlad med fetoskop för att skapa en relativ uppfattning om hur de står sig beroende på indata. Tre tester på data insamlad med webkamera genomfördes för att utforska hur väl metoderna hanterade data med många potentiella intressepunkter, rörelse orsakad av handhållen enhet/kamera, repetitiva mönster, översiktskartor som resulterande i större upplösning, och liten möjlighet att hitta intressepunkter. Samma metoder testades sedan på icke behandlad data insamlad med fetoskop innan den testades på förbehandlad data insamlad med fetoskop för att se förbehandlingensnödvändighet och prestation. Resultaten visar att det är skillnader mellan de två metoderna använda i denna studie, både när det gäller data insamlad med fetoskop och webkamera. SIFT och FAST visar potential av olika grad när det gäller att skapa en översiktskarta med data insamlad av webkamera. SIFT visade sig vara mer robust under alla tester inklusive data insamlad med fetoskop. SIFT presterade speciellt bättre under testen som omfattade få antal möjliga intressepunkter vilket är en fördel när det gäller data insamlad med fetoskop. SIFT lyckades också skapa översiktskartor med större area än FAST när förbehandlad fetoskopisk data testades. När det gäller SIFT så visade resultaten en förbättring när data insamlad med fetoskop förbehandlades men att ytterligare förbättringar är nödvändiga.

This thesis presents new approaches to image registration and super-resolution mosaicing as well as their applications. Firstly, a feature-based image registration method is proposed for a multisensor surveillance system that consists of an optical camera and an infrared camera. By integrating a non-rigid object tracking technique into this method, a novel approach to simultaneous object tracking and multisensor image registration is proposed. Based on the registration and fusion of multisensor information, automatic face detection is greatly improved. Secondly, some extensions of a gradient-based image registration method, called inverse compositional algorithm, are proposed. These extensions include cumulative multi-image registration and the incorporation of illumination change and lens distortion correction. They are incorporated into the framework of the original algorithm in a consistent manner and efficiency can still be achieved for multi-image registration with illumination and lens distortion correction. Thirdly, new super-resolution mosaicing algorithms are proposed for multiple uncompressed and compressed images. Considering the process of image formation, observation models are introduced to describe the relationship between the superresolution mosaic image and the uncompressed and compressed low-resolution images. To improve the performance of super-resolution mosaicing, a wavelet-based image interpolation technique and an approach to adaptive determination of the regularization parameter are presented. For compressed images, a spatial-domain algorithm and a transform-domain algorithm are proposed. All the proposed superresolution mosaicing algorithms are robust against outliers. They can produce superresolution mosaics and reconstructed super-resolution images with improved subjective quality. Finally, new techniques for super-resolution sprite generation and super-resolution sprite coding are proposed. Considering both short-term and long-term motion influences, an object-based image registration method is proposed for handling long image sequences. In order to remove the influence of outliers, a robust technique for super-resolution sprite generation is presented. This technique produces sprite images and reconstructed super-resolution images with high visual quality. Moreover, it provides better reconstructed low-resolution images compared with low-resolution sprite generation techniques. Due to the advantages of the super-resolution sprite, a super-resolution sprite coding technique is also proposed. It achieves high coding efficiency especially at a low bit-rate and produces both decoded low-resolution and super-resolution images with improved subjective quality. Throughout this work, the performance of all the proposed algorithms is evaluated using both synthetic and real image sequences.

This thesis investigates the problem of developing new computer vision techniques for early detection of skin cancer. The first part of this work presents a novel methodology to correct color reproduction in dermatological images when different cameras and/or dermoscopes are used. Next, the problem of automatic full body mapping is addressed by proposing a mosaicing method based on an on-the-shelf digital compact camera and a set of markers. This method increases the possibilities of total body photography by taking the low-resolution images of a whole body exploration and automatically combining them into a high-resolution photomosaic. The third contribution of this work consists of the development of a full body scanner for acquiring cutaneous images. On one hand, the scanner reduces the long time-consuming examinations done in dermoscopy explorations, and on the other hand, it increases the resolution of total body photography systems. / En aquesta tesi s'investiga el desenvolupament de noves tècniques de visió per computador per a la detecció del càncer de pell. La primera part del treball presenta una nova metodologia per a la correcció del color en imatges dermatològiques quan s'utilitzen diferents càmeres i/o els dermatoscops. A continuació és proposa una solució al problema del registre automàtic d'imatges de cos complert amb la proposta d’un mètode de mosaicing basat en l'ús de càmeres compactes i un conjunt de markers. Incrementant les possibilitats de la fotografia de cos complert mitjançant la combinació automàtica d’imatges de baixa resuloció per a l'obtenció d'un fotomosaic d’alta resolució. La tercera contribució d'aquest treball consisteix en el desenvolupament d'un escàner de cos complert per a l'adquisició d'imatges cutànies. D'una banda l'escàner redueix el llarg temps necessari per a les exploracions dermatoscòpiques, i de l'altre, incrementa la resolució de la fotografia de cos complet.

In this thesis, we address the problem of panoramic scene construction in which a single image covering the entire visible area of the scene is constructed from an aerial image video. In the literature, there are several algorithms developed for construction of panoramic scene of a video sequence. These algorithms can be categorized as feature based and featureless algorithms. In this thesis, we concentrate on the feature based algorithms and comparison of these algorithms is performed for aerial videos. The comparison is performed on video sequences captured by non-stationary cameras, whose optical axis does not have to be the same. In addition, the matching and tracking performances of the algorithms are separately analyzed, their advantages-disadvantages are presented and several modifications are proposed.