This is a master thesis of the Master of Science degree program in Applied Physics and Electrical Engineering (Y) at Linköping University. The goal of the projectis to develop an application for creating a map in real time from a video camera on a miniature unmanned aerial vehicle. This thesis project and report is a first exploratory study for this application. It implements a prototype method and evaluates it on sample sequences from an on-board video camera. The method first looks for good points to follow in the image and then tracks them in a sequence.The image is then pasted, or merged, together with previous images so that points from the different images align. Two methods to find good points to follow are examined with focus on real-time performance. The result is that the much faster FAST detector method yielded satisfactory results good enough to replace the slower standard method of the Harris-Stephens corner detector. It is also examined whether it is possible to assume that the ground is a flat surface in this application or if a computationally more expensive method estimating altitude information has to be used. The result is that at high altitudes or when the ground is close to flat in reality and the camera points straight downwards a two-dimensional method will do. If flying lower or with high objects in the picture, which is often the case in this application, it must to be taken into account that the points really are at different heights, hence the ground can not be assumed to be flat.

Confocal microendoscopy is a promising new diagnostic imaging technique that is minimally invasive and provides in-vivo cellular-level images of tissue. In this study, we developed various image analysis techniques for ovarian cancer detection using the confocal microendoscope system. Firstly, we developed a technique for automatic classification of images based on focus, to prune out the out-of-focus images from the ovarian dataset. Secondly, we modified the texture analysis technique developed earlier to improve the stability of the textural features. The modified technique gives stable features and more consistent performance for ovarian cancer detection. Although confocal microendoscopy provides cellular-level resolution, it is limited by a small field of view. We present a fast technique for stitching the individual frames of the tissue to form a large mosaic. Such a mosaic will aid the physician in diagnosis, and also makes quantitative and statistical analysis possible on a larger field of view.

Le travail de thèse que nous présentons s’articule autour de l’utilisation de caméras motorisées à trois degrés de liberté, également appelées caméras PTZ. Ces caméras peuvent être pilotées suivant deux angles. L’angle de panorama (θ) permet une rotation autour d’un axe vertical et l’angle de tangage (ϕ) permet une rotation autour d’un axe horizontal. Si, théoriquement, ces caméras permettent donc une vue omnidirectionnelle, elles limitent le plus souvent la rotation suivant l’angle de panorama mais surtout suivant l’angle de tangage. En plus du pilotage des rotations, ces caméras permettent également de contrôler la distance focale permettant ainsi un degré de liberté supplémentaire. Par rapport à d’autres modèles, les caméras PTZ permettent de construire un panorama - représentation étendue d’une scène construite à partir d’une collection d’image - de très grande résolution. La première étape dans la construction d’un panorama est l’acquisition des différentes prises de vue. A cet effet, nous avons réalisé une étude théorique permettant une couverture optimale de la sphère à partir de surfaces rectangulaires en limitant les zones de recouvrement. Cette étude nous permet de calculer une trajectoire optimale de la caméra et de limiter le nombre de prises de vues nécessaires à la représentation de la scène. Nous proposons également différents traitements permettant d’améliorer sensiblement le rendu et de corriger la plupart des défauts liés à l’assemblage d’une collection d’images acquises avec des paramètres de prises de vue différents. Une part importante de notre travail a été consacrée au recalage automatique d’images en temps réel, c’est à dire que chaque étapes est effectuée en moins de 40ms pour permettre le traitement de 25 images par seconde. La technologie que nous avons développée permet d’obtenir un recalage particulièrement précis avec un temps d’exécution de l’ordre de 4ms (AMD1.8MHz). Enfin, nous proposons deux applications de suivi d’objets en mouvement directement issues de nos travaux de recherche. La première associe une caméra PTZ à un miroir sphérique. L’association de ces deux éléments permet de détecter tout objet en mouvement dans la scène puis de se focaliser sur l’un d’eux. Dans le cadre de cette application, nous proposons un algorithme de calibrage automatique de l’ensemble caméra et miroir. La deuxième application n’exploite que la caméra PTZ et permet la segmentation et le suivi des objets dans la scène pendant le mouvement de la caméra. Par rapport aux applications classiques de suivi de cible en mouvement avec une caméra PTZ, notre approche se différencie par le fait que réalisons une segmentation fine des objets permettant leur classification. / The thesis considers the of use motorized cameras with 3 degrees of freedom which are commonly called PTZ cameras. The orientation of such cameras is controlled according to two angles: the panorama angle (θ) describes the degree of rotation around on vertical axis and the tilt angle (ϕ) refers to rotation along a meridian line. Theoretically, these cameras can cover an omnidirectional field of vision of 4psr. Generally, the panorama angle and especially the tilt angle are limited for such cameras. In addition to control of the orientation of the camera, it is also possible to control focal distance, thus allowing an additional degree of freedom. Compared to other material, PTZ cameras thus allow one to build a panorama of very high resolution. A panorama is a wide representation of a scene built starting from a collection of images. The first stage in the construction of a panorama is the acquisition of the various images. To this end, we made a theoretical study to determine the optimal paving of the sphere with rectangular surfaces to minimize the number of zones of recovery. This study enables us to calculate an optimal trajectory of the camera and to limit the number of images necessary to the representation of the scene. We also propose various processing techniques which appreciably improve the rendering of the mosaic image and correct the majority of the defaults related to the assembly of a collection of images which were acquired with differing image capture parameters. A significant part of our work was used to the automatic image registration in real time, i.e. lower than 40ms. The technology that we developed makes it possible to obtain a particularly precise image registration with an computation time about 4ms (AMD1.8MHz). Our research leads directly to two proposed applications for the tracking of moving objects. The first involves the use of a PTZ camera and a spherical mirror. The combination of these two elements makes it possible to detect any motion object in the scene and to then to focus itself on one of them. Within the framework of this application, we propose an automatic algorithm of calibration of the system. The second application exploits only PTZ camera and allows the segmentation and the tracking of the objects in the scene during the movement of the camera. Compared to the traditional applications of motion detection with a PTZ camera, our approach is different by the fact that it compute a precise segmentation of the objects allowing their classification.

Cette thèse propose un algorithme de mosaïquage pour la construction d'images panoramiques des parois internes de la vessie. Le temps de construction de ces images correspondant aux parties intéressantes de la vessie doit être inférieur à la durée d'un examen clinique standard. La méthode de mosaïquage doit aussi être robuste vis-à-vis des variabilités inter-examens liées aux patients et aux instruments. Ces images panoramiques pourront être utilisées par le clinicien comme référence pour guider des examens ultérieurs, pour l'archivage des données et pour suivre l'évolution des lésions. La première étape de l'algorithme est le pré-traitement des images cystoscopiques consistant en l'atténuation des inhomogénéités d'illumination et du motif de fibres optiques visible dans les images acquises par un fibroscope. La deuxième étape est le recalage des images. La solution retenue consiste en la corrélation par les transformées de Fourier des images qui fournit des translations initiales à un algorithme itératif basé sur la différence d'intensité entre les images. Ce dernier délivre les paramètres de la transformation perspective reliant deux images successives de la séquence. Dans la troisième étape nous projetons les images dans un repère commun en utilisant des transformations globales calculées avec les résultats des recalages. Nous utilisons un moyennage pondéré des intensités des pixels pour atténuer les bords visibles lors de la projection. Les résultats quantitatifs obtenus avec un fantôme et des résultats qualitatifs calculés pour des séquences réelles montrent que notre approche automatique de mosaïquage est robuste et rapide (temps compatible avec la durée d'un examen cystoscopique clinique). Nos tests ont également prouvé que l'algorithme de recalage fonctionne pour des transformations géométriques plus grandes que celles rencontrées typiquement entre images d'une séquence vidéo (90% de recouvrement entre images consécutives pour ces dernières) / This work describes a mosaicing algorithm for constructing panoramic images of internal walls of the bladder. Time relating to the construction of panoramic images including the interesting parts must be shorter than that required by a standard cystoscopic examination. The mosaicing algorithm must be robust against lighting conditions, morphologic and texture variations relating to instruments and patient anatomy. These panoramic images could be used by a clinician for guiding further exams, storing non-redundant data and following-up evolution of lesions. The preprocessing of cystoscopic images is the first stage of the algorithm. Preprocessing consists of shading correction and fiber optics pattern attenuation occurring in fiberscope acquired images. The second stage is image registration. The chosen solution consist of cross-correlating images (using their Fourier transforms) in order to have initial translations for an iterative registration algorithm based on the sum of squared differences of images. In the third stage, images are projected in the coordinate system of the panoramic image using global transformations computed with matrices given by the iterative registration. We use a weighted average of pixel intensities to blend visible borders of images produced in the projection process. Numerical results obtained with a phantom and qualitative results obtained with real sequences show that our automatic approach is robust and allows for a fast construction of panoramic images in a period of time that is shorter than the duration of a clinical cystoscopic examination. Our experiments showed that the registration algorithm can handle geometric transformations that are larger than those existing typically in a video-sequence (90% of superposition between successive images in this case)

In this thesis, we address the problem of panoramic scene construction in which a single image covering the entire visible area of the scene is constructed from an aerial image video. In the literature, there are several algorithms developed for construction of panoramic scene of a video sequence. These algorithms can be categorized as feature based and featureless algorithms. In this thesis, we concentrate on the feature based algorithms and comparison of these algorithms is performed for aerial videos. The comparison is performed on video sequences captured by non-stationary cameras, whose optical axis does not have to be the same. In addition, the matching and tracking performances of the algorithms are separately analyzed, their advantages-disadvantages are presented and several modifications are proposed.

This project is mainly based on mosaicing of images and similarity measurements with different methods. The map of a floor is created from a database of small-images that have been captured by a camera-mounted robot scanning the wooden floor of a living room. We call this ground mapping. After the ground mapping, the robot can achieve self-positioning on the map by using novel small images it captures as it displaces on the ground. Similarity measurements based on the Schwartz inequality have been used to achieve the ground mapping, as well as to position the robot once the ground map is available. Because the natural light affects the gray value of images, this effect must be accounted for in the envisaged similarity measurements. A new approach to mosaicing is suggested. It uses the local texture orientation, instead of the original gray values, in ground mapping as well as in positioning. Additionally, we report on ground mapping results using other features, gray-values as features. The robot can find its position with few pixel errors by using the novel approach and similarity measurements based on the Schwartz inequality.

No description available.

La cystoscopie est l’examen de référence pour le diagnostic et le traitement du cancer de la vessie. Le champ de vue (CdV) réduit des endoscopes complique le diagnostic et le suivi des lésions. Les mosaïques d’images sont une solution à ce problème car elles visualisent des CdV étendus. Toutefois, pour la vessie, le mosaïque d’images est un véritable défi à cause du faible contraste dans les images, des textures peu prononcées, de la variabilité intra- et inter-patient et des changements d’illumination dans les séquences. Ce défi est également à relever dans d’autres modalités endoscopiques ou dans des scènes non médicales comme les vidéos sous-marines. Dans cette thèse, une énergie variationnelle totale a d’abord été minimisée à l’aide d’un algorithme primal-dual du premier ordre pour obtenir un flot optique fournissant une correspondance dense et précise entre les points homologues des paires d’images. Les correspondances sont ensuite utilisées pour déterminer les paramètres des transformations requises pour le placement des images dans le repère global de la mosaïque. Les méthodes proposées pour l’estimation du flot optique dense incluent un terme d’attache aux données qui minimise le nombre des vecteurs aberrants et un terme de régularisation conçu pour préserver les discontinuités du champ devecteurs. Un algorithme de flot optique qui est robuste vis-à-vis de changements d’illumination importants (et utilisable pour différentes modalités) a également été développé dans ce contexte. La précision et la robustesse des méthodes de recalage proposées ont été testées sur des jeux de données (de flot optique) publiquement accessibles et sur des fantômes de vessies et de la peau. Des résultats sur des données patients acquises avec des cystoscopes rigides et flexibles, en lumière blanche ou en fluorescence, montrent la robustesse des algorithmes proposés. Ces résultats sont complétés par ceux obtenus pour d’autres séquences endoscopiques réelles de dermatoscopie, de scène sous-marine et de données d’exploration spatiale. / Cystoscopy is the reference procedure for the diagnosis and treatment of bladder cancer. The small field of view (FOV) of endoscopes makes both the diagnosis and follow-up of lesions difficult. Image mosaics are a solution to this problem since they visualize large FOVs of the bladder scene. However, due to low contrast, weak texture, inter- and intra-patient texture variability and illumination changes in these image sequences, the task of image mosaicing becomes challenging. This is also a major concern in other endoscopic data and non-medical scenes like underwater videos. In this thesis, a total variational energy has been first minimized using a first-order primal-dual algorithm in convex optimization to obtain optical flow vector fields giving a dense and accurate correspondence between homologous points of the image pairs. The correspondences are then used to obtain transformation parameters for registering the images to one global mosaic coordinate system. The proposed methods for dense optical flow estimation include a data-term which is modeled to minimize at most the outliers and a regularizer which is designed to preserve at their best the flow field discontinuities. An optical flow algorithm, which is robust to strong illumination changes (and which suits to different modalities), has also been developed in this framework. The registration accuracy and robustness of the proposed methods are tested on both publicly available datasets for optical flow estimation and on simulated bladder and skin phantoms. Results on patient data acquired with rigid and flexible cystoscopes under the white light and the fluorescence modality show the robustness of the proposed approaches. These results are also complemented with those of other real endoscopic data, dermoscopic sequences, underwater scenes and space exploration data.

Submitted by Angela Maria de Oliveira (amolivei@uepg.br) on 2018-06-19T17:15:59Z No. of bitstreams: 2 license_rdf: 811 bytes, checksum: e39d27027a6cc9cb039ad269a5db8e34 (MD5) PEDRO HENRIQUE SOARES DE ALMEIDA.pdf: 8669671 bytes, checksum: bf6252d5566d0b626215c08edce94dca (MD5) / Made available in DSpace on 2018-06-19T17:15:59Z (GMT). No. of bitstreams: 2 license_rdf: 811 bytes, checksum: e39d27027a6cc9cb039ad269a5db8e34 (MD5) PEDRO HENRIQUE SOARES DE ALMEIDA.pdf: 8669671 bytes, checksum: bf6252d5566d0b626215c08edce94dca (MD5) Previous issue date: 2018-05-15 / O mosaico de imagens é o alinhamento de múltiplas imagens em composições maiores que representam partes de uma cena 3D. Diversos algoritmos de mosaico de imagens foram propostos nas últimas duas décadas. Ao mesmo tempo, o advento contínuo de novos métodos de mosaico torna muito difícil escolher um algoritmo apropriado para uma finalidade específica. Este trabalho teve por objetivo avaliar métodos de mosaico baseados em característica de baixo nível utilizando imagens agrícolas obtidas por meio de Aeronave Remotamente Pilotada (RPA). Algoritmos detectores de característica de baixo nível podem ser invariantes à escala e rotação, dentre outras transformações que comumente ocorrem em imagens agrícolas obtidas por meio de RPA. O detector de cantos de Harris, detector de cantos FAST, detector de característica SIFT e detector SURF foram avaliados de acordo com o desempenho computacional e a qualidade do mosaico gerado. Para avaliar o desempenho, foram levados em consideração fatores como a média de características detectadas por imagem, o número de imagens utilizadas para compor o mosaico e o tempo de processamento (tempo de usuário ou user time). Para avaliar a qualidade, os mosaicos gerados pelos métodos foram utilizados para estimar a severidade da ferrugem asiática da soja e uma comparação com o software comercial Pix4Dmapper foi realizada. Em relação à qualidade, não houve diferença significativa e todos os métodos demonstraram estar no mesmo patamar. O detector SURF, dentre todos os métodos, obteve o pior desempenho utilizando, em média, apenas 33,1% das imagens de entrada para compor os mosaicos. O detector de cantos de Harris mostrou-se como a solução mais rápida, chegando a ser 7,27% mais rápido para compor o mosaico. Porém, em seu último mosaico gerado, o aproveitamento das imagens de entrada foi pobre: apenas 52%. O detector de cantos FAST obteve o melhor aproveitamento das imagens de entrada, porém, descontinuidades significativas de objetos ocorreram em seu último mosaico gerado. Além disso, obteve um tempo de processamento consideravelmente superior ao dos demais métodos, chegando a ser 6,42 vezes mais lento para compor o mosaico. O detector de característica SIFT obteve o segundo melhor tempo de processamento e o segundo melhor aproveitamento das imagens de entrada, sem apresentar problemas de descontinuidades de objetos. Portanto, mostrou-se como o método mais adequado para imagens agrícolas obtidas por meio de RPA. / Image mosaicing is the alignment of multiple images into larger compositions which represent portions of a 3D scene. A number of image mosaicing algorithms have been proposed over the last two decades. At the same time, the continuous advent of new mosaicing methods in recent years makes it really difficult to choose an appropriate mosaicing algorithm for a specific purpose. This study aimed to evaluate low level feature-based mosaicing methods using agricultural images obtained by Remotely Piloted Aircraft (RPA). Low-level feature detecting algorithms can be invariant to scale and rotation, among other transformations that commonly occur in agricultural images obtained by RPA. Harris corner detector, FAST corner detector, SIFT feature detector and SURF detector were evaluated according to the computational performance and the quality of the generated mosaic. To evaluate computational performance, were taken into account factors such as the detected features average per image, the number of images used to compose the mosaic and the processing time (user time). To evaluate quality, the mosaics generated by each method were used to estimate the Asian soybean rust severity and a comparison with the commercial software Pix4Dmapper was performed. Regarding quality, there was no significant difference and all methods proved to be on the same level. SURF detector, among all methods, got the worst performance using, on average, only 33.1% of the input images to compose the mosaics. Harris corner detector proved to be the fastest solution, becoming 7.27% faster to compose the mosaic. However, in its final mosaic, the use of the input images was poor: only 52%. FAST corner detector had the best utilization of the input images, however, significant discontinuities of objects occurred in its final mosaic. In addition, it had a considerably longer processing time than the other methods, becoming 6.42 times slower to compose the mosaic. SIFT feature detector had the second best processing time and the second best utilization of the input images, without presenting object discontinuities problems. Therefore, presented itself as the most suitable method for agricultural images obtained by RPA.

Large scale image mosaicing methods are in great demand among scientists who study different aspects of the seabed, and have been fostered by impressive advances in the capabilities of underwater robots in gathering optical data from the seafloor. Cost and weight constraints mean that lowcost Remotely operated vehicles (ROVs) usually have a very limited number of sensors. When a low-cost robot carries out a seafloor survey using a down-looking camera, it usually follows a predetermined trajectory that provides several non time-consecutive overlapping image pairs. Finding these pairs (a process known as topology estimation) is indispensable to obtaining globally consistent mosaics and accurate trajectory estimates, which are necessary for a global view of the surveyed area, especially when optical sensors are the only data source. This thesis presents a set of consistent methods aimed at creating large area image mosaics from optical data obtained during surveys with low-cost underwater vehicles. First, a global alignment method developed within a Feature-based image mosaicing (FIM) framework, where nonlinear minimisation is substituted by two linear steps, is discussed. Then, a simple four-point mosaic rectifying method is proposed to reduce distortions that might occur due to lens distortions, error accumulation and the difficulties of optical imaging in an underwater medium. The topology estimation problem is addressed by means of an augmented state and extended Kalman filter combined framework, aimed at minimising the total number of matching attempts and simultaneously obtaining the best possible trajectory. Potential image pairs are predicted by taking into account the uncertainty in the trajectory. The contribution of matching an image pair is investigated using information theory principles. Lastly, a different solution to the topology estimation problem is proposed in a bundle adjustment framework. Innovative aspects include the use of fast image similarity criterion combined with a Minimum spanning tree (MST) solution, to obtain a tentative topology. This topology is improved by attempting image matching with the pairs for which there is the most overlap evidence. Unlike previous approaches for large-area mosaicing, our framework is able to deal naturally with cases where time-consecutive images cannot be matched successfully, such as completely unordered sets. Finally, the efficiency of the proposed methods is discussed and a comparison made with other state-of-the-art approaches, using a series of challenging datasets in underwater scenarios / Els mètodes de generació de mosaics de gran escala gaudeixen d’una gran demanda entre els científcs que estudien els diferents aspectes del fons submarí, afavorida pels impressionants avenços en les capacitats dels robots submarins per a l’obtenció de dades ptiques del fons. El cost i el pes constitueixen restriccions que impliquen que els vehicles operats remotament disposin habitualment d’un nombre limitat de sensors. Quan un robot de baix cost duu a terme una exploració del fons submarí utilitzant una càmera apuntant cap al terreny, aquest segueix habitualment una trajectòria que dóna com a resultat diverses parelles d’imatges amb superposició de manera sequencial. Trobar aquestes parelles (estimació de la topologia) és una tasca indispensable per a l’obtenció de mosaics globalment consistents així com una estimació de trajectòria precisa, necessària per disposar d’una visió global de la regió explorada, especialment en el cas en què els sensors òptics constitueixen la única font de dades. Aquesta tesi presenta un conjunt de mètodes robustos destinats a la creació de mosaics d’àrees de grans dimensions a partir de dades òptiques (imatges) obtingudes durant exploracions realitzades amb vehicles submarins de baix cost. En primer lloc, es presenta un mètode d’alineament global desenvolupat en el context de la generació de mosaics basat en característiques 2D, substituint una minimització no lineal per dues etapes lineals. Així mateix, es proposa un mètode simple de rectificació de mosaics basat en quatre punts per tal de reduir les distorsions que poden aparèixer a causa de la distorsió de les lents, l’acumulació d’errors i les dificultats d’adquisició d’imatges en el medi submarí. El problema de l’estimació de la topologia s’aborda mitjanant la combinació d’un estat augmentat amb un altre de Kalman estès, amb l’objectiu de minimitzar el nombre total d’intents de cerca de correspondències i obtenir simultàniament la millor trajectòria possible. La predicció de les parelles d’imatges potencials té en compte la incertesa de la trajectòria, i la contribució de l’obtenció de correspondències per a un parell d’imatges s’estudia d’acord amb principis de la teoria de la informació. Així mateix, el problema de l’estimació de la topologia és abordat en el context d’un alineament global. Les innovacions inclouen l’ús d’un criteri ràpid per a determinació de la similitud entre imatges combinat amb una solució basada en arbres d’expansió mínima, per tal d’obtenir una topologia provisional. Aquesta topologia és millorada mitjançant l’intent de cerca de correspondències entre parelles d’imatges amb major probabilitat de superposició. Contràriament al que succeïa en solucions prèvies per a la construcció de mosaics de grans àrees, el nostre entorn de treball és capaç de tractar amb casos en què imatges consecutives en el temps no han pogut ser relacionades satisfactòriament, com és el cas de conjunts d’imatges totalment desordenats. Finalment, es discuteix l’eficiència del mètode proposat i es compara amb altres solucions de l’estat de l’art, utilitzant una sèrie de conjunts de dades complexos en escenaris subaquàtics.