<p>This thesis presents work done within the EC-founded project VISATEC. Due to the different directions of the VISATEC project this thesis has a few different threads.</p><p>A novel presentation scheme for medium level vision features applied to range sensor data and to image sequences. Some estimation procedures for this representation have been implemented and tested. The representation is tensor based and uses higher order tensors in a projective space. The tensor can hold information on several local structures including their relative position and orientation. This information can also be extracted from the tensor.</p><p>A number of well-known techniques are combined in a novel way to be able to perform object pose estimation under changes of the object in position, scale and rotation from a single 2D image. The local feature used is a patch which is resampled in a log-polar pattern. A number of local features are matched to a database and the k nearest neighbors vote an object state parameters. This most probable object states are found through mean-shift clustering.</p><p>A system using multi-cue integration as a means of reaching a higher level of system-level robustness and a higher lever of accuracy is developed and evaluated in an industrial-like-setting. The system is based around a robotic manipulator arm with an attached camera. The system is designed to solve parts of the bin-picking problem. The above mentioned 2D technique for object pose estimation is also evaluated within this system.</p>

No description available.

Algoritmos de detecção de movimento em seqüências de imagens devem satisfazer os requisitos de precisão, robustez e velocidade de processamento. A forma de combinar estes três itens depende do desenvolvimento do algoritmo e da aplicação a que se destina, sem que os itens de robustez e precisão sejam comprometidos. Neste trabalho investigamos técnicas para detecção do movimento que satisfazem tais requisitos. A técnica escolhida para detecção de movimento foi a do fluxo Ótico (FO) devido as suas características de precisão nos resultados. Como esta técnica exige elevado esforço computacional, propõe-se nesta tese uma metodologia que aplica as equações de fluxo ótico em reduzidas áreas da imagem processada. Estas áreas são selecionadas utilizando algumas técnicas de pré-processamento que identificam regiões da imagem com maior probabilidade de movimento presente. Posteriormente a esta identificação são aplicadas as equações de FO nas regiões de interesse. Para avaliação e validação do método proposto, comparam-se os diagramas de agulhas resultantes das áreas reduzidas aos diagramas resultantes de toda a imagem mediante critérios estatísticos e de tempo de processamento envolvido. Os algoritmos são testados utilizando imagens sintéticas e imagens reais. / Algorithms for motion detection in image sequences must satisfy the following requirements: accuracy, robustness and speed. The way that accuracy, robustness and speed are combined depends on the algorithm development and on the application. In this work, it has investigated motion detection techniques that satisfy the mentioned requirements. The Optical Flow technique was chosen for the motion detection due to its good performance in the results. As the Optical Flow requires intensive computational load, we propose in this thesis a methodology where Optical Flow Equations are applied in specific areas of the processed image. These areas were selected using pre-processing techniques that identify regions of image with larger motion probability. After the motion areas identification, Optical Flow Equations are applied to the regions of interest. To assess and validate the proposed method, the needle diagrams obtained in the reduced areas are compared with the ones obtained from the whole image according to statistical criteria and processing time. The proposed algorithms have been tested in synthetic and real images.

Les séquences fluoroscopiques, acquises à de faibles doses de rayons X, sont utilisées au cours de certaines opérations médicales pour guider le personnel médical dans ces actes. Cependant, la qualité des images obtenues est inversement proportionnelle à cette dose. Nous proposons dans ces travaux un algorithme de réduction de bruit permettant de compenser les effets d'une réduction de la dose d'acquisition et donc garantissant une meilleure protection pour le patient et le personnel médical. Le filtrage développé est un filtre spatio-temporel s'appuyant sur les représentations multi-échelles 2D des images de la séquence pour de meilleures performances. Le filtre temporel récursif d'ordre 1 et compensé en mouvement permet une forte réduction de bruit. Il utilise une détection et un suivi des objets de la séquence. Ces deux étapes déterminent le filtrage spatio-temporel de chaque coefficient multi-échelles. Le filtrage spatial est un seuillage contextuel utilisant le voisinage multi-échelles des coefficients pour éviter l'apparition d'artefacts de forme dans les images reconstruites. La méthode proposée est testée dans deux espaces multi-échelles différents, les curvelets et les ondelettes complexes suivant l'arbre dual. Elle offre des performances supérieures à celles des meilleures méthodes de l'état de l'art. / Acquired with low doses of X-rays, fluoroscopic sequences are used to guide the medical staff during some medical procedures. However, image quality is inversely proportional to acquisition doses. We present here a noise reduction algorithm compensating for the effects of an acquisition at a reduced dose. Such a reduction enables better health protection for the patient as well as for the medical staff. The proposed method is based on a spatio-temporal filter applied on the 2D multi-scales representations of the sequence images to allow for a greater noise reduction. The motion-compensated, recursive filter acccounts for most of the noise reduction. It is composed of a detection and pairing step, which output determines how a coefficient is filtered. Spatial filtering is based on a contextual thresholding to avoid introducing shape-like artifacts. We compare this filtering both in the curvelet and dual-tree complex wavelet domains and show it offers better results than state-of-the-art methods.

Algoritmos de detecção de movimento em seqüências de imagens devem satisfazer os requisitos de precisão, robustez e velocidade de processamento. A forma de combinar estes três itens depende do desenvolvimento do algoritmo e da aplicação a que se destina, sem que os itens de robustez e precisão sejam comprometidos. Neste trabalho investigamos técnicas para detecção do movimento que satisfazem tais requisitos. A técnica escolhida para detecção de movimento foi a do fluxo Ótico (FO) devido as suas características de precisão nos resultados. Como esta técnica exige elevado esforço computacional, propõe-se nesta tese uma metodologia que aplica as equações de fluxo ótico em reduzidas áreas da imagem processada. Estas áreas são selecionadas utilizando algumas técnicas de pré-processamento que identificam regiões da imagem com maior probabilidade de movimento presente. Posteriormente a esta identificação são aplicadas as equações de FO nas regiões de interesse. Para avaliação e validação do método proposto, comparam-se os diagramas de agulhas resultantes das áreas reduzidas aos diagramas resultantes de toda a imagem mediante critérios estatísticos e de tempo de processamento envolvido. Os algoritmos são testados utilizando imagens sintéticas e imagens reais. / Algorithms for motion detection in image sequences must satisfy the following requirements: accuracy, robustness and speed. The way that accuracy, robustness and speed are combined depends on the algorithm development and on the application. In this work, it has investigated motion detection techniques that satisfy the mentioned requirements. The Optical Flow technique was chosen for the motion detection due to its good performance in the results. As the Optical Flow requires intensive computational load, we propose in this thesis a methodology where Optical Flow Equations are applied in specific areas of the processed image. These areas were selected using pre-processing techniques that identify regions of image with larger motion probability. After the motion areas identification, Optical Flow Equations are applied to the regions of interest. To assess and validate the proposed method, the needle diagrams obtained in the reduced areas are compared with the ones obtained from the whole image according to statistical criteria and processing time. The proposed algorithms have been tested in synthetic and real images.

This thesis presents work done within the EC-founded project VISATEC. Due to the different directions of the VISATEC project this thesis has a few different threads. A novel presentation scheme for medium level vision features applied to range sensor data and to image sequences. Some estimation procedures for this representation have been implemented and tested. The representation is tensor based and uses higher order tensors in a projective space. The tensor can hold information on several local structures including their relative position and orientation. This information can also be extracted from the tensor. A number of well-known techniques are combined in a novel way to be able to perform object pose estimation under changes of the object in position, scale and rotation from a single 2D image. The local feature used is a patch which is resampled in a log-polar pattern. A number of local features are matched to a database and the k nearest neighbors vote an object state parameters. This most probable object states are found through mean-shift clustering. A system using multi-cue integration as a means of reaching a higher level of system-level robustness and a higher lever of accuracy is developed and evaluated in an industrial-like-setting. The system is based around a robotic manipulator arm with an attached camera. The system is designed to solve parts of the bin-picking problem. The above mentioned 2D technique for object pose estimation is also evaluated within this system.

Die vorliegende Arbeit beschäftigt sich mit der Bestimmung von räumlich und zeitlich hochaufgelösten Bewegungsvektorfeldern von Gletschern aus monokularen Bildsequenzen. Diese stellen eine wertvolle Grundlage für glaziologische Analysen des Bewegungsverhaltens von Gletschern dar. Im Rahmen der Arbeit wurden Bildsequenzmessungen an fünf schnellfließenden Gletschern im Bereich der Diskobucht in Westgrönland durchgeführt. Insbesondere erfolgte die Aufnahme von Bildsequenzen und multi-temporalen Laserscannerdaten am Jakobshavn Isbræ, einem der schnellsten und produktivsten Gletscher Grönlands. Diese Messungen bilden die Datengrundlage der Arbeit. Es werden Messkonzepte zur Aufnahme der entsprechenden Bildsequenzen und multi-temporalen Laserscans bereitgestellt sowie Methoden entwickelt, um die Auswertung dieser Daten nach dem Prinzip der monoskopischen Bildsequenzanalyse zu ermöglichen. Die Bildsequenzen und multi-temporalen Laserscans werden von einem festen Standpunkt aus aufgenommen. Die Ableitung von Bewegungsvektorfeldern erfolgt dann durch eine automatische Zuordnung von Grauwertmustern in den Bildsequenzen bzw. durch die Zuordnung von 3D-Punktmustern in den multi-temporalen Laserscannerdaten. Bestehende Punktzuordnungsmethoden werden einerseits an die besonderen Eigenschaften der Gletscherdaten angepasst, andererseits werden geeignete Methoden zur Lösung von Detailproblemen neu entwickelt. Die Methodik der Bildsequenzanalyse wird dabei vor allem hinsichtlich ihrer Robustheit – beispielsweise gegenüber durch Schattenwurf verursachten Störungen im Bild – optimiert und es werden Bewegungseffekte in den Bildsequenzen korrigiert, die durch die Eigenbewegung der Kamera verursacht werden. Bei der Entwicklung der Methodik zur Analyse multitemporaler Laserscannerdaten werden vor allem Effekte berücksichtigt, die durch das sequenzielle Aufnahmeprinzip eines Scanners auftreten. Auf Basis der entwickelten und implementierten Methodik erfolgt die Auswertung der aufgenommenen Bildsequenzen und multi-temporalen Laserscans. Das Ergebnis der monoskopischen Bildsequenzauswertung ist ein dichtes Raster an Bewegungskurven für jede Bildsequenz. Die einzelnen Translationen der Bewegungskurven können mit einer Genauigkeit von einigen Zentimetern bis zu einem Dezimeter bestimmt werden. Die Auswertung der Laserscannerdaten liefert räumlich hochaufgelöste digitale Geländemodelle der Gletscheroberfläche sowie ein dichtes Raster von 3D-Bewegungsvektoren, deren Genauigkeit im Dezimeterbereich liegt. Anhand von Beispielen wird gezeigt, dass sich die aus monokularen Bildsequenzen abgeleiteten Bewegungsvektorfelder zur Bestimmung frontnaher Geschwindigkeitsfelder mit hoher räumlicher Auflösung, zur Ableitung der Lage und der Migration der Aufsetzlinie aus gezeiteninduzierten Vertikalbewegungen sowie zur Untersuchung des Geschwindigkeitsverhalten von Gletschern bei Kalbungsereignissen eignen. Aus den Laserscannerdaten können hochaufgelöste digitale Geländemodelle zur Dokumentation von Fronthöhen und Gletscherstrukturen abgeleitet werden, zudem eignen sie sich zur Bestimmung von Geschwindigkeitsfeldern, die eine sehr hohe räumliche Auflösung besitzen. / This research aims to determine the motion vector fields of glaciers with high spatial and temporal resolution. These vector fields can be derived from monocular image sequences and are a valuable data source for glaciological analysis of the motion behaviour of glaciers. Image sequence measurements have been conducted at five fast-flowing glaciers in the Disko Bay region in western Greenland. Especially at the Jakobshavn Isbræ – one of the fastest and most productive glaciers in Greenland – numerous image sequences have been recorded, as well as multi-temporal laser scanner data sets. These measurements provide the basic data sets for this thesis. The measurement concepts for the acquisition of image sequences and multi-temporal laser scans are presented, and procedures for the processing of the recorded data are developed, based on the principle of monoscopic image sequence analysis. Both the image sequences and multi-temporal laser scans are acquired statically. Motion vector fields can be derived by applying automatic co-registration methods on grey value patterns in the image sequences and on 3D point patterns in the laser scanner datasets respectively. Thus, standard matching techniques have been adapted to the special characteristics of the glacier data, and suitable methods that solve detail problems have been developed in addition. The method of the image sequence analysis has been optimised with respect to its robustness against errors caused by moving shadows. Furthermore, motion effects caused by small instabilities in the camera setup have been corrected. Regarding the analysis of multi-temporal laser scanner data, effects that occur because of the sequential acquisition principle of a laser scanner must also be considered. Based on the developed method, the image sequences and multi temporal laser scans have been processed. The result of the monoscopic image sequence analysis is a dense raster of trajectories for each image sequence. Each translation component from these trajectories can be determined with an accuracy of some centimeters up to one decimetre. The processing of the laser scanner data provides digital surface models of the glacier with high spatial resolution, and a dense raster of 3D motion vectors with accuracy in the range of decimetres. Specific examples show that motion vector fields derived from monocular image sequences can be used for the determination of high resolution velocity fields of glaciers, for the determination of the position and migration of the grounding line and for the investigation of a glacier’s motion behaviour during calving events. From the multi-temporal laser scanner data, velocity fields with high spatial resolution can be derived as well as digital surface models from single scans that document glacier front heights and glacier structures.

Die vorliegende Arbeit beschäftigt sich mit der Bestimmung von räumlich und zeitlich hochaufgelösten Bewegungsvektorfeldern von Gletschern aus monokularen Bildsequenzen. Diese stellen eine wertvolle Grundlage für glaziologische Analysen des Bewegungsverhaltens von Gletschern dar. Im Rahmen der Arbeit wurden Bildsequenzmessungen an fünf schnellfließenden Gletschern im Bereich der Diskobucht in Westgrönland durchgeführt. Insbesondere erfolgte die Aufnahme von Bildsequenzen und multi-temporalen Laserscannerdaten am Jakobshavn Isbræ, einem der schnellsten und produktivsten Gletscher Grönlands. Diese Messungen bilden die Datengrundlage der Arbeit. Es werden Messkonzepte zur Aufnahme der entsprechenden Bildsequenzen und multi-temporalen Laserscans bereitgestellt sowie Methoden entwickelt, um die Auswertung dieser Daten nach dem Prinzip der monoskopischen Bildsequenzanalyse zu ermöglichen. Die Bildsequenzen und multi-temporalen Laserscans werden von einem festen Standpunkt aus aufgenommen. Die Ableitung von Bewegungsvektorfeldern erfolgt dann durch eine automatische Zuordnung von Grauwertmustern in den Bildsequenzen bzw. durch die Zuordnung von 3D-Punktmustern in den multi-temporalen Laserscannerdaten. Bestehende Punktzuordnungsmethoden werden einerseits an die besonderen Eigenschaften der Gletscherdaten angepasst, andererseits werden geeignete Methoden zur Lösung von Detailproblemen neu entwickelt. Die Methodik der Bildsequenzanalyse wird dabei vor allem hinsichtlich ihrer Robustheit – beispielsweise gegenüber durch Schattenwurf verursachten Störungen im Bild – optimiert und es werden Bewegungseffekte in den Bildsequenzen korrigiert, die durch die Eigenbewegung der Kamera verursacht werden. Bei der Entwicklung der Methodik zur Analyse multitemporaler Laserscannerdaten werden vor allem Effekte berücksichtigt, die durch das sequenzielle Aufnahmeprinzip eines Scanners auftreten. Auf Basis der entwickelten und implementierten Methodik erfolgt die Auswertung der aufgenommenen Bildsequenzen und multi-temporalen Laserscans. Das Ergebnis der monoskopischen Bildsequenzauswertung ist ein dichtes Raster an Bewegungskurven für jede Bildsequenz. Die einzelnen Translationen der Bewegungskurven können mit einer Genauigkeit von einigen Zentimetern bis zu einem Dezimeter bestimmt werden. Die Auswertung der Laserscannerdaten liefert räumlich hochaufgelöste digitale Geländemodelle der Gletscheroberfläche sowie ein dichtes Raster von 3D-Bewegungsvektoren, deren Genauigkeit im Dezimeterbereich liegt. Anhand von Beispielen wird gezeigt, dass sich die aus monokularen Bildsequenzen abgeleiteten Bewegungsvektorfelder zur Bestimmung frontnaher Geschwindigkeitsfelder mit hoher räumlicher Auflösung, zur Ableitung der Lage und der Migration der Aufsetzlinie aus gezeiteninduzierten Vertikalbewegungen sowie zur Untersuchung des Geschwindigkeitsverhalten von Gletschern bei Kalbungsereignissen eignen. Aus den Laserscannerdaten können hochaufgelöste digitale Geländemodelle zur Dokumentation von Fronthöhen und Gletscherstrukturen abgeleitet werden, zudem eignen sie sich zur Bestimmung von Geschwindigkeitsfeldern, die eine sehr hohe räumliche Auflösung besitzen. / This research aims to determine the motion vector fields of glaciers with high spatial and temporal resolution. These vector fields can be derived from monocular image sequences and are a valuable data source for glaciological analysis of the motion behaviour of glaciers. Image sequence measurements have been conducted at five fast-flowing glaciers in the Disko Bay region in western Greenland. Especially at the Jakobshavn Isbræ – one of the fastest and most productive glaciers in Greenland – numerous image sequences have been recorded, as well as multi-temporal laser scanner data sets. These measurements provide the basic data sets for this thesis. The measurement concepts for the acquisition of image sequences and multi-temporal laser scans are presented, and procedures for the processing of the recorded data are developed, based on the principle of monoscopic image sequence analysis. Both the image sequences and multi-temporal laser scans are acquired statically. Motion vector fields can be derived by applying automatic co-registration methods on grey value patterns in the image sequences and on 3D point patterns in the laser scanner datasets respectively. Thus, standard matching techniques have been adapted to the special characteristics of the glacier data, and suitable methods that solve detail problems have been developed in addition. The method of the image sequence analysis has been optimised with respect to its robustness against errors caused by moving shadows. Furthermore, motion effects caused by small instabilities in the camera setup have been corrected. Regarding the analysis of multi-temporal laser scanner data, effects that occur because of the sequential acquisition principle of a laser scanner must also be considered. Based on the developed method, the image sequences and multi temporal laser scans have been processed. The result of the monoscopic image sequence analysis is a dense raster of trajectories for each image sequence. Each translation component from these trajectories can be determined with an accuracy of some centimeters up to one decimetre. The processing of the laser scanner data provides digital surface models of the glacier with high spatial resolution, and a dense raster of 3D motion vectors with accuracy in the range of decimetres. Specific examples show that motion vector fields derived from monocular image sequences can be used for the determination of high resolution velocity fields of glaciers, for the determination of the position and migration of the grounding line and for the investigation of a glacier’s motion behaviour during calving events. From the multi-temporal laser scanner data, velocity fields with high spatial resolution can be derived as well as digital surface models from single scans that document glacier front heights and glacier structures.

Thesis (MScEng (Mathematical Sciences)--University of Stellenbosch, 2009. / ENGLISH ABSTRACT: To reliably classify parts of an image sequence as foreground or background is an important part of many computer vision systems, such as video surveillance, tracking and robotics. It can also be important in applications where bandwidth is the limiting factor, such as video conferencing. Independent foreground motion is an attractive source of information for this task, and with static cameras, background subtraction is a particularly popular type of approach. The idea behind background subtraction is to compare the current image with a reference image of the background, and from there decide on a pixel by pixel basis, what is foreground and what is background by observing the changes in the pixel sequence. The problem is to get the useful reference image, especially when large parts of the background are occluded by moving/stationary foreground objects; i.e. some parts of the background are never seen. In this thesis four algorithms are reviewed that segment an image sequence into foreground and background components with varying degrees of success that can be measured on speed, comparative accuracy and/or memory requirements. These measures can be then effectively used to decide the application scope of the individual algorithms. / AFRIKAANSE OPSOMMING: Om betroubaar dele van ’n beeld reeks te klassifiseer as voorgrond of agtergrond is ’n belangrike deel van baie rekenaarvisie sisteme, byvoorbeeld video bewaking, volging en robotika. Dit kan ook belangrik wees in toepassings waar bandwydte die beperkende faktor is, byvoorbeeld video konferensie gesprekke. Onafhanklik voorgrond beweging is ’n aantreklike bron van informasie vir hierdie taak, en met statiese kameras, is agtergrond aftrekking ’n populêre benadering. Die idee agter agtergrond aftrekking is om die huidige beeld met ’n naslaan beeld van die agtergrond te vergelyk, en daarvandaan besluit op ’n piksel-na-piksel basis, wat is voorgrond en wat is agtergrond deur die observasies van die veranderinge in die piksel-reeks. Die probleem is om die naslaan beeld te kry om mee te werk, veral wanneer groot dele van die agtergrond onsigbaar bly as gevolg van bewegende of stilstaande voorgrond objekte en sommige dele van die agtergrond word dalk nooit gesien nie. In hierdie tesis word vier algorithms ondersoek wat ’n beeld reeks segmenteer in respektiewe voorgrond en agtergrond komponente met wisselende grade van sukses wat gemeet kan word deur spoed, vergelykbare akkuraatheid en/of geheu gebruik. Hierdie metings kan dan effektief gebruik word om die applikasie veld van die individuele algoritmes the bepaal.

Les travaux de recherche développés dans cette thèse concernent la mise en oeuvre d'un système de vidéo surveillance intelligente en milieu hospitalier. Dans le contexte d'une application en unité de soins intensifs médicale, nous introduisons la notion originale de Boite Noire Médicale et nous proposons un nouveau système de monitoring visuel de Détection Automatique de Situations à risque et d'Alerte (DASA) basé sur un système de vidéosurveillance multi-caméra intelligent. L'objectif étant d'interpréter les flux d'informations visuelles et de détecter en temps réel les situations à risque afin de prévenir l'équipe médicale et ensuite archiver les évènements dans une base de donnée vidéo qui représente la Boite Noire Médicale. Le système d'interprétation est basé sur des algorithmes de reconnaissance de scénarios qui exploitent les Modèles de Markovs Cachés (MMCs). Une extension du modèle MMC standard est proposé afin de gérer la structure hiérarchique interne des scénarios et de contrôler la durée de chaque état du modèle markovien. La contribution majeure de ce travail repose sur l'intégration d'un raisonnement de type évènementiel, pour gérer la décision de reconnaissance en tenant compte des imperfections des informations disponibles. Les techniques de reconnaissance de scénarios proposées ont été testées et évaluées sur une base de séquences vidéo médicales et comparés aux modèles de Markov cachés probabilistiques classiques. / This thesis focuses on the study and the implementation of an intelligent visual monitoring system in hospitals. In the context of an application for patient monitoring in mediacal intensive care unit, we introduce an original concept of the Medical Black Box and we propose a new system for visual monitoring of Automatic Detection of risk Situations and Alert (DASA) based on a CCTV system with network smart camera. The aim is to interpret the visual information flow and to detect at real-time risk situations to prevent the mediacl team and then archive the events in a video that is based Medical Black Box data. The interpretation system is based on scenario recognition algorithms that exploit the Hidden Markov Models (HMM). An extension of the classic model of HMM is proposed to handle the internal reporting structure of the scenarios and to control the duration of each state of the Markov model. The main contribution of this work relies on the integration of an evidential reasoning, in order to manage the recognition decision taking into account the imperfections of available information. The proposed scenarios recognition method have been tested and assessed on database of medical video sequences and compared to standard probabilistic Hidden Markov Models.