The thesis deals with head pose estimation in stereo data. The theoretical part provides the basis for understanding the geometry of the camera, its parameters and the method of calibration. The following describes the principles of stereo analysis and creating of disparity maps. In the research section, the methods used for head pose modelling are presented and an analysis of selected published articles is given. In the course of the master’s thesis, a system of two cameras for stereoscopic acquisition of motion of the head was designed and several measurements were carried out. The obtained data was prepared for creation of disparity maps and further processing. Based on the detection of facial features, in particular the inner and outer corners of the eyes and corners of the mouth, and their correspondences, a simple geometric model in shape of triangle was created to illustrate the inclination of the facial plane in space. By computing the angle of inclination in three axes, the current head pose is obtained. Motion is modelled by tracking detected points during video sequences.

La télédermatologie présente plusieurs avantages par rapport aux consultations traditionnelles en cabinet avec un dermatologue. Elle est particulièrement utile pour faciliter l'accès aux soins dermatologiques pour les patients ayant des problèmes de mobilité ou habitant loin des secteurs géographiques médicalisés. Un schéma de mosaïquage automatique d’images dédié à la création des panoramas étendus des vidéo-séquences de peau est proposé pour surmonter les limitations posées par le champ de vue réduit des images stationnaires acquises par les dispositifs actuellement utilisés. Les vidéo-séquences utilisées à cet effet sont acquises en utilisant un dispositif spécialement conçu pour un rendu colorimétrique contrôlé de la surface de la peau. Après une étude des diverses méthodes de recalage d'images existantes, une approche optimale est proposée, avec un certain compromis entre la précision de recalage et le temps de calcul, pour la superposition des parties communes des images cutanées. En outre, une approche pour affiner la correspondance initiale des points caractéristiques extraits est présentée. L'étude présentée porte principalement sur la construction cohérente d’une mosaïque dans son ensemble. Pour atteindre cet objectif, un schéma de mosaïque capable de générer des panoramas cohérents à partir de vidéo-séquences longues est présenté. Ce schéma estime dynamiquement la topologie de la trajectoire des images dans le plan de mosaïquage. Cela permet de placer les images sur le plan panoramique avec un nombre réduit d'images sur le chemin suivi pour atteindre une image donnée à partir d'une image de référence, ce qui réduit non seulement l'accumulation des erreurs, mais permet également d'éviter les interruptions dans le mosaïquage en excluant les paires d'images dont le recalage ne serait pas réussi. L'approche proposée offre une robustesse vis-à-vis des recalages échoués en trouvant des trajets alternatifs. En outre, un mode d'ajustement global pour améliorer davantage la cohérence de la mosaïque est présenté / Teledermatology offers several advantages in comparison to the traditional in-place consultations with a dermatologist. It is particularly useful for easing the access to the dermatological care for patients with mobility or travel constraints. A dedicated mosaicing scheme for creating extended panoramas of skin video sequences is proposed to surmount the limitations posed by the small field of view of stationary images acquired by currently used devices. The video sequences used for this purpose are acquired using a specially designed device for a colorimetrically correct rendering of the skin surface. After a study of various image registration approaches, an approach optimally suited to skin image registration with some compromise between registration accuracy and computation time is selected. In addition, an approach for refining the initially detected key-point correspondence is presented. Central focus of this study is on the overall coherent construction of the mosaic. To achieve this objective, a mosaicing scheme capable of generating coherent panoramas from long video sequences is presented. This scheme dynamically estimates the topology of the image trajectory in the panoramic plane to mosaic the images by reducing the number of images over the path used for reaching a given image from a reference image in order to place it on the panoramic plane. A small number of images reduces the accumulated errors, thus improving the visual coherency of the overall mosaic. Besides, the proposed approach offers robustness against failed registrations, which would interrupt the mosaicing process in the absence of the alternative paths. Moreover, a global adjustment scheme for further improving the coherency of the mosaic is presented

In the past decade, camera manufacturers have offered smaller form factors, smaller pixel sizes (leading to higher resolution images), and faster processing chips to increase the performance of consumer cameras. However, these conventional approaches have failed to capitalize on the spatio-temporal redundancy inherent in images, nor have they adequately provided a solution for finding $3$D point correspondences for cameras sampling different bands of the visible spectrum. In this thesis, we pose the following question---given the repetitious nature of image patches, and appropriate camera architectures, can statistical models be used to increase temporal, structural, or spectral resolution? While many techniques have been suggested to tackle individual aspects of this question, the proposed solutions either require prohibitively expensive hardware modifications and/or require overly simplistic assumptions about the geometry of the scene. We propose a two-stage solution to facilitate image reconstruction; 1) design a linear camera system that optically encodes scene information and 2) recover full scene information using prior models learned from statistics of natural images. By leveraging the tendency of small regions to repeat throughout an image or video, we are able to learn prior models from patches pulled from exemplar images. The quality of this approach will be demonstrated for two application domains, using low-speed video cameras for high-speed video acquisition and multi-spectral fusion using an array of cameras. We also investigate a conventional approach for finding 3D correspondence that enables a generalized assorted array of cameras to operate in multiple modalities, including multi-spectral, high dynamic range, and polarization imaging of dynamic scenes.

Σκοπός της παρούσας διδακτορικής διατριβής είναι η ανάπτυξη αλγορίθμων επεξεργασίας ιατρικής εικόνας για την ενσωμάτωση τους σε ιατρικές εφαρμογές ακτινοθεραπευτικού ενδιαφέροντος. Οι αλγόριθμοι αυτοί στηρίζονται στην αρχή λειτουργίας των χαρτών αυτο-οργάνωσης Kohonen και αξιοποιούν την πληροφορία που περιέχεται σε περιοχές των εικόνων γύρω από σημεία ενδιαφέροντος, ώστε να εντοπίσουν αυτόματα, με ακρίβεια και αξιοπιστία, αντιστοιχίες μεταξύ των εικόνων. Πιο συγκεκριμένα, ένας επαναληπτικός αλγόριθμος προτείνεται για την αυτόματη εύρεση αντίστοιχων σημείων σε ιατρικές εικόνες δύο διαστάσεων. Ο προτεινόμενος αλγόριθμος προϋποθέτει την εύρεση σημείων ενδιαφέροντος μόνο στη μια από τις δύο εικόνες και εντοπίζει τα αντίστοιχα σημεία στη δεύτερη εικόνα μέσα από μια επαναληπτική διαδικασία, η οποία προσομοιάζει τη φάση εκπαίδευσης του νευρωνικού δικτύου. Με βάση τα ζεύγη των αντίστοιχων σημείων, υπολογίζονται στη συνέχεια οι παράμετροι ενός μετασχηματισμού, κατάλληλου για να περιγράψει τη σχέση μεταξύ των δεδομένων εικόνων. Ο αλγόριθμος ευθυγράμμισης εφαρμόζεται σε δεδομένες εικόνες ηλεκτρονικής πυλαίας απεικόνισης (Electronic Portal Images), που λαμβάνονται πριν από κάθε συνεδρία της ακτινοθεραπείας, για τον υπολογισμό του σφάλματος τοποθέτησης του ασθενούς. Το ζήτημα της επαλήθευσης της θέσης του ασθενούς στην ακτινοθεραπεία αντιμετωπίζεται επίσης με τη βοήθεια μιας αυτόματης μεθόδου εύρεσης αντίστοιχων σημείων σε τρισδιάστατα δεδομένα, η οποία εφαρμόζεται για την ευθυγράμμιση της αξονικής τομογραφίας του σχεδιασμού της ακτινοθεραπείας και μιας αξονικής τομογραφίας επαλήθευσης, που λαμβάνεται πριν την πρώτη συνεδρία της ακτινοθεραπείας. Ο προτεινόμενος αλγόριθμος εντοπίζει αντίστοιχα σημεία ενδιαφέροντος στις δεδομένες τομογραφικές εικόνες και υπολογίζει τις παραμέτρους ενός μη γραμμικού μετασχηματισμού ευθυγράμμισης. Μετά την ευθυγράμμιση των δύο τομογραφιών, υπολογίζεται η μετατόπιση του ισοκέντρου στην τομογραφία επαλήθευσης σε σχέση με τη θέση του ισοκέντρου που προβλέπεται στην αρχική τομογραφία του σχεδιασμού. Με την ενσωμάτωση αυτής της μεθόδου ευθυγράμμισης στη διαδικασία της ακτινοθεραπείας, ικανοποιούνται δύο ανάγκες της κλινικής πρακτικής. Αφενός, η μετατόπιση του ισοκέντρου, όπως υπολογίζεται από την προτεινόμενη μέθοδο, παρέχει μια αξιόπιστη ένδειξη για τη μετατόπιση του ασθενούς που απαιτείται πριν τη χορήγηση της ακτινοβολίας. Αφετέρου, επιχειρείται η καλύτερη αξιοποίηση των πόρων του τμήματος της ακτινοθεραπείας με τη διαδικασία της εύρεσης του ισοκέντρου της ακτινοθεραπείας να λαμβάνει χώρα στην αίθουσα του αξονικού τομογράφου και να μειώνεται συνεπώς ο χρόνος που απαιτείται για την προετοιμασία του ασθενούς στον γραμμικό επιταχυντή κατά την πρώτη συνεδρία της ακτινοθεραπείας. / Aim of the present thesis is the development of image processing algorithms for radiotherapy applications. These algorithms are based on the principles of Kohonen Self Organizing Maps and exploit the information contained in image regions around distinctive points of interest, in order to determine image correspondences in an automatic, accurate and robust way. In particular, an iterative algorithm is proposed for automatic detection of point correspondences in two-dimensional medical images. The proposed algorithm requires the extraction of interest points only in one image and detects the homologous points in the second image through an iterative procedure, respective to the training phase of a neural network. Subsequently, the parameters of an appropriate registration transformation are computed to describe the mapping between the two images. The computation is based on the detected point correspondence. The proposed registration algorithm is applied to Electronic Portal Images, acquired prior to the radiotherapy treatment delivery, in order to estimate the setup error of the patient. The issue of patient position verification in radiotherapy is also addressed in the present thesis by developing an algorithm for automatic detection of point correspondences in three-dimensional medical data. The algorithm is used to register the CT data of radiotherapy planning to an additional verification CT, acquired prior to the first treatment fraction. The proposed algorithm detects corresponding points in the two CT images and computes the parameters of a non-rigid registration transformation. After the registration of the two CT images, the isocenter displacement of the verification CT is calculated with respect to the ideal isocenter position, defined in the planning CT. By integrating the proposed registration procedure in the clinical practice, two needs are met. Firstly, the isocenter displacement, calculated by the proposed method, provides a reliable indication of the patient shift, needed before the treatment delivery, for optimization of the dose delivery. Secondly, an improvement of the radiotherapy department efficiency is attempted by performing the procedure of isocenter marking in the CT scanner room and, consequently, reducing the time expenditure of the patient in the LINAC during the first radiotherapy fraction.

In this thesis, the expression variation problem in two-dimensional (2D) and three-dimensional (3D) face recognition is tackled. While discriminant analysis (DA) methods are effective solutions for recognizing expression-variant 2D face images, they are not directly applicable when only a single sample image per subject is available. This problem is addressed in this thesis by introducing expression subspaces which can be used for synthesizing new expression images from subjects with only one sample image. It is proposed that by augmenting a generic training set with the gallery and their synthesized new expression images, and then training DA methods using this new set, the face recognition performance can be significantly improved. An important advantage of the proposed method is its simplicity; the expression of an image is transformed simply by projecting it into another subspace. The above proposed solution can also be used in general pattern recognition applications. The above method can also be used in 3D face recognition where expression variation is a more serious issue. However, DA methods cannot be readily applied to 3D faces because of the lack of a proper alignment method for 3D faces. To solve this issue, a method is proposed for sampling the points of the face that correspond to the same facial features across all faces, denoted as the closest-normal points (CNPs). It is shown that the performance of the linear discriminant analysis (LDA) method, applied to such an aligned representation of 3D faces, is significantly better than the performance of the state-of-the-art methods which, rely on one-by-one registration of the probe faces to every gallery face. Furthermore, as an important finding, it is shown that the surface normal vectors of the face provide a higher level of discriminatory information rather than the coordinates of the points. In addition, the expression subspace approach is used for the recognition of 3D faces from single sample. By constructing expression subspaces from the surface normal vectors at the CNPs, the surface normal vectors of a 3D face with single sample can be synthesized under other expressions. As a result, by improving the estimation of the within-class scatter matrix using the synthesized samples, a significant improvement in the recognition performance is achieved.

In this thesis, the expression variation problem in two-dimensional (2D) and three-dimensional (3D) face recognition is tackled. While discriminant analysis (DA) methods are effective solutions for recognizing expression-variant 2D face images, they are not directly applicable when only a single sample image per subject is available. This problem is addressed in this thesis by introducing expression subspaces which can be used for synthesizing new expression images from subjects with only one sample image. It is proposed that by augmenting a generic training set with the gallery and their synthesized new expression images, and then training DA methods using this new set, the face recognition performance can be significantly improved. An important advantage of the proposed method is its simplicity; the expression of an image is transformed simply by projecting it into another subspace. The above proposed solution can also be used in general pattern recognition applications. The above method can also be used in 3D face recognition where expression variation is a more serious issue. However, DA methods cannot be readily applied to 3D faces because of the lack of a proper alignment method for 3D faces. To solve this issue, a method is proposed for sampling the points of the face that correspond to the same facial features across all faces, denoted as the closest-normal points (CNPs). It is shown that the performance of the linear discriminant analysis (LDA) method, applied to such an aligned representation of 3D faces, is significantly better than the performance of the state-of-the-art methods which, rely on one-by-one registration of the probe faces to every gallery face. Furthermore, as an important finding, it is shown that the surface normal vectors of the face provide a higher level of discriminatory information rather than the coordinates of the points. In addition, the expression subspace approach is used for the recognition of 3D faces from single sample. By constructing expression subspaces from the surface normal vectors at the CNPs, the surface normal vectors of a 3D face with single sample can be synthesized under other expressions. As a result, by improving the estimation of the within-class scatter matrix using the synthesized samples, a significant improvement in the recognition performance is achieved.

We formulate Local Shape Descriptor selection for model-based object recognition in range data as an optimization problem and offer a platform that facilitates a solution. The goal of object recognition is to identify and localize objects of interest in an image. Recognition is often performed in three phases: point matching, where correspondences are established between points on the 3-D surfaces of the models and the range image; hypothesis generation, where rough alignments are found between the image and the visible models; and pose refinement, where the accuracy of the initial alignments is improved. The overall efficiency and reliability of a recognition system is highly influenced by the effectiveness of the point matching phase. Local Shape Descriptors are used for establishing point correspondences by way of encapsulating local shape, such that similarity between two descriptors indicates geometric similarity between their respective neighbourhoods. We present a generalized platform for constructing local shape descriptors that subsumes a large class of existing methods and allows for tuning descriptors to the geometry of specific models and to sensor characteristics. Our descriptors, termed as Variable-Dimensional Local Shape Descriptors, are constructed as multivariate observations of several local properties and are represented as histograms. The optimal set of properties, which maximizes the performance of a recognition system, depend on the geometry of the objects of interest and the noise characteristics of range image acquisition devices and is selected through pre-processing the models and sample training images. Experimental analysis confirms the superiority of optimized descriptors over generic ones in recognition tasks in LIDAR and dense stereo range images. / Thesis (Ph.D, Electrical & Computer Engineering) -- Queen's University, 2009-09-01 11:07:32.084