Multi-camera systems are becoming ubiquitous and have found application in a variety of domains including surveillance, immersive visualization, sports entertainment and movie special effects amongst others. From a computer vision perspective, the challenging task is how to most efficiently fuse information from multiple views in the absence of detailed calibration information and a minimum of human intervention. This thesis presents a new approach to fuse foreground likelihood information from multiple views onto a reference view without explicit processing in 3D space, thereby circumventing the need for complete calibration. Our approach uses a homographic occupancy constraint (HOC), which states that if a foreground pixel has a piercing point that is occupied by foreground object, then the pixel warps to foreground regions in every view under homographies induced by the reference plane, in effect using cameras as occupancy detectors. Using the HOC we are able to resolve occlusions and robustly determine ground plane localizations of the people in the scene. To find tracks we obtain ground localizations over a window of frames and stack them creating a space time volume. Regions belonging to the same person form contiguous spatio-temporal tracks that are clustered using a graph cuts segmentation approach. Second, we demonstrate that the HOC is equivalent to performing visual hull intersection in the image-plane, resulting in a cross-sectional slice of the object. The process is extended to multiple planes parallel to the reference plane in the framework of plane to plane homologies. Slices from multiple planes are accumulated and the 3D structure of the object is segmented out. Unlike other visual hull based approaches that use 3D constructs like visual cones, voxels or polygonal meshes requiring calibrated views, ours is purely-image based and uses only 2D constructs i.e. planar homographies between views. This feature also renders it conducive to graphics hardware acceleration. The current GPU implementation of our approach is capable of fusing 60 views (480x720 pixels) at the rate of 50 slices/second. We then present an extension of this approach to reconstructing non-rigid articulated objects from monocular video sequences. The basic premise is that due to motion of the object, scene occupancies are blurred out with non-occupancies in a manner analogous to motion blurred imagery. Using our HOC and a novel construct: the temporal occupancy point (TOP), we are able to fuse multiple views of non-rigid objects obtained from a monocular video sequence. The result is a set of blurred scene occupancy images in the corresponding views, where the values at each pixel correspond to the fraction of total time duration that the pixel observed an occupied scene location. We then use a motion de-blurring approach to de-blur the occupancy images and obtain the 3D structure of the non-rigid object. In the final part of this thesis, we present an object class detection method employing 3D models of rigid objects constructed using the above 3D reconstruction approach. Instead of using a complicated mechanism for relating multiple 2D training views, our approach establishes spatial connections between these views by mapping them directly to the surface of a 3D model. To generalize the model for object class detection, features from supplemental views (obtained from Google Image search) are also considered. Given a 2D test image, correspondences between the 3D feature model and the testing view are identified by matching the detected features. Based on the 3D locations of the corresponding features, several hypotheses of viewing planes can be made. The one with the highest confidence is then used to detect the object using feature location matching. Performance of the proposed method has been evaluated by using the PASCAL VOC challenge dataset and promising results are demonstrated.

Emotional intelligence has a prominent role in education, health care, and day to day interaction. With the increasing use of computer technology, computers are interacting with more and more individuals. This interaction provides an opportunity to increase knowledge about human emotion for human consumption, well-being, and improved computer adaptation. This thesis explores the efficacy of using up to four different sensors in three domains for computational affect detection. We first consider computer-based education, where a collection of four sensors is used to detect student emotions relevant to learning, such as frustration, confidence, excitement and interest while students use a computer geometry tutor. The best classier of each emotion in terms of accuracy ranges from 78% to 87.5%. We then use voice data collected in a clinical setting to differentiate both gender and culture of the speaker. We produce classifiers with accuracies between 84% and 94% for gender, and between 58% and 70% for American vs. Asian culture, and we find that classifiers for distinguishing between four cultures do not perform better than chance. Finally, we use video and audio in a health care education scenario to detect students' emotions during a clinical simulation evaluation. The video data provides classifiers with accuracies between 63% and 88% for the emotions of confident, anxious, frustrated, excited, and interested. We find the audio data to be too complex to single out the voice source of the student by automatic means. In total, this work is a step forward in the automatic computational detection of affect in realistic settings.

No description available.

No description available.

This master thesis deals with the analysis of the current possibilities for object tracking in the image, based on which is designed a procedure for creating a system capable of tracking an object of interest. Part of this work is designing virtual reality for the needs of implementation of the tracking system, which is finally deployed and tested on a real prototype of unmanned vehicle.

Using some form of dynamical model in a visual tracking system is a well-known method for increasing robustness and indeed performance in general. Often, quite simple models are used and can be effective, but prior knowledge of the likely motion of the tracking target can often be exploited by using a specially-tailored model. Specifying such a model by hand, while possible, is a time-consuming and error-prone process. Much more desirable is for an automated system to learn a model from training data. A dynamical model learnt in this manner can also be a source of useful information in its own right, and a set of dynamical models can provide discriminatory power for use in classification problems. Methods exist to perform such learning, but are limited in that they assume the availability of 'ground truth' data. In a visual tracking system, this is rarely the case. A learning system must work from visual data alone, and this thesis develops methods for learning dynamical models while explicitly taking account of the nature of the training data --- they are noisy measurements. The algorithms are developed within two tracking frameworks. The Kalman filter is a simple and fast approach, applicable where the visual clutter is limited. The recently-developed Condensation algorithm is capable of tracking in more demanding situations, and can also employ a wider range of dynamical models than the Kalman filter, for instance multi-mode models. The success of the learning algorithms is demonstrated experimentally. When using a Kalman filter, the dynamical models learnt using the algorithms presented here produce better tracking when compared with those learnt using current methods. Learning directly from training data gathered using Condensation is an entirely new technique, and experiments show that many aspects of a multi-mode system can be successfully identified using very little prior information. Significant computational effort is required by the implementation of the methods, and there is scope for improvement in this regard. Other possibilities for future work include investigation of the strong links this work has with learning problems in other areas. Most notable is the study of the 'graphical models' commonly used in expert systems, where the ideas presented here promise to give insight and perhaps lead to new techniques.

Dans le contexte de la navigation autonome en environnement urbain, une localisation précise du véhicule est importante pour une navigation sure et fiable. La faible précision des capteurs bas coût existants tels que le système GPS, nécessite l'utilisation d'autres capteurs eux aussi à faible coût. Les caméras mesurent une information photométrique riche et précise sur l'environnement, mais nécessitent l'utilisation d'algorithmes de traitement avancés pour obtenir une information sur la géométrie et sur la position de la caméra dans l'environnement. Cette problématique est connue sous le terme de Cartographie et Localisation Simultanées (SLAM visuel). En général, les techniques de SLAM sont incrémentales et dérivent sur de longues trajectoires. Pour simplifier l'étape de localisation, il est proposé de découpler la partie cartographie et la partie localisation en deux phases: la carte est construite hors-ligne lors d'une phase d'apprentissage, et la localisation est effectuée efficacement en ligne à partir de la carte 3D de l'environnement. Contrairement aux approches classiques, qui utilisent un modèle 3D global approximatif, une nouvelle représentation égo-centrée dense est proposée. Cette représentation est composée d'un graphe d'images sphériques augmentées par l'information dense de profondeur (RGB+D), et permet de cartographier de larges environnements. Lors de la localisation en ligne, ce type de modèle apporte toute l'information nécessaire pour une localisation précise dans le voisinage du graphe, et permet de recaler en temps-réel l'image perçue par une caméra embarquée sur un véhicule, avec les images du graphe, en utilisant une technique d'alignement d'images directe. La méthode de localisation proposée, est précise, robuste aux aberrations et prend en compte les changements d'illumination entre le modèle de la base de données et les images perçues par la caméra. Finalement, la précision et la robustesse de la localisation permettent à un véhicule autonome, équipé d'une caméra, de naviguer de façon sure en environnement urbain. / In an autonomous navigation context, a precise localisation of the vehicule is important to ensure a reliable navigation. Low cost sensors such as GPS systems are inacurrate and inefficicent in urban areas, and therefore the employ of such sensors alone is not well suited for autonomous navigation. On the other hand, camera sensors provide a dense photometric measure that can be processed to obtain both localisation and mapping information. In the robotics community, this problem is well known as Simultaneous Localisation and Mapping (SLAM) and it has been studied for the last thirty years. In general, SLAM algorithms are incremental and prone to drift, thus such methods may not be efficient in large scale environments for real-time localisation. Clearly, an a-priori 3D model simplifies the localisation and navigation tasks since it allows to decouple the structure and motion estimation problems. Indeed, the map can be previously computed during a learning phase, whilst the localisation can be handled in real-time using a single camera and the pre-computed model. Classic global 3D model representations are usually inacurrate and photometrically inconsistent. Alternatively, it is proposed to use an ego-centric model that represents, as close as possible, real sensor measurements. This representation is composed of a graph of locally accurate spherical panoramas augmented with dense depth information. These augmented panoramas allow to generate varying viewpoints through novel view synthesis. To localise a camera navigating locally inside the graph, we use the panoramas together with a direct registration technique. The proposed localisation method is accurate, robust to outliers and can handle large illumination changes. Finally, autonomous navigation in urban environments is performed using the learnt model, with only a single camera to compute localisation.

Este trabalho propõe uma abordagem para o rastreamento de objetos observados em seqüências de imagens. O objetivo principal é o desenvolvimento de uma metodologia eficiente, capaz de realizar o rastreamento de um ou mais alvos heterogêneos, usando pouca informação a priori sobre os mesmos. Para alcançar este objetivo é proposta a descrição dos alvos livre de um modelo explícito de forma, através de uma representação baseada em contornos, a qual é interessante pois tem a capacidade de adaptar-se dinamicamente a alvos com formas heterogêneas de modo eficaz. Além disso, é usado um modelo de movimento único e simples, considerando somente translação e mudança de escala quadro a quadro. Este modelo possibilita o tratamento de movimentos suaves e previamente desconhecidos dos alvos. O rastreamento de cada alvo é executado com a combinação de dois Filtros de Kalman: um para estimação do movimento e outro para estimação do contorno. O modelo de observação é baseado em linhas de medida 1D fixadas ao longo do contorno estimado e tem baixo custo computacional. Experimentos foram conduzidos para avaliar a eficácia e eficiência da proposta, utilizando seqüências de imagens controladas e reais. Os resultados mostram que a abordagem proposta é capaz de rastrear alvos distintos (figuras geométricas, pessoas e robôs móveis), executando diferentes movimentos considerando a posição de observação da câmera. Embora haja uma relação crítica entre a variação quadro a quadro do movimento e da forma dos alvos, e o nível de ruído nas imagens, a abordagem é adequada nos casos em que informações detalhadas a respeito do movimento e da forma dos alvos não são disponíveis. / This work proposes an approach to track objects observed in sequences of images. The main objective is the development of an efficient methodology, capable of performing the tracking of one or more heterogeneous targets by using a small amount of a priori information about them. To accomplish this objective we propose a description of the targets free of an explicit shape model. This description is a contour-based representation, which is interesting because it is capable of adapting dynamically to targets that have heterogeneous shapes in an effective way. Besides this, a unique and simple movement model is used, considering only translation and scaling frame by frame. This model allows treating smooth and previously unknown targets movements. The tracking of each target is executed by the combination of two Kalman Filters: one used to estimate movement and another one to estimate contour. The observation model is based on 1D measurement lines fixed along the estimated contour and requires low computational power. Experiments were performed to evaluate the efficacy and the efficiency of the proposal, using controlled and real image sequences. Results show that the proposed approach is capable of tracking distinct targets (geometric figures, human bodies and mobile robots), which execute different movements regarding the observation position of the camera. Despite the critical tradeoff between the frame by frame variation of the targets movements and shapes and the level of noise in the images, the approach showed to be adequate for those cases of application where detailed information about target movement and shape are not available.

Este trabalho propõe uma abordagem para o rastreamento de objetos observados em seqüências de imagens. O objetivo principal é o desenvolvimento de uma metodologia eficiente, capaz de realizar o rastreamento de um ou mais alvos heterogêneos, usando pouca informação a priori sobre os mesmos. Para alcançar este objetivo é proposta a descrição dos alvos livre de um modelo explícito de forma, através de uma representação baseada em contornos, a qual é interessante pois tem a capacidade de adaptar-se dinamicamente a alvos com formas heterogêneas de modo eficaz. Além disso, é usado um modelo de movimento único e simples, considerando somente translação e mudança de escala quadro a quadro. Este modelo possibilita o tratamento de movimentos suaves e previamente desconhecidos dos alvos. O rastreamento de cada alvo é executado com a combinação de dois Filtros de Kalman: um para estimação do movimento e outro para estimação do contorno. O modelo de observação é baseado em linhas de medida 1D fixadas ao longo do contorno estimado e tem baixo custo computacional. Experimentos foram conduzidos para avaliar a eficácia e eficiência da proposta, utilizando seqüências de imagens controladas e reais. Os resultados mostram que a abordagem proposta é capaz de rastrear alvos distintos (figuras geométricas, pessoas e robôs móveis), executando diferentes movimentos considerando a posição de observação da câmera. Embora haja uma relação crítica entre a variação quadro a quadro do movimento e da forma dos alvos, e o nível de ruído nas imagens, a abordagem é adequada nos casos em que informações detalhadas a respeito do movimento e da forma dos alvos não são disponíveis. / This work proposes an approach to track objects observed in sequences of images. The main objective is the development of an efficient methodology, capable of performing the tracking of one or more heterogeneous targets by using a small amount of a priori information about them. To accomplish this objective we propose a description of the targets free of an explicit shape model. This description is a contour-based representation, which is interesting because it is capable of adapting dynamically to targets that have heterogeneous shapes in an effective way. Besides this, a unique and simple movement model is used, considering only translation and scaling frame by frame. This model allows treating smooth and previously unknown targets movements. The tracking of each target is executed by the combination of two Kalman Filters: one used to estimate movement and another one to estimate contour. The observation model is based on 1D measurement lines fixed along the estimated contour and requires low computational power. Experiments were performed to evaluate the efficacy and the efficiency of the proposal, using controlled and real image sequences. Results show that the proposed approach is capable of tracking distinct targets (geometric figures, human bodies and mobile robots), which execute different movements regarding the observation position of the camera. Despite the critical tradeoff between the frame by frame variation of the targets movements and shapes and the level of noise in the images, the approach showed to be adequate for those cases of application where detailed information about target movement and shape are not available.

This thesis makes significant contributions to the object detection problem in computer vision. The object detection problem is, given a digital image of a scene, to detect the relevant object in the image. One technique for performing object detection, called ``active contours,' optimizes a constructed energy that is defined on contours (closed curves) and is tailored to image features. An optimization method can be used to perform the optimization of the energy, and thereby deform an initially placed contour to the relevant object. The typical optimization technique used in almost every active contour paper is evolving the contour by the energy's gradient descent flow, i.e., the steepest descent flow, in order to drive the initial contour to (hopefully) the minimum curve. The problem with this technique is that often times the contour becomes stuck in a sub-optimal and undesirable local minimum of the energy. This problem can be partially attributed to the fact that the gradient flows of these energies make use of only local image and contour information. By local, we mean that in order to evolve a point on the contour, only information local to that point is used. Therefore, in this thesis, we introduce a new class of flows that are global in that the evolution of a point on the contour depends on global information from the entire curve. These flows help avoid a number of problems with traditional flows including helping in avoiding undesirable local minima. We demonstrate practical applications of these flows for the object detection problem, including applications to both image segmentation and visual object tracking.