Podemos afirmar que deslocamentos da imagem em quadros consecutivos de uma seqüência de vídeo são causados por pequenas vibrações da câmera e/ou movimentos desejados pelo operador da câmera. A estabilização de imagem consiste no processo de remoção de pequenas vibrações que caracterizam movimentos indesejados de uma seqüência de imagens. Com este propósito, atualmente técnicas de processamento digital de vídeo vêm sendo comumente aplicadas na indústria eletrônica. No processo digital de estabilização de imagens são necessários métodos computacionais de estimação, de suavização e de correção de movimento, para os quais, existe uma grande variedade de técnicas de processamento. O emprego de uma técnica específica de processamento é determinado conforme o tipo de aplicação. Técnicas para a estimação de movimento como casamento de blocos (CB), e para a suavização de movimento como filtro de freqüência passa baixa, são freqüentemente encontradas na literatura. Este trabalho apresenta um sistema de estabilização digital de imagens em tempo real capturadas por uma câmera digital, estimando e compensando movimentos translacionais e rotacionais indesejados. / Undesirable shakes or jiggles, object motion within image or desirable motions caused by the camera operator causes image differences in consecutive frames of video sequences. The image stabilization consists of the process of removing inevitable and undesirable fluctuations, shakes and jiggles; with this purpose, nowadays digital processing techniques have been commonly applied in the electronic industry. On the digital processing of image stabilization, computational methods of estimation, smoothing and motion correction are necessary. In the literature various digital processing techniques for image stabilization are described, the most suitable technique should be chosen according to the kind of application. Techniques such as block matching used in motion estimation and low-pass filters used in motion smoothing are found in a great number of papers. This work presents a real time digital image stabilization system capable of stabilizing video sequences with undesirable translational and rotational displacements between frames.

The estimation of motion of multi-camera systems is one of the most important tasks in computer vision research. Recently, some issues have been raised about general camera models and multi-camera systems. Using many cameras as a single camera is studied [60], and the epipolar geometry constraints of general camera models is theoretically derived. Methods for calibration, including a self-calibration method for general camera models, are studied [78, 62]. Multi-camera systems are an example of practically implementable general camera models and they are widely used in many applications nowadays because of both the low cost of digital charge-coupled device (CCD) cameras and the high resolution of multiple images from the wide field of views. To our knowledge, no research has been conducted on the relative motion of multi-camera systems with non-overlapping views to obtain a geometrically optimal solution. ¶ In this thesis, we solve the camera motion problem for multi-camera systems by using linear methods and convex optimization techniques, and we make five substantial and original contributions to the field of computer vision. First, we focus on the problem of translational motion of omnidirectional cameras, which are multi-camera systems, and present a constrained minimization method to obtain robust estimation results. Given known rotation, we show that bilinear and trilinear relations can be used to build a system of linear equations, and singular value decomposition (SVD) is used to solve the equations. Second, we present a linear method that estimates the relative motion of generalized cameras, in particular, in the case of non-overlapping views. We also present four types of generalized cameras, which can be solvable using our proposed, modified SVD method. This is the first study finding linear relations for certain types of generalized cameras and performing experiments using our proposed linear method. Third, we present a linear 6-point method (5 points from the same camera and 1 point from another camera) that estimates the relative motion of multi-camera systems, where cameras have no overlapping views. In addition, we discuss the theoretical and geometric analyses of multi-camera systems as well as certain critical configurations where the scale of translation cannot be determined. Fourth, we develop a global solution under an L∞ norm error for the relative motion problem of multi-camera systems using second-order cone programming. Finally, we present a fast searching method to obtain a global solution under an L∞ norm error for the relative motion problem of multi-camera systems, with non-overlapping views, using a branch-and-bound algorithm and linear programming (LP). By testing the feasibility of LP at the earlier stage, we reduced the time of computation of solving LP.¶ We tested our proposed methods by performing experiments with synthetic and real data. The Ladybug2 camera, for example, was used in the experiment on estimation of the translation of omnidirectional cameras and in the estimation of the relative motion of non-overlapping multi-camera systems. These experiments showed that a global solution using L∞ to estimate the relative motion of multi-camera systems could be achieved.

Le développement des véhicules autonomes a reçu une attention croissant ces dernières années, notamment les secteurs de la défense et de l'industrie automobile. L'intérêt pour l'industrie automobile est motivé par la conception de véhicules sûrs et confortables. Une raison commune derrière la plupart des accidents de la circulation est due au manque de vigilance du conducteur sur la route. Cette thèse se trouve dans le problématique de l'estimation des risques de collision pour un véhicule dans les secondes qui suivent en condition de circulation urbaines. Les systèmes actuellement disponibles dans le commerce sont pour la plupart conçus pour prévenir les collisions avant, arrières, ou latérales. Ces systèmes sont généralement équipés d'un capteur de type radar, à l'arrière, à l'avant ou sur les côtés pour mesurer la vitesse et la distance aux obstacles. Les algorithmes pour déterminer le risque de collision sont fondés sur des variantes du TTC (time-to-collision en anglais). Cependant, un véhicule peut se trouver dans des situations où les routes ne sont pas droites et l'hypothèse que le mouvement est linéaire ne tient pas pour le calcul du TTC. Dans ces situations, le risque est souvent sous-estimé. De plus, les instances où les routes ne sont pas tout droit se trouve assez souvent dans les environnement urbain ; par exemple, les rond point ou les intersections. Un argument de cette thèse est que, savoir simplement qu'il y ait un objet à une certaine position et à une instance spécifique dans le temps ne suffit pas à évaluer sa sécurité dans le futur. Un système capable de comprendre les comportements de déplacement du véhicule est indispensable. En plus, les contraintes environnementales doivent être prises en considération. Le cas le plus simple du mouvement « libre » est d'abord traité. Dans cette situation il n'ya pas de contraintes environnementales ou de comportement explicite. Ensuite, les contraintes environnementales des routes sur trafic urbain et le comportement des conducteurs des véhicules sont introduits et pris en compte explicitement. Cette thèse propose un modèle probabiliste pour les trajectoires des véhicules fondé sur le processus gaussien (GP). Son avantage est le pouvoir d'exprimer le mouvement dans le futur indépendamment de la discrétisation d'espace et d'état. Les comportements des conducteurs sont modélisés avec une variante du modèle de Markov caché. La combinaison de ces deux modèles donne un modèle probabiliste de l'évolution complète du véhicule dans le temps. En plus, une méthode générale pour l'évaluation probabiliste des risques de collision est présentée, où différentes valeurs de risque, chacun avec sa propre sémantique.

<p>In this thesis we present a system to extract the eye motion from a video stream containing a human face and applying this eye motion into a virtual character. By the notation eye motion estimation, we mean the information which describes the location of the eyes in each frame of the video stream. Applying this eye motion estimation into a virtual character, we achieve that the virtual face moves the eyes in the same way than the human face, synthesizing eye motion into a virtual character. In this study, a system capable of face tracking, eye detection and extraction, and finally iris position extraction using video stream containing a human face has been developed. Once an image containing a human face is extracted from the current frame of the video stream, the detection and extraction of the eyes is applied. The detection and extraction of the eyes is based on edge detection. Then the iris center is determined applying different image preprocessing and region segmentation using edge features on the eye picture extracted.</p><p>Once, we have extracted the eye motion, using MPEG-4 Facial Animation, this motion is translated into the Facial Animation arameters (FAPs). Thus we can improve the quality and quantity of Facial Animation expressions that we can synthesize into a virtual character.</p>

<p>This thesis covers topics relevant to model-based coding. Model-based coding is a promising very low bit rate video coding technique. The idea behind this technique is to parameterize a talking head and to extract and transmit the parameters describing facial movements. At the receiver, the parameters are used to reconstruct the talking head. Since only high-level animation parameters are transmitted, very high compression can be achieved with this coding scheme. This thesis covers the following three key problems.</p><p>Although it is a fundamental problem, the initialization problem, has been neglected some extent in the literature. In this thesis, we pay particular attention to the study of this problem. We propose a pseudo-automatic initialization scheme: an Analysis-by-Synthesis scheme based on Simulated Annealing. It has been proved to be an efficient scheme.</p><p>Owing to technical advance today and the newly emerged MPEG-4 standard, new schemes of performing texture mapping and motion estimation are suggested which use sample based direct texture mapping; the feasibility of using active motion estimation is explored which proves to be able to give more than 10 times tracking resolution. Based on the matured face detection technique, Dynamic Programming is introduced to face detection module and work for face tracking.</p><p>Another important problem addressed in this thesis is how to evaluate the face tracking techniques. We studied the evaluation problems by examining the commonly used method, which employs a physical magnetic sensor to provide "ground truth". In this thesis we point out that it is quite misleading to use such a method.</p>

<p>In this thesis we study algorithms and architectures that can provide a better Quality of Experience (QoE) for streaming video systems and services. With cases and examples taken from the application scenarios of football on mobile phones, we address the fundamental problems behind streaming video services. Thus, our research results can be applied and extended to other networks, to other sports and to other cultural activities.</p><p>In algorithm development, we propose five different schemes. We suggest a blind motion estimation and a trellis based motion estimation with dynamic programming algorithms for Wyner-Ziv coding. We develop a trans-media technology, vibrotactile coding of visual signals for mobile phones. We propose a new bandwidth prediction scheme for real-time video conference. We also provide an effective method based on dynamic programming to select optimal services and maximize QoE.</p><p>In architecture design, we offer three architectures for real-time interactive video and two for streaming live football information. The former three are: a structure of motion estimation in Wyner-Ziv coding for real-time video; a variable bit rate Wyner-Ziv video coding structure based on multi-view camera array; and a dynamic resource allocation structure based on 3-D object motion. The latter two are: a vibrotactile signal rendering system for live information; and a Universal Multimedia Access architecture for streaming live football video.</p><p>In QoE exploration, we give a detailed and deep discussion of QoE and the enabling techniques. We also develop a conceptual model for QoE. Moreover we place streaming video services in a framework of QoE. The new general framework of streaming video services allows for the interaction between the user, content and technology.</p><p>We demonstrate that it is possible to develop algorithms and architectures that take into account the user's perspective. Quality of Experience in video mobile services is within our reach.</p>

This thesis covers topics relevant to model-based coding. Model-based coding is a promising very low bit rate video coding technique. The idea behind this technique is to parameterize a talking head and to extract and transmit the parameters describing facial movements. At the receiver, the parameters are used to reconstruct the talking head. Since only high-level animation parameters are transmitted, very high compression can be achieved with this coding scheme. This thesis covers the following three key problems. Although it is a fundamental problem, the initialization problem, has been neglected some extent in the literature. In this thesis, we pay particular attention to the study of this problem. We propose a pseudo-automatic initialization scheme: an Analysis-by-Synthesis scheme based on Simulated Annealing. It has been proved to be an efficient scheme. Owing to technical advance today and the newly emerged MPEG-4 standard, new schemes of performing texture mapping and motion estimation are suggested which use sample based direct texture mapping; the feasibility of using active motion estimation is explored which proves to be able to give more than 10 times tracking resolution. Based on the matured face detection technique, Dynamic Programming is introduced to face detection module and work for face tracking. Another important problem addressed in this thesis is how to evaluate the face tracking techniques. We studied the evaluation problems by examining the commonly used method, which employs a physical magnetic sensor to provide "ground truth". In this thesis we point out that it is quite misleading to use such a method.

In this thesis we study algorithms and architectures that can provide a better Quality of Experience (QoE) for streaming video systems and services. With cases and examples taken from the application scenarios of football on mobile phones, we address the fundamental problems behind streaming video services. Thus, our research results can be applied and extended to other networks, to other sports and to other cultural activities. In algorithm development, we propose five different schemes. We suggest a blind motion estimation and a trellis based motion estimation with dynamic programming algorithms for Wyner-Ziv coding. We develop a trans-media technology, vibrotactile coding of visual signals for mobile phones. We propose a new bandwidth prediction scheme for real-time video conference. We also provide an effective method based on dynamic programming to select optimal services and maximize QoE. In architecture design, we offer three architectures for real-time interactive video and two for streaming live football information. The former three are: a structure of motion estimation in Wyner-Ziv coding for real-time video; a variable bit rate Wyner-Ziv video coding structure based on multi-view camera array; and a dynamic resource allocation structure based on 3-D object motion. The latter two are: a vibrotactile signal rendering system for live information; and a Universal Multimedia Access architecture for streaming live football video. In QoE exploration, we give a detailed and deep discussion of QoE and the enabling techniques. We also develop a conceptual model for QoE. Moreover we place streaming video services in a framework of QoE. The new general framework of streaming video services allows for the interaction between the user, content and technology. We demonstrate that it is possible to develop algorithms and architectures that take into account the user's perspective. Quality of Experience in video mobile services is within our reach.

Motion estimation and compensation plays an important role in video processing applications. Two-dimensional block-based and mesh-based models are widely used in this area. A 2-D mesh-based model provides a better representation of complex real world motion than a block-based model. Mesh-based motion estimation algorithms are employed in both frame-based and object-based video compression and coding. A hierarchical mesh-based algorithm is applied to improve the motion field generated by a single-layer algorithm. 2-D mesh-based models also enable the manipulation of video objects which is included in the MPEG-4 standard. A video object in a video clip can be replaced by another object by the use of a dynamic mesh structure. In this thesis, a comparative analysis of 2-D block-based and mesh-based motion estimation algorithms in both frame-based and object-based video representations is performed. The experimental results indicate that a mesh-based algorithm produces better motion compensation results than a block-based algorithm. Moreover, a two-layer mesh-based algorithm shows improvement over a one-layer mesh-based algorithm. The application of mesh-based motion estimation and compensation to video object replacement and animation is also performed.

In this thesis we study Particle Filter for visual tracking applications. The sequential Monte Carlo methods or Particle Filter provides approximate solutions when the tracking problem involves non-linear and/or non-Gaussian state space models. Also in this study, in order to make the visual tracker robust against change in target appearance and unexpected target motion, an adaptive target appearance model and a first order motion estimator are embedded in particle filtering. Additionally, since pixels that don&rsquo / t belong to target makes the motion estimation biased, the algorithm includes robust estimators to make the tracker reliable. Within the scope of this thesis the visual tracker proposed in [5] is implemented and the same problem is solved by proposing a Rao-Blackwellized Particle Filter. To deal with problems encountered during the implementation phase of the algorithm some improvements are made such as utilizing learning rate for the computation of adaptive velocity estimation. Moreover, some precautions are taken such as checking the velocity estimations to validate them. Finally, we have done several experiments both in indoor and outdoor environments to demonstrate the effectiveness and robustness of the implemented algorithm. Experimental results show that most of the time the visual tracker performs well. On the other hand the drawbacks of the implemented tracker are indicated and we explain how to eliminate them.