Modern computer vision has seen recently significant progress in learning visual concepts from examples. This progress has been fuelled by recent models of visual appearance as well as recently collected large-scale datasets of manually annotated still images. Video is a promising alternative, as it inherently contains much richer information compared to still images. For instance, in video we can observe an object move which allows us to differentiate it from its surroundings, or we can observe a smooth transition between different viewpoints of the same object instance. This richness in information allows us to effectively tackle tasks that would otherwise be very difficult if we only considered still images, or even adress tasks that are video-specific. Our first contribution is a computationally efficient technique for video object segmentation. Our method relies solely on motion in order to rapidly create a rough initial estimate of the foreground object. This rough initial estimate is then refined through an energy formulation to be spatio-temporally smooth. The method is able to handle rapidly moving backgrounds and objects, as well as non-rigid deformations and articulations without having prior knowledge about the objects appearance, size or location. In addition to this class-agnostic method, we present a class-specific method that incorporates additional class-specific appearance cues when the class of the foreground object is known in advance (e.g. a video of a car). For our second contribution, we propose a novel model for temporal video alignment with regard to the viewpoint of the foreground object (i.e., a pair of aligned frames shows the same object viewpoint) Our work relies on our video object segmentation technique to automatically localise the foreground objects and extract appearance measurements solely from them instead of the background. Our model is able to temporally align realistic videos, where events may occur in a different order, or occur only in one of the videos. This is in contrast to previous works that typically assume that the videos show a scripted sequence of events and can simply be aligned by stretching or compressing one of the videos. As a final contribution, we once again use our video object segmentation technique as a basis for automatic visual aspect discovery from videos of an object class. Compared to previous works, we use a broader definition of an aspect that considers four factors of variation: viewpoint, articulated pose, occlusions and cropping by the image border. We pose the aspect discovery task as a clustering problem and provide an extensive experimental exploration on the benefits of object segmentation for this task.

International Telemetering Conference Proceedings / October 20-23, 2003 / Riviera Hotel and Convention Center, Las Vegas, Nevada / In many test ranges, it is often required to acquire a number of telemetry streams and to process the data simultaneously. Frequently, the streams have different delay characteristics, requiring temporal alignment before the processing step. It is desired to have the capability to align these streams so that the events in each stream are coincident in time. Terawave Communications has developed technology to perform temporal alignment for a number of streams automatically. Additionally, the algorithm performs the delay compensation independent of the source data rate of each stream. Terawave will present the algorithm and share the results of their testing in a test installation.

Many computer vision problems, such as object classification, motion estimation or shape registration rely on solving the correspondence problem. Existing algorithms to solve spatial or temporal correspondence problems are usually NP-hard, difficult to approximate, lack flexible models and mechanism for feature weighting. This proposal addresses the correspondence problem in computer vision, and proposes two new spatio-temporal correspondence problems and three algorithms to solve spatial, temporal and spatio-temporal matching between video and other sources. The main contributions of the thesis are: (1) Factorial graph matching (FGM). FGM extends existing work on graph matching (GM) by finding an exact factorization of the affinity matrix. Four are the benefits that follow from this factorization: (a) There is no need to compute the costly (in space and time) pairwise affinity matrix; (b) It provides a unified framework that reveals commonalities and differences between GM methods. Moreover, the factorization provides a clean connection with other matching algorithms such as iterative closest point; (c) The factorization allows the use of a path-following optimization algorithm, that leads to improved optimization strategies and matching performance; (d) Given the factorization, it becomes straight-forward to incorporate geometric transformations (rigid and non-rigid) to the GM problem. (2) Canonical time warping (CTW). CTW is a technique to temporally align multiple multi-dimensional and multi-modal time series. CTW extends DTW by incorporating a feature weighting layer to adapt different modalities, allowing a more flexible warping as combination of monotonic functions, and has linear complexity (unlike DTW that has quadratic). We applied CTW to align human motion captured with different sensors (e.g., audio, video, accelerometers). (3) Spatio-temporal matching (STM). Given a video and a 3D motion capture model, STM finds the correspondence between subsets of video trajectories and the motion capture model. STM is efficiently and robustly solved using linear programming. We illustrate the performance of STM on the problem of human detection in video, and show how STM achieves state-of-the-art performance.

Made available in DSpace on 2015-04-11T14:03:15Z (GMT). No. of bitstreams: 1 DISSERTACAO DIEGO FRONER.pdf: 2152405 bytes, checksum: e2cee67bca7f2b5d58460ec7502da76c (MD5) Previous issue date: 2012-02-29 / Fundação de Amparo à Pesquisa do Estado do Amazonas / This work presents the use of different sensors improving the information to perform spatio-temporal alignment of sequential images. The existing proposals called feature-based uses the dynamics of scenes as a major indicator that simultaneously events are ocurring at the same time, and lately indicating their relative position in space. Adding motion sensors to a multiple video cameras system with overlapping fields of coverage, it s possible to acquire information about positions of the monitored objects that serves to aid the alignment between images. / Este trabalho apresenta a utilização de diferentes sensores no aprimoramento da informação necessária para realizar o alinhamento espaço-temporal de imagens sequenciais. As propostas existentes chamadas feature-based utilizam-se da dinâmica da cena como maior indicador de que eventos estão ocorrendo simultaneamente no tempo, e assim posteriormente indicando suas posições relativas no espaço. Adicionando sensores de movimentação a um sistema com múltiplas câmeras de vídeo que possuam sobreposição de campos de cobertura, é possível adquirir informações de posicionamento dos objetos monitorados, servindo assim de auxílio para o alinhamento entre as imagens.

Many applications use multiple cameras to simultaneously capture imagery of a scene from different vantage points on a rigid, moving camera system over time. Multiple cameras often provide unique viewing angles but also additional levels of detail of a scene at different spatio-temporal resolutions. However, in order to benefit from this added information the sources must be temporally aligned. As a result of cost and physical limitations it is often impractical to synchronize these sources via an external clock device. Most methods attempt synchronization through the recovery of a constant scale factor and offset with respect to time. This limits the generality of such alignment solutions. We present an unsupervised method that utilizes a content-based clustering mechanism in order to temporally align multiple non-synchronized image sequences of different and varying spatio-temporal resolutions. We show that the use of temporal constraints and dynamic programming adds robustness to changes in capture rates, field of view, and resolution.

Dans cette thèse nous proposons des algorithmes pour le suivi 3D du mouvement des objects déformables à partir de plusieurs caméras vidéo. Bien qu’une suite de reconstructions tridimensionnelles peut être obtenue par des méthodes de reconstruction statique, celle-ci ne représente pas le mouvement. Nous voulons produire une représentation temporellement cohérente de la suite de formes prises par l’object. Précisément, nous souhaitons représenter l’objet par une surface maillée 3D dont les sommets se déplacent au cours du temps mais dont la topologie reste identique.Contrairement à beaucoup d’approches existantes, nous proposons de représenter le mouvement du volume intérieur des formes, dans le but de mieux représenter la nature volumétrique des objets. Nous traitons de manière volumétrique les problèmes fondamentaux du suivi déformable que sont l’association d’éléments semblables entre deux formes et la modélisation de la déformation. En particulier, nous adaptons au formes volumétriques les modèles d’association EM-ICP non-rigide ansi que l’association par détection par apprentissage automatique.D’autre part, nous abordons la question de la modélisation de l’évolution temporelle de la déformation au cours d’une séquence dans le but de mieux contraindre le problème du suivi temporel. Pour cela, nous modélisons un espace de forme construit autour de propriétés de déformations locales que nous apprenons automatiqument lors du suivi.Nous validons nos algorithmes de suivi sur des séquences vidéo multi-caméras avec vérité terrain (silhouettes et suivi par marqueurs). Nos résultats se révèlent meilleurs ou équivalents à ceux obtenus avec les méthodes de l’état de l’art.Enfin, nous démontrons que le suivi volumétrique et la représentation que nous avons choisie permettent de produire des animations 3D qui combinent l’acquisition et la simulation de mouvement. / In this thesis we propose algorithms for tracking 3D deformable shapes in motion from multiview video. Although series of reconstructed 3D shapes can be obtained by applying a static reconstruction algorithm to each temporal frame independently, such series do not represent motion. Instead, we want to provide a temporally coherent representation of the sequence of shapes resulting from temporal evolutions of a shape. Precisely, we want to represent the observed shape sequence as a 3D surface mesh whose vertices move in time but whose topology is constant.In contrast with most existing approaches, we propose to represent the motion of inner shape volumes, with the aim of better accounting for the volumetric nature of the observed object. We provide a fully volumetric approach to the fundamental problems of deformable shape tracking, which are the association between corresponding shape elements and the deformation model. In particular, we extend to a volumetric shape representation the EM-ICP tracking framework and the association-by-detection strategy.Furthermore, in order to better constrain the shape tracking problem, we propose a model for the temporal evolution of deformation. Our deformation model defines a shape space parametrized by variables that capture local deformation properties of the shape and whose values are automatically learned during the tracking process.We validate our tracking algorithms on several multiview video sequences with ground truth (silhouette and marker-based tracking). Our results are better or comparable to state of the art approaches.Finally, we show that volumetric tracking and the shape representation we choose can be leveraged for producing shape animations which combine captured and simulatated motion.

L’alignement de multiples séries temporelles est un problème important non résolu dans de nombreuses disciplines scientifiques. Les principaux défis pour l’alignement temporel de multiples séries comprennent la détermination et la modélisation des caractéristiques communes et différentielles de classes de séries. Cette thèse est motivée par des travaux récents portant sur l'extension de la DTW pour l’alignement de séries multiples issues d’applications diverses incluant la reconnaissance vocale, l'analyse de données micro-array, la segmentation ou l’analyse de mouvements humain. Ces travaux fondés sur l’extension de la DTW souffrent cependant de plusieurs limites : 1) Ils se limitent au problème de l'alignement par pair de séries 2) Ils impliquent uniformément les descripteurs des séries 3) Les alignements opérés sont globaux. L'objectif de cette thèse est d'explorer de nouvelles approches d’alignement temporel pour la classification non supervisée de séries. Ce travail comprend d'abord le problème de l'extraction de prototypes, puis de l'alignement de séries multiples multidimensionnelles. / Temporal alignment of multiple time series is an important unresolved problem in many scientific disciplines. Major challenges for an accurate temporal alignment include determining and modeling the common and differential characteristics of classes of time series. This thesis is motivated by recent works in extending Dynamic time warping for aligning multiple time series from several applications including speech recognition, curve matching, micro-array data analysis, temporal segmentation or human motion. However these DTW-based works suffer of several limitations: 1) They address the problem of aligning two time series regardless of the remaining time series, 2) They involve uniformly the features of the multiple time series, 3) The time series are aligned globally by including the whole observations. The aim of this thesis is to explore a generalized dynamic time warping for time series clustering. This work includes first the problem of prototype extraction, then the alignment of multiple and multidimensional time series.

Das Finden von zuverlässigen Informationen über gegebene Ereignisse aus großen und dynamischen Textsammlungen, wie dem Web, ist ein wichtiges Thema. Zum Beispiel sind Rettungsteams und Versicherungsunternehmen an prägnanten Fakten über Schäden nach Katastrophen interessiert, die heutzutage online in Web-Blogs, Zeitungsartikeln, Social Media etc. zu finden sind. Solche Fakten helfen, die erforderlichen Hilfsmaßnahmen zu bestimmen und unterstützen deren Koordination. Allerdings ist das Finden, Extrahieren und Aggregieren nützlicher Informationen ein hochkomplexes Unterfangen: Es erfordert die Ermittlung geeigneter Textquellen und deren zeitliche Einordung, die Extraktion relevanter Fakten in diesen Texten und deren Aggregation zu einer verdichteten Sicht auf die Ereignisse, trotz Inkonsistenzen, vagen Angaben und Veränderungen über die Zeit. In dieser Arbeit präsentieren und evaluieren wir Techniken und Lösungen für jedes dieser Probleme, eingebettet in ein vierstufiges Framework. Die angewandten Methoden beruhen auf Verfahren des Musterabgleichs, der Verarbeitung natürlicher Sprache und des maschinellen Lernens. Zusätzlich berichten wir über die Ergebnisse zweier Fallstudien, basierend auf dem Einsatz des gesamten Frameworks: Die Ermittlung von Daten über Erdbeben und Überschwemmungen aus Webdokumenten. Unsere Ergebnisse zeigen, dass es unter bestimmten Umständen möglich ist, automatisch zuverlässige und zeitgerechte Daten aus dem Internet zu erhalten. / Finding reliable information about given events from large and dynamic text collections, such as the web, is a topic of great interest. For instance, rescue teams and insurance companies are interested in concise facts about damages after disasters, which can be found today in web blogs, online newspaper articles, social media, etc. Knowing these facts helps to determine the required scale of relief operations and supports their coordination. However, finding, extracting, and condensing specific facts is a highly complex undertaking: It requires identifying appropriate textual sources and their temporal alignment, recognizing relevant facts within these texts, and aggregating extracted facts into a condensed answer despite inconsistencies, uncertainty, and changes over time. In this thesis, we present and evaluate techniques and solutions for each of these problems, embedded in a four-step framework. Applied methods are pattern matching, natural language processing, and machine learning. We also report the results for two case studies applying our entire framework: gathering data on earthquakes and floods from web documents. Our results show that it is, under certain circumstances, possible to automatically obtain reliable and timely data from the web.