Nas últimas décadas, as texturas dinâmicas ou texturas temporais, que são texturas com movimento, tornaram-se objetos de intenso interesse por parte de pesquisadores das áreas de processamento digital de imagens e visão computacional. Várias técnicas vêm sendo desenvolvidas, ou aperfeiçoadas, para a extração de características baseada em texturas dinâmicas. Essas técnicas, em vários casos, são a combinação de duas ou mais metodologias pré-existentes que visam apenas a extração de características e não a melhora da qualidade das características extraídas. Além disso, para os casos em que as características são \"pobres\" em qualidade, o resultado final do processamento poderá apresentar queda de desempenho. Assim, este trabalho propõe descritores que extraiam características dinâmicas de sequências de vídeos e realize a fusão de informações buscando aumentar o desempenho geral na segmentação e/ou reconhecimento de texturas ou cenas em movimento. Os resultados obtidos utilizando-se duas bases de vídeos demonstram que os descritores propostos chamados de D-LMP e D-SLMP foram superiores ao descritor da literatura comparado e denominado de LBP-TOP. Além de apresentarem taxas globais de acurácia, precisão e sensibilidade superiores, os descritores propostos extraem características em um tempo inferior ao descritor LBP-TOP, o que os tornam mais práticos para a maioria das aplicações. A fusão de dados oriundos de regiões com diferentes características dinâmicas aumentou o desempenho dos descritores, demonstrando assim, que a técnica pode ser aplicada não somente para a classificação de texturas dinâmicas em sí, mas também para a classificação de cenas gerais em vídeos. / In the last decades, the dynamic textures or temporal textures, which are textures with movement, have become objects of intense interest on the part of researchers of the areas of digital image processing and computer vision. Several techniques have been developed, or perfected, for feature extraction based on dynamic textures. These techniques, in several cases, are the combination of two or more pre-existing methodologies that aim only the feature extraction and not the improvement of the quality of the extracted features. Moreover, in cases that the features are \"poor\" in quality, the final result of processing may present low performance. Thus, this work proposes descriptors that extract dynamic features of video sequences and perform the fusion of information seeking to increase the overall performance in the segmentation and/or recognition of textures or moving scenes. The results obtained using two video bases show that the proposed descriptors called D-LMP and D-SLMP were superior to the descriptor of the literature compared and denominated of LBP-TOP. In addition to presenting higher overall accuracy, precision and sensitivity rates, the proposed descriptors extract features at a shorter time than the LBP-TOP descriptor, which makes them more practical for most applications. The fusion of data from regions with different dynamic characteristics increased the performance of the descriptors, thus demonstrating that the technique can be applied not only to the classification of dynamic textures, but also to the classification of general scenes in videos.

Nas últimas décadas, as texturas dinâmicas ou texturas temporais, que são texturas com movimento, tornaram-se objetos de intenso interesse por parte de pesquisadores das áreas de processamento digital de imagens e visão computacional. Várias técnicas vêm sendo desenvolvidas, ou aperfeiçoadas, para a extração de características baseada em texturas dinâmicas. Essas técnicas, em vários casos, são a combinação de duas ou mais metodologias pré-existentes que visam apenas a extração de características e não a melhora da qualidade das características extraídas. Além disso, para os casos em que as características são \"pobres\" em qualidade, o resultado final do processamento poderá apresentar queda de desempenho. Assim, este trabalho propõe descritores que extraiam características dinâmicas de sequências de vídeos e realize a fusão de informações buscando aumentar o desempenho geral na segmentação e/ou reconhecimento de texturas ou cenas em movimento. Os resultados obtidos utilizando-se duas bases de vídeos demonstram que os descritores propostos chamados de D-LMP e D-SLMP foram superiores ao descritor da literatura comparado e denominado de LBP-TOP. Além de apresentarem taxas globais de acurácia, precisão e sensibilidade superiores, os descritores propostos extraem características em um tempo inferior ao descritor LBP-TOP, o que os tornam mais práticos para a maioria das aplicações. A fusão de dados oriundos de regiões com diferentes características dinâmicas aumentou o desempenho dos descritores, demonstrando assim, que a técnica pode ser aplicada não somente para a classificação de texturas dinâmicas em sí, mas também para a classificação de cenas gerais em vídeos. / In the last decades, the dynamic textures or temporal textures, which are textures with movement, have become objects of intense interest on the part of researchers of the areas of digital image processing and computer vision. Several techniques have been developed, or perfected, for feature extraction based on dynamic textures. These techniques, in several cases, are the combination of two or more pre-existing methodologies that aim only the feature extraction and not the improvement of the quality of the extracted features. Moreover, in cases that the features are \"poor\" in quality, the final result of processing may present low performance. Thus, this work proposes descriptors that extract dynamic features of video sequences and perform the fusion of information seeking to increase the overall performance in the segmentation and/or recognition of textures or moving scenes. The results obtained using two video bases show that the proposed descriptors called D-LMP and D-SLMP were superior to the descriptor of the literature compared and denominated of LBP-TOP. In addition to presenting higher overall accuracy, precision and sensitivity rates, the proposed descriptors extract features at a shorter time than the LBP-TOP descriptor, which makes them more practical for most applications. The fusion of data from regions with different dynamic characteristics increased the performance of the descriptors, thus demonstrating that the technique can be applied not only to the classification of dynamic textures, but also to the classification of general scenes in videos.

Textured motion - generally known as dynamic or temporal texture - is a popular research area for synthesis, segmentation and recognition. Dynamic texture is a spatially repetitive, time-varying visual pattern that forms an image sequence with certain temporal stationarity. In dynamic texture, the notion of self-similarity central to conventional image texture is extended to the spatiotemporal domain. Dynamic textures are typically videos of processes, such as waves, smoke, fire, a flag blowing in the wind, a moving escalator, or a walking crowd. Creation of synthetic frames is a key issue especially for movie screen industry to enrich their scenes from a white screen into a shining reality. In robotics world, for example an autonomous vehicle must decide what is traversable terrain (e.g. grass) and what is not (e.g. water). This problem can be addressed by classifying portions of the image into a number of categories, for instance grass, dirt, bushes or water. If these parts are identifiable, then segmentation and recognition of these textures results with an efficient path planning for the autonomous vehicle. In this thesis, we aimed to characterize these textured motions like mentioned above. We tried to implement several known techniques and compared the results.

No description available.

A análise de texturas dinâmicas tem se apresentado como uma área de pesquisa crescente e em potencial nos últimos anos em visão computacional. As texturas dinâmicas são sequências de imagens de textura (i.e. vídeo) que representam objetos dinâmicos. Exemplos de texturas dinâmicas são: evolução de colônia de bactérias, crescimento de tecidos do corpo humano, escada rolante em movimento, cachoeiras, fumaça, processo de corrosão de metal, entre outros. Apesar de existirem pesquisas relacionadas com o tema e de resultados promissores, a maioria dos métodos da literatura possui limitações. Além disso, em muitos casos as texturas dinâmicas são resultado de fenômenos complexos, tornando a tarefa de caracterização um desafio ainda maior. Esse cenário requer o desenvolvimento de um paradigma de métodos baseados em complexidade. A complexidade pode ser compreendida como uma medida de irregularidade das texturas dinâmicas, permitindo medir a estrutura dos pixels e quantificar os aspectos espaciais e temporais. Neste contexto, o objetivo deste mestrado é estudar e desenvolver métodos para caracterização de texturas dinâmicas baseado em metodologias de complexidade advindas da área de sistemas complexos. Em particular, duas metodologias já utilizadas em problemas de visão computacional são consideradas: redes complexas e caminhada determinística parcialmente auto-repulsiva. A partir dessas metodologias, três métodos de caracterização de texturas dinâmicas foram desenvolvidos: (i) baseado em difusão em redes - (ii) baseado em caminhada determinística parcialmente auto-repulsiva - (iii) baseado em redes geradas por caminhada determinística parcialmente auto-repulsiva. Os métodos desenvolvidos foram aplicados em problemas de nanotecnologia e tráfego de veículos, apresentando resultados potenciais e contribuindo para o desenvolvimento de ambas áreas. / Dynamic texture analysis has been an area of research increasing and in potential in recent years in computer vision. Dynamic textures are sequences of texture images (i.e. video) that represent dynamic objects. Examples of dynamic textures are: evolution of the colony of bacteria, growth of body tissues, moving escalator, waterfalls, smoke, process of metal corrosion, among others. Although there are researches related to the topic and promising results, most literature methods have limitations. Moreover, in many cases the dynamic textures are the result of complex phenomena, making a characterization task even more challenging. This scenario requires the development of a paradigm of methods based on complexity. The complexity can be understood as a measure of irregularity of the dynamic textures, allowing to measure the structure of the pixels and to quantify the spatial and temporal aspects. In this context, this masters aims to study and develop methods for the characterization of dynamic textures based on methodologies of complexity from the area of complex systems. In particular, two methodologies already used in computer vision problems are considered: complex networks and deterministic walk partially self-repulsive. Based on these methodologies, three methods of characterization of dynamic textures were developed: (i) based on diffusion in networks - (ii) based on deterministic walk partially self-repulsive - (iii) based on networks generated by deterministic walk partially self-repulsive. The developed methods were applied in problems of nanotechnology and vehicle traffic, presenting potencial results and contribuing to the development of both areas.

La détection et la reconnaissance d’objets dans des vidéos numériques est l’un des principaux challenges dans de nombreuses applications de vidéo surveillance. Dans le cadre de cette thèse, nous nous sommes attaqué au problème difficile de la segmentation d’objets dans des vidéos dont le fond est en mouvement permanent. Il s’agit de situations qui se produisent par exemple lorsque l’on filme des cours d’eau, ou le ciel,ou encore une scène contenant de la fumée, de la pluie, etc. Il s’agit d’un sujet assez peu étudié dans la littérature car très souvent les scènes traitées sont plutôt statiques et seules quelques parties bougent, telles que les feuillages par exemple, ou les seuls mouvements sont des changements de luminosité. La principale difficulté dans le cadre des scènes dont le fond est en mouvement est de différencier le mouvement de l’objet du mouvement du fond qui peuvent parfois être très similaires. En effet, par exemple, un objet dans une rivière peut se déplacer à la même allure que l’eau. Les algorithmes de la littérature extrayant des champs de déplacement échouent alors et ceux basés sur des modélisations de fond génèrent de très nombreuses erreurs. C’est donc dans ce cadre compliqué que nous avons tenté d’apporter des solutions.La segmentation d’objets pouvant se baser sur différents critères : couleur, texture,forme, mouvement, nous avons proposé différentes méthodes prenant en compte un ou plusieurs de ces critères.Dans un premier temps, nous avons travaillé dans un contexte bien précis qui était celui de la détection des bois morts dans des rivières. Ce problème nous a été apporté par des géographes avec qui nous avons collaboré dans le cadre du projet DADEC (Détection Automatique de Débris pour l’Aide à l’Etude des Crues). Dans ce cadre, nous avons proposé deux méthodes l’une dite " naïve " basée sur la couleur des objets à détecter et sur leur mouvement et l’autre, basée sur une approche probabiliste mettant en oeuvre une modélisation de la couleur de l’objet et également basée sur leur déplacement. Nous avons proposé une méthode pour le comptage des bois morts en utilisant les résultats des segmentations.Dans un deuxième temps, supposant la connaissance a priori du mouvement des objets,dans un contexte quelconque, nous avons proposé un modèle de mouvement de l’objet et avons montré que la prise en compte de cet a priori de mouvement permettait d’améliorer nettement les résultats des segmentations obtenus par les principaux algorithmes de modélisation de fond que l’on trouve dans la littérature.Enfin, dans un troisième temps, en s’inspirant de méthodes utilisées pour caractériser des textures 2D, nous avons proposé un modèle de fond basé sur une approche fréquentielle.Plus précisément, le modèle prend en compte non seulement le voisinage spatial d’un pixel mais également le voisinage temporel de ce dernier. Nous avons appliqué la transformée de Fourier locale au voisinage spatiotemporel d’un pixel pour construire un modèle de fond.Nous avons appliqué nos méthodes sur plusieurs vidéos, notamment les vidéos du projet DADEC, les vidéos de la base DynTex, des vidéos synthétiques et des vidéos que nous avons faites. / Moving object detection is one of the main challenges in many video monitoring applications.In this thesis, we address the difficult problem that consists in object segmentation when background moves permanently. Such situations occur when the background contains water flow, smoke or flames, snowfall, rainfall etc. Object detection in moving background was not studied much in the literature so far. Video backgrounds studied in the literature are often composed of static scenes or only contain a small portion of moving regions (for example, fluttering leaves or brightness changes). The main difficulty when we study such situations is to differentiate the objects movements and the background movements that may be almost similar. For example, an object in river moves at the same speed as water. Therefore, motion-based techniques of the literature, relying on displacements vectors in the scene, may fail to discriminate objects from the background, thus generating a lot of false detections. In this complex context, we propose some solutions for object detection.Object segmentation can be based on different criteria including color, texture, shape and motion. We propose various methods taking into account one or more of these criteria.We first work on the specific context of wood detection in rivers. It is a part of DADEC project (Détection Automatique de Débris pour l’Aide à l’Etude des Crues) in collaboration with geographers. We propose two approaches for wood detection: a naïve method and the probabilistic image model. The naïve approach is based on binary decisions based on object color and motion, whereas the probabilistic image model uses wood intensity distribution with pixel motion. Such detection methods are used fortracking and counting pieces of wood in rivers.Secondly, we consider a context in which we suppose a priori knowledge about objectmotion is available. Hence, we propose to model and incorporate this knowledge into the detection process. We show that combining this prior motion knowledge with classical background model improves object detection rate.Finally, drawing our inspiration from methods used for 2D texture representation, we propose to model moving backgrounds using a frequency-based approach. More precisely, the model takes into account the spatial neighborhoods of pixels but also their temporal neighborhoods. We apply local Fourier transform on the obtained regions in order to extract spatiotemporal color patterns.We apply our methods on multiple videos, including river videos under DADEC project, image sequences from the DynTex video database, several synthetic videos andsome of our own made videos. We compare our object detection results with the existing methods for real and synthetic videos quantitatively as well as qualitatively

Abstract Image description plays an important role in representing inherent properties of entities and scenes in static images. Within the last few decades, it has become a fundamental issue of many practical vision tasks, such as texture classification, face recognition, material categorization, and medical image processing. The study of static image analysis can also be extended to video analysis, such as dynamic texture recognition, classification and synthesis. This thesis contributes to the research and development of image and video analysis from two aspects. In the first part of this work, two image description methods are presented to provide discriminative representations for image classification. They are designed in unsupervised (i.e., class labels of texture images are not available) and supervised (i.e., class labels of texture images are available) manner, respectively. First, a supervised model is developed to learn discriminative local patterns, which formulates the image description as an integrated three-layered model to estimate an optimal pattern subset of interest by simultaneously considering the robustness, discriminative power and representation capability of features. Second, in the case that class labels of training images are unavailable, a linear configuration model is presented to describe microscopic image structures in an unsupervised manner, which is subsequently combined together with a local descriptor: local binary pattern (LBP). This description is theoretically verified to be rotation invariant and is able to provide a discriminative complement to the conventional LBPs. In the second part of the thesis, based on static image description and deformable image registration, video analysis is studied for the applications of dynamic texture description, synthesis and recognition. First, a dynamic texture synthesis model is proposed to create a continuous and infinitely varying stream of images given a finite input video, which stitches video clips in the time domain by selecting proper matching frames and organizing them into a logical order. Second, a method for the application of facial expression recognition, which formulates the dynamic facial expression recognition problem as the construction of longitudinal atlases and groupwise image registration problem, is proposed. / Tiivistelmä Kuvan deskriptiolla on tärkeä rooli staattisissa kuvissa esiintyvien luontaisten kokonaisuuksien ja näkymien kuvaamisessa. Viime vuosikymmeninä se on tullut perustavaa laatua olevaksi ongelmaksi monissa käytännön konenäön tehtävissä, kuten tekstuurien luokittelu, kasvojen tunnistaminen, materiaalien luokittelu ja lääketieteellisten kuvien analysointi. Staattisen kuva-analyysin tutkimusala voidaan myös laajentaa videoanalyysiin, kuten dynaamisten tekstuurien tunnistukseen, luokitteluun ja synteesiin. Tämä väitöskirjatutkimus myötävaikuttaa kuva- ja videoanalyysin tutkimukseen ja kehittymiseen kahdesta näkökulmasta. Työn ensimmäisessä osassa esitetään kaksi kuvan deskriptiomenetelmää erottelukykyisten esitystapojen luomiseksi kuvien luokitteluun. Ne suunnitellaan ohjaamattomiksi (eli tekstuurikuvien luokkien leimoja ei ole käytettävissä) tai ohjatuiksi (eli luokkien leimat ovat saatavilla). Aluksi kehitetään ohjattu malli oppimaan erottelukykyisiä paikallisia kuvioita, mikä formuloi kuvan deskriptiomenetelmän integroituna kolmikerroksisena mallina - tavoitteena estimoida optimaalinen kiinnostavien kuvioiden alijoukko ottamalla samanaikaisesti huomioon piirteiden robustisuus, erottelukyky ja esityskapasiteetti. Seuraavaksi, sellaisia tapauksia varten, joissa luokkaleimoja ei ole saatavilla, esitetään työssä lineaarinen konfiguraatiomalli kuvaamaan kuvan mikroskooppisia rakenteita ohjaamattomalla tavalla. Tätä käytetään sitten yhdessä paikallisen kuvaajan, eli local binary pattern (LBP) –operaattorin kanssa. Teoreettisella tarkastelulla osoitetaan kehitetyn kuvaajan olevan rotaatioinvariantti ja kykenevän tuottamaan erottelukykyistä, täydentävää informaatiota perinteiselle LBP-menetelmälle. Työn toisessa osassa tutkitaan videoanalyysiä, perustuen staattisen kuvan deskriptioon ja deformoituvaan kuvien rekisteröintiin – sovellusaloina dynaamisten tekstuurien kuvaaminen, synteesi ja tunnistaminen. Aluksi ehdotetaan sellainen malli dynaamisten tekstuurien synteesiin, joka luo jatkuvan ja äärettömän kuvien virran annetusta äärellisen mittaisesta videosta. Menetelmä liittää yhteen videon pätkiä aika-avaruudessa valitsemalla keskenään yhteensopivia kuvakehyksiä videosta ja järjestämällä ne loogiseen järjestykseen. Seuraavaksi työssä esitetään sellainen uusi menetelmä kasvojen ilmeiden tunnistukseen, joka formuloi dynaamisen kasvojen ilmeiden tunnistusongelman pitkittäissuuntaisten kartastojen rakentamisen ja ryhmäkohtaisen kuvien rekisteröinnin ongelmana.

Abstract This thesis investigates vision based description and recognition of human movements. Automated vision based human motion analysis is a fundamental technology for creating video based human computer interaction systems. Because of its wide range of potential applications, the topic has become an active area of research in the computer vision community. This thesis proposes the use of low level description of dynamics for human movement description and recognition. Two groups of approaches are developed: first, texture based methods that extract dynamic features for human movement description, and second, a framework that considers ballistic dynamics for human movement segmentation and recognition. Two texture based descriptions for human movement analysis are introduced. The first method uses the temporal templates as a preprocessing stage and extracts a motion description using local binary pattern texture features. This approach is then extended to a spatiotemporal space and a dynamic texture based method that uses local binary patterns from three orthogonal planes is proposed. The method needs no accurate segmentation of silhouettes, rather, it is designed to work on image data. The dynamic texture based description is also applied to gait recognition. The proposed descriptions have been experimentally validated on publicly available databases. Psychological studies on human movement indicate that common movements such as reaching and striking are ballistic by nature. Based on the psychological observations this thesis considers the segmentation and recognition of ballistic movements using low level motion features. Experimental results on motion capture and video data show the effectiveness of the method. / Tiivistelmä Tässä väitöskirjassa tutkitaan ihmisen liikkeen kuvaamista ja tunnistamista konenäkömenetelmillä. Ihmisen liikkeen automaattinen analyysi on keskeinen teknologia luotaessa videopohjaisia järjestelmiä ihmisen ja koneen vuorovaikutukseen. Laajojen sovellusmahdollisuuksiensa myötä aiheesta on tullut aktiivinen tutkimusalue konenäön tutkimuksen piirissä. Väitöskirjassa tutkitaan matalan tason piirteiden käyttöä ihmisen liikkeen dynaamiikan kuvaamiseen ja tunnistamiseen. Työssä esitetään kaksi tekstuuripohjaista mentelmää ihmisen liikkeen kuvaamiseen ja viitekehys ballististen liikkeiden segmentointiin ja tunnistamiseen. Työssä esitetään kaksi tekstuuripohjaista menetelmää ihmisen liikkeen analysointiin. Ensimmäinen menetelmä käyttää esikäsittelynä ajallisia kuvamalleja ja kuvaa mallit paikallisilla binäärikuvioilla. Menetelmä laajennetaan myös tila-aika-avaruuteen. Dynaamiseen tekstuuriin perustuva menetelmä irroittaa paikalliset binäärikuviot tila-aika-avaruuden kolmelta ortogonaaliselta tasolta. Menetelmä ei vaadi ihmisen siluetin tarkkaa segmentointia kuvista, koska se on suunniteltu toimimaan suoraan kuvatiedon perusteella. Dynaamiseen tekstuuriin pohjautuvaa menetelmää sovelletaan myös henkilön tunnistamiseen kävelytyylin perusteella. Esitetyt menetelmät on kokeellisesti vahvistettu yleisesti käytetyillä ja julkisesti saatavilla olevilla tietokannoilla. Psykologiset tutkimukset ihmisen liikkumisesta osoittavat, että yleiset liikkeet, kuten kurkoittaminen ja iskeminen, ovat luonteeltaan ballistisia. Tässä työssä tarkastellaan ihmisen liikkeen ajallista segmentointia ja tunnistamista matalan tason liikepiirteistä hyödyntäen psykologisia havaintoja. Kokeelliset tulokset liikkeenkaappaus ja video aineistolla osoittavat menetelmän toimivan hyvin.

A análise de texturas dinâmicas tem se apresentado como uma área de pesquisa crescente e em potencial nos últimos anos em visão computacional. As texturas dinâmicas são sequências de imagens de textura (i.e. vídeo) que representam objetos dinâmicos. Exemplos de texturas dinâmicas são: evolução de colônia de bactérias, crescimento de tecidos do corpo humano, escada rolante em movimento, cachoeiras, fumaça, processo de corrosão de metal, entre outros. Apesar de existirem pesquisas relacionadas com o tema e de resultados promissores, a maioria dos métodos da literatura possui limitações. Além disso, em muitos casos as texturas dinâmicas são resultado de fenômenos complexos, tornando a tarefa de caracterização um desafio ainda maior. Esse cenário requer o desenvolvimento de um paradigma de métodos baseados em complexidade. A complexidade pode ser compreendida como uma medida de irregularidade das texturas dinâmicas, permitindo medir a estrutura dos pixels e quantificar os aspectos espaciais e temporais. Neste contexto, o objetivo deste mestrado é estudar e desenvolver métodos para caracterização de texturas dinâmicas baseado em metodologias de complexidade advindas da área de sistemas complexos. Em particular, duas metodologias já utilizadas em problemas de visão computacional são consideradas: redes complexas e caminhada determinística parcialmente auto-repulsiva. A partir dessas metodologias, três métodos de caracterização de texturas dinâmicas foram desenvolvidos: (i) baseado em difusão em redes - (ii) baseado em caminhada determinística parcialmente auto-repulsiva - (iii) baseado em redes geradas por caminhada determinística parcialmente auto-repulsiva. Os métodos desenvolvidos foram aplicados em problemas de nanotecnologia e tráfego de veículos, apresentando resultados potenciais e contribuindo para o desenvolvimento de ambas áreas. / Dynamic texture analysis has been an area of research increasing and in potential in recent years in computer vision. Dynamic textures are sequences of texture images (i.e. video) that represent dynamic objects. Examples of dynamic textures are: evolution of the colony of bacteria, growth of body tissues, moving escalator, waterfalls, smoke, process of metal corrosion, among others. Although there are researches related to the topic and promising results, most literature methods have limitations. Moreover, in many cases the dynamic textures are the result of complex phenomena, making a characterization task even more challenging. This scenario requires the development of a paradigm of methods based on complexity. The complexity can be understood as a measure of irregularity of the dynamic textures, allowing to measure the structure of the pixels and to quantify the spatial and temporal aspects. In this context, this masters aims to study and develop methods for the characterization of dynamic textures based on methodologies of complexity from the area of complex systems. In particular, two methodologies already used in computer vision problems are considered: complex networks and deterministic walk partially self-repulsive. Based on these methodologies, three methods of characterization of dynamic textures were developed: (i) based on diffusion in networks - (ii) based on deterministic walk partially self-repulsive - (iii) based on networks generated by deterministic walk partially self-repulsive. The developed methods were applied in problems of nanotechnology and vehicle traffic, presenting potencial results and contribuing to the development of both areas.

In this dissertation, we address the problem of understanding human activities in videos by developing a two-pronged approach: coarse level modeling of scene activities and fine level modeling of individual activities. At the coarse level, where the resolution of the video is low, we rely on person tracks. At the fine level, richer features are available to identify different parts of the human body, therefore we rely on the body joint tracks. There are three main goals of this dissertation: (1) identify unusual activities at the coarse level, (2) recognize different activities at the fine level, and (3) predict the behavior for synthesizing and tracking activities at the fine level. The first goal is addressed by modeling activities at the coarse level through two novel and complementing approaches. The first approach learns the behavior of individuals by capturing the patterns of motion and size of objects in a compact model. Probability density function (pdf) at each pixel is modeled as a multivariate Gaussian Mixture Model (GMM), which is learnt using unsupervised expectation maximization (EM). In contrast, the second approach learns the interaction of object pairs concurrently present in the scene. This can be useful in detecting more complex activities than those modeled by the first approach. We use a 14-dimensional Kernel Density Estimation (KDE) that captures motion and size of concurrently tracked objects. The proposed models have been successfully used to automatically detect activities like unusual person drop-off and pickup, jaywalking, etc. The second and third goals of modeling human activities at the fine level are addressed by employing concepts from theory of chaos and non-linear dynamical systems. We show that the proposed model is useful for recognition and prediction of the underlying dynamics of human activities. We treat the trajectories of human body joints as the observed time series generated from an underlying dynamical system. The observed data is used to reconstruct a phase (or state) space of appropriate dimension by employing the delay-embedding technique. This transformation is performed without assuming an exact model of the underlying dynamics and provides a characteristic representation that will prove to be vital for recognition and prediction tasks. For recognition, properties of phase space are captured in terms of dynamical and metric invariants, which include the Lyapunov exponent, correlation integral, and correlation dimension. A composite feature vector containing these invariants represents the action and will be used for classification. For prediction, kernel regression is used in the phase space to compute predictions with a specified initial condition. This approach has the advantage of modeling dynamics without making any assumptions about the exact form (polynomial, radial basis, etc.) of the mapping function. We demonstrate the utility of these predictions for human activity synthesis and tracking.