Conditional Random Fields (CRFs) is a discriminative and supervised approach for simultaneous sequence segmentation and frame labeling. Latent-Dynamic Conditional Random Fields (LDCRFs) incorporates hidden state variables within CRFs which model sub-structure motion patterns and dynamics between labels. Motivated by the success of LDCRFs in gesture recognition, we propose a framework for automatic facial expression recognition from continuous video sequence by modeling temporal variations within shapes using LDCRFs. We show that the proposed approach outperforms CRFs for recognizing facial expressions. Using Principal Component Analysis (PCA) we study the separability of various expression classes in lower dimension projected spaces. By comparing the performance of CRFs and LDCRFs against that of Support Vector Machines (SVMs) and a template based approach, we demonstrate that temporal variations within shapes are crucial in classifying expressions especially for those with small facial motion like anger and sadness. We also show empirically that only using changes in facial appearance over time without using the shape variations fails to obtain high performance for facial expression recognition. This reflects the importance of geometric deformations on face for recognizing expressions. / text

No description available.

The increased concentration of greenhouse gases is associated with the global warming in the lower troposphere. For over twenty years, the space physics community has studied a hypothesis of global cooling in the thermosphere, attributable to greenhouse gases. While the global temperature increase in the lower troposphere has been relatively well established, the existence of global changes in the thermosphere is still under investigation. A central difficulty in reaching definite conclusions is the absence of data with sufficiently long temporal and sufficiently broad spatial coverage. Time series of data that cover several decades exist only in a few separated regions. The space physics community has struggled to combine the information contained in these data, and often contradictory conclusions have been reported based on the analyses relying on one or a few locations. To detect global changes in the ionosphere, we present a novel statistical methodology that uses all data, even those with incomplete temporal coverage. It is based on a new functional regression approach that can handle unevenly spaced, partially observed curves. While this research makes a solid contribution to the space physics community, our statistical methodology is very flexible and can be useful in other applied problems.

The increased concentration of greenhouse gases is associated with the global warming in the lower troposphere. For over twenty years, the space physics community has studied a hypothesis of global cooling in the thermosphere, attributable to greenhouse gases. While the global temperature increase in the lower troposphere has been relatively well established, the existence of global changes in the thermosphere is still under investigation. A central difficulty in reaching definite conclusions is the absence of data with sufficiently long temporal and sufficiently broad spatial coverage. Time series of data that cover several decades exist only in a few separated regions. The space physics community has struggled to combine the information contained in these data, and often contradictory conclusions have been reported based on the analyses relying on one or a few locations. To detect global changes in the ionosphere, we present a novel statistical methodology that uses all data, even those with incomplete temporal coverage. It is based on a new functional regression approach that can handle unevenly spaced, partially observed curves. While this research makes a solid contribution to the space physics community, our statistical methodology is very flexible and can be useful in other applied problems.

Philosophiae Doctor - PhD / This research investigated the application of recurrent neural networks (RNNs) for recognition of facial expressions based on facial action coding system (FACS). Support vector machines (SVMs) were used to validate the results obtained by RNNs. In this approach, instead of recognizing whole facial expressions, the focus was on the recognition of action units (AUs) that are defined in FACS. Recurrent neural networks are capable of gaining knowledge from temporal data while SVMs, which are time invariant, are known to be very good classifiers. Thus, the research consists of four important components: comparison of the use of image sequences against single static images, benchmarking feature selection and network optimization approaches, study of inter-AU correlations by implementing multiple output RNNs, and study of difference images as an approach for performance improvement. In the comparative studies, image sequences were classified using a combination of Gabor filters and RNNs, while single static images were classified using Gabor filters and SVMs. Sets of 11 FACS AUs were classified by both approaches, where a single RNN/SVM classifier was used for classifying each AU. Results indicated that classifying FACS AUs using image sequences yielded better results than using static images. The average recognition rate (RR) and false alarm rate (FAR) using image sequences was 82.75% and 7.61%, respectively, while the classification using single static images yielded a RR and FAR of 79.47% and 9.22%, respectively. The better performance by the use of image sequences can be at- tributed to RNNs ability, as stated above, to extract knowledge from time-series data. Subsequent research then investigated benchmarking dimensionality reduction, feature selection and network optimization techniques, in order to improve the performance provided by the use of image sequences. Results showed that an optimized network, using weight decay, gave best RR and FAR of 85.38% and 6.24%, respectively. The next study was of the inter-AU correlations existing in the Cohn-Kanade database and their effect on classification models. To accomplish this, a model was developed for the classification of a set of AUs by a single multiple output RNN. Results indicated that high inter-AU correlations do in fact aid classification models to gain more knowledge and, thus, perform better. However, this was limited to AUs that start and reach apex at almost the same time. This suggests the need for availability of a larger database of AUs, which could provide both individual and AU combinations for further investigation. The final part of this research investigated use of difference images to track the motion of image pixels. Difference images provide both noise and feature reduction, an aspect that was studied. Results showed that the use of difference image sequences provided the best results, with RR and FAR of 87.95% and 3.45%, respectively, which is shown to be significant when compared to use of normal image sequences classified using RNNs. In conclusion, the research demonstrates that use of RNNs for classification of image sequences is a new and improved paradigm for facial expression recognition.

As a core problem in video analysis, action recognition is of great significance for many higher-level tasks, both in research and industrial applications. With more and more video data being produced and shared daily, effective automatic action recognition methods are needed. Although, many deep-learning methods have been proposed to solve the problem, recent research reveals that single-stream, RGB-based networks are always outperformed by two-stream networks using both RGB and optical flow as inputs. This dependence on optical flow, which indicates a deficiency in learning motion, is present not only in 2D networks but also in 3D networks. This is somewhat surprising since 3D networks are explicitly designed for spatio-temporal learning. In this thesis, we assume that this deficiency is caused by difficulties associated with learning from videos exhibiting strong temporal variations, such as sudden motion, occlusions, acceleration, or deceleration. Temporal variations occur commonly in real-world videos and force a neural network to account for them, but often are not useful for recognizing actions at coarse granularity. We propose a Dynamic Equilibrium Module (DEM) for spatio-temporal learning through adaptive Eulerian motion manipulation. The proposed module can be inserted into existing networks with separate spatial and temporal convolutions, like the R(2+1)D model, to effectively handle temporal video variations and learn more robust spatio-temporal features. We demonstrate performance gains due to the use of DEM in the R(2+1)D model on miniKinetics, UCF-101, and HMDB-51 datasets.

No description available.

No description available.

No description available.

Neste trabalho e proposta a abordagem TempER-Tr, uma técnica de modelagem conceitual, fundamentada em rede de Petri, que integra a especificação das propriedades dinâmicas de um sistema a um modelo de dados temporal do tipo entidade relacionamento. Um modelo ou esquema conceitual descreve as propriedades identificadas de um sistema a ser desenvolvido. Estas propriedades podem ser classificadas em propriedades estáticas e propriedades dinâmicas As propriedades estáticas descrevem os estados que o sistema pode alcançar, enquanto que as propriedades dinâmicas descrevem as transições entre estes estados. A modelagem conceitual das propriedades estáticas é normalmente conhecida como modelagem de dados. A modelagem das propriedades dinâmicas é denominada de modelagem funcional ou comportamental. Mais especificamente, o modelo TempER-Tr é uma extensão de um trabalho anterior, conhecido como ER-Tr. No modelo ER-Tr, para descrever as propriedades estáticas de um sistema utiliza-se o modelo entidade-relacionamento convencional. No modelo TempER-Tr passa-se a adotar um modelo entidade-relacionamento temporal. Aliado a isto, uma nova linguagem de anotação, baseada em SQL, com mais poder de expressão é proposta. O modelo entidade-relacionamento convencional não possui dispositivos de modelagem capazes de especificar restrições que envolvam a associação dos objetos com o tempo, exigindo que isto se faca ao nível da modelagem das propriedades dinâmicas. Em um modelo entidade-relacionamento convencional, os conjuntos de entidades e relacionamentos apresentam apenas duas dimensões: a primeira refere-se as instâncias (linhas) e a segunda aos atributos (colunas). Em uma abordagem entidade relacionamento temporal, uma nova dimensão e acrescentada: o eixo temporal, possibilitando que as restrições temporais decorrentes da associação entre os objetos possam ser especificadas ao nível do modelo estático. Um requisito importante a ser preenchido por um modelo de dados temporal é permitir que em um mesmo diagrama seja possível associar objetos (entidades, relacionamentos ou atributos) temporalizados com objetos não temporalizados. lsto porque em sistemas de informação alguns dados precisam ser explicitamente referenciados ao tempo e outros não, ou porque não mudam com o tempo, ou porque é irrelevante ao usuário saber quando os fatos ocorreram. O modelo de dados temporal proposto neste trabalho, denominado TempER, pressupõe que todas as entidades, sejam elas temporalizadas ou não temporalizadas, apresentam uma "existência", ou seja, uma validade temporal. No caso das entidades temporalizadas esta existência é um subconjunto de pontos do eixo temporal. Em virtude disto são chamadas de entidades transitórias. Em relação as entidades não temporalizadas, e assumido que "existem sempre", ou seja, a sua validade temporal é constante, implícita e igual a todo o eixo temporal. Por isto são denominadas entidades perenes. Tanto as entidades transitórias quanto as entidades perenes, são focalizadas pelo modelo TempER através de duas perspectivas: uma intemporal e outra temporal. Através da perspectiva intemporal as entidades apresentam duas dimensões, semelhança do que ocorre em um modelo entidade-relacionamento convencional. Através da perspectiva temporal as entidades apresentam três dimensões, as duas convencionais e mais o tempo. Enquanto que o modelo de dados temporal descreve as propriedades estáticas de um sistema, o modelo comportamental, a outra face da abordagem TempER-Tr, focaliza as transações executadas no interior do sistema, em resposta a eventos que ocorrem no ambiente externo. Estas transações, quando efetivadas, provocam mudanças de estados no sistema. Entretanto, para estarem habilitadas a ocorrer, é necessário que um determinado conjunto de restrições dinâmicas sejam atendidas, o que se configura em um comportamento análogo ao de uma rede de Petri. O modelo TempER-Tr é completamente mapeável, inclusive o modelo de dados temporal, para a rede CEM, um tipo de rede de Petri de alto nível. Isto permite que a sua semântica seja formalmente especificada e possibilita o aproveitamento das características das redes de Petri. / This dissertation presents TempER-Tr approach. TempER-Tr is a conceptual modeling technique based on Petri nets that integrates the specification of the dynamic properties of system to a temporal entity-relationship data model. A model or conceptual schema describes the identified properties of a system. These properties can be classified into static and dynamic properties. The static properties describe the states that the system can reach, while the dynamic properties describe the transitions between the states. The conceptual modeling of the static properties is usually known as data modeling, while behavioral or functional modeling deals with dynamic properties. The TempER-Tr model is an extension of a model known as ER-Tr. In the ER-Tr model, the conventional entity-relationship model is used to describe the static properties of a system. In the TempER-Tr model, it is adopted a kind of temporal entityrelationship model. In addition, a new notation language is proposed, based on SQL, with more expression power. The conventional entity-relationship model doesn't provide tools to specify constraints that involve the association of objects with the time dimension, requiring that this have to be done at the dynamic properties modeling level. At the conventional entityrelationship model the entities and relationships sets present just two dimensions: the first one is related to the instance (lines) and the second to the attributes (columns). At a temporal entity-relationship approach, a new dimension is added: the time line. This way, the temporal constraints can be specified at the level of the static diagrams. An important requirement to be supplied by any temporal data model is the possibility to relate, into the same diagram, time-varying objects with time-invarying objects. This is due to the fact that in information systems some data need to be explicitly related to time and others don't, either because they don't change with time, or because users don't need to know when the facts occurred. • The temporal data model proposed in this work, nominated TempER, presupposes that all entities, being them time-varying or time-invarying, have an "existence", or a temporal validity. At the time-varying entities, named transitory entities, this existence is a subset of points from the time line. In time-invarying entities, named perennial entities, it is assumed that they "always exist", i.e., their temporal validity is constant, implicit, and equal to all points of the time line. Transitory entities, as much as perennial entities, are focused by the TempER model through two perspectives: a temporal perspective and a non-temporal perspective. Through the non-temporal perspective the entities present two dimensions - lines and columns - similar to a conventional entity-relationship model. Through the temporal perspective the entities present three dimensions: the two conventional dimensions and, in addition, the time dimension. While the temporal data model describes the static properties of a system, the behavioral model in the TempER-Tr approach focus the transactions that are executed by the system, in response to the events that occur at the external environment. A certain set of dynamic constraints must be attended so that transactions are enable to occur. This configures a behavior similar to a Petri net. The TempER-Tr model is completely mappeable, inclusive the temporal data model, to the CEM net, a kind of high level Petri net. This way, the semantic of TempER-Tr model is formally specified. In addition, the utilization of the characteristics of Petri nets is possible.