Research that involves human-beings depends on the data collection. As technology solutions become popular in the context of healthcare, researchers highlight the need for monitoring and caring patients in situ. Human Activity Recognition (HAR) is a research field that combines two areas: Ubiquitous Computing and Artificial Intelligence. HAR is daily applied in several service sectors including military, security (surveillance), health and entertainment. A HAR system aims to identify and recognize the activities and actions a user performs, in real time or not. Ambient sensors (e.g. cameras) and wearable devices (e.g. smartwatches) collect information about users and their context (e.g. localization, time, companions). This data is processed by machine learning algorithms that extract information and classify the corresponding activity. Although there are several works in the literature related to HAR systems, most studies focusing on elderly users are limited and do not use, as ground truth, data collected from elder volunteers. Databases and sensors reported in the literature are geared towards a generic audience, which leads to loss in accuracy and robustness when targeted at a specific audience. Considering this gap, this work presents a Human Activity Recognition system and corresponding database focusing on the elderly, raising requirements and guidelines for supportive HAR system and the selection of sensor devices. The system evaluation was carried out checking the accuracy of the activity recognition process, defining the best statistical features and classification algorithms for the Elderly Activity Recognition System (EARS). The results suggest that EARS is a promising supportive technology for the elderly, having an accuracy of 98.37% with KNN (k = 1). / Pesquisas e serviços no campo da saúde se valem da coleta, em tempo real ou não, de dados de ordem física, psicológica, sentimental, comportamental, entre outras, de pacientes ou participantes em experimentos: o objetivo é melhorar tratamentos e procedimentos. As soluções tecnológicas estão se tornando populares no contexto da saúde, pesquisadores da área de saúde destacam a necessidade de monitoramento e cuidado dos pacientes in situ. O campo de pesquisa de Reconhecimento de Atividade Humana (sigla em inglês HAR, Human Activity Recognition) envolve as áreas de computação ubíqua e de inteligência artificial, sendo aplicado nos mais diversos domínios. Com o uso de sensores como câmeras, microfones e acelerômetros, entre outros, um sistema HAR tem por tarefa identificar as atividades que uma pessoa realiza em um determinado momento. As informações coletadas pelos sensores e os dados do usuário são processados por algoritmos de aprendizado de máquina para identificar a atividade humana realizada. Apesar de existirem vários trabalhos na literatura de sistemas HAR, poucos são voltados para o público ancião. Bases de dados e sensores reportados em trabalhos relacionados são voltadas para um público genérico, perdendo precisão e robustez quando se trata de um público específico. Diante dessa lacuna, este trabalho propõe um sistema de Reconhecimento de Atividade Humana voltado para o idoso, levantando requisitos para o sistema HAR assistido e selecionando os dispositivos sensores. Um banco de dados HAR com dados coletados de voluntários mais velhos também é fornecido e disponibilizado. A avaliação do sistema foi realizada verificando a acurácia do processo de reconhecimento da atividade, definindo as melhores características estatísticas e algoritmos de classificação para o sistema de reconhecimento de atividades do idoso. Os resultados sugerem que esse sistema é uma tecnologia de suporte promissora para idosos, tendo uma acurácia de 98.37% com KNN (k = 1).

Predicting and understanding the dynamic of human motion has many applications such as motion synthesis, augmented reality, security, education, reinforcement learning, autonomous vehicles, and many others. In this thesis, we create a novel end-to-end pipeline that can predict multiple future poses from the same input, and, in addition, can classify the entire sequence. Our focus is on the following two aspects of human motion understanding: Probabilistic human action prediction: Given a sequence of human poses as input, we sample multiple possible future poses from the same input sequence using a new GAN-based network. Human motion understanding: Given a sequence of human poses as input, we classify the actual action performed in the sequence and improve the classification performance using the presentation learned from the prediction network. We also demonstrate how to improve model training from noisy labels, using facial expression recognition as an example. More specifically, we have 10 taggers to label each input image, and compare four different approaches: majority voting, multi-label learning, probabilistic label drawing, and cross-entropy loss. We show that the traditional majority voting scheme does not perform as well as the last two approaches that fully leverage the label distribution. We shared the enhanced FER+ data set with multiple labels for each face image with the research community (https://github.com/Microsoft/FERPlus). For predicting and understanding of human motion, we propose a novel sequence-to-sequence model trained with an improved version of generative adversarial networks (GAN). Our model, which we call HP-GAN2, learns a probability density function of future human poses conditioned on previous poses. It predicts multiple sequences of possible future human poses, each from the same input sequence but seeded with a different vector z drawn from a random distribution. Moreover, to quantify the quality of the non-deterministic predictions, we simultaneously train a motion-quality-assessment model that learns the probability that a given skeleton pose sequence is a real or fake human motion. In order to classify the action performed in a video clip, we took two approaches. In the first approach, we train on a sequence of skeleton poses from scratch using random parameters initialization with the same network architecture used in the discriminator of the HP-GAN2 model. For the second approach, we use the discriminator of the HP-GAN2 network, extend it with an action classification branch, and fine tune the end-to-end model on the classification tasks, since the discriminator in HP-GAN2 learned to differentiate between fake and real human motion. So, our hypothesis is that if the discriminator network can differentiate between synthetic and real skeleton poses, then it also has learned some of the dynamics of a real human motion, and that those dynamics are useful in classification as well. We will show through multiple experiments that that is indeed the case. Therefore, our model learns to predict multiple future sequences of human poses from the same input sequence. We also show that the discriminator learns a general representation of human motion by using the learned features in an action recognition task. And we train a motion-quality-assessment network that measure the probability of a given sequence of poses are valid human poses or not. We test our model on two of the largest human pose datasets: NTURGB-D, and Human3.6M. We train on both single and multiple action types. The predictive power of our model for motion estimation is demonstrated by generating multiple plausible futures from the same input and showing the effect of each of the several loss functions in the ablation study. We also show the advantage of switching to GAN from WGAN-GP, which we used in our previous work. Furthermore, we show that it takes less than half the number of epochs to train an activity recognition network by using the features learned from the discriminator.

Human activity recognition (HAR) based on time series data is the problem of classifying various patterns. Its widely applications in health care owns huge commercial benefit. With the increasing spread of smart devices, people have strong desires of customizing services or product adaptive to their features. Deep learning models could handle HAR tasks with a satisfied result. However, training a deep learning model has to consume lots of time and computation resource. Consequently, developing a HAR system effectively becomes a challenging task. In this study, we develop a solid HAR system using Convolutional Neural Network based on transfer learning, which can eliminate those barriers.

Human activity analysis has attracted great interest from computer vision researchers due to its promising applications in many areas such as automated visual surveillance, computer-human interactions, and motion-based identification and diagnosis. This dissertation presents work in two areas: general human activity recognition from video, and human activity analysis for the purpose of identifying pathological gait from both 3D captured data and from video. Even though the research in human activity recognition has been going on for many years, still there are many issues that need more research. This includes the effective representation and modeling of human activities and the segmentation of sequences of continuous activities. In this thesis we present an algorithm that combines shape and motion features to represent human activities. In order to handle the activity recognition from any viewing angle we quantize the viewing direction and build a set of Hidden Markov Models (HMMs), where each model represents the activity from a given view. Finally, a voting based algorithm is used to segment and recognize a sequence of human activities from video. Our method of representing activities has good attributes and is suitable for both low resolution and high resolution video. The voting based algorithm performs the segmentation and recognition simultaneously. Experiments on two sets of video clips of different activities show that our method is effective. Our work on identifying pathological gait is based on the assumption of gait symmetry. Previous work on gait analysis measures the symmetry of gait based on Ground Reaction Force data, stance time, swing time or step length. Since the trajectories of the body parts contain information about the whole body movement, we measure the symmetry of the gait based on the trajectories of the body parts. Two algorithms, which can work with different data sources, are presented. The first algorithm works on 3D motion-captured data and the second works on video data. Both algorithms use support vector machine (SVM) for classification. Each of the two methods has three steps: the first step is data preparation, i.e., obtaining the trajectories of the body parts; the second step is gait representation based on a measure of gait symmetry; and the last step is SVM based classification. For 3D motion-captured data, a set of features based on Discrete Fourier Transform (DFT) is used to represent the gait. We demonstrate the accuracy of the classification by a set of experiments that shows that the method for 3D motion-captured data is highly effective. For video data, a model based tracking algorithm for human body parts is developed for preparing the data. Then, a symmetry measure that works on the sequence of 2D data, i.e. sequence of video frames, is derived to represent the gait. We performed experiments on both 2D projected data and real video data to examine this algorithm. The experimental results on 2D projected data showed that the presented algorithm is promising for identifying pathological gait from video. The experimental results on the real video data are not good as the results on 2D projected data. We believe that better results could be obtained if the accuracy of the tracking algorithm is improved.

Thesis (M.S.)--University of North Texas, Dec., 2009. / Title from title page display. Includes bibliographical references.

Αντικείμενο της εργασίας αυτής είναι η μελέτη και κατασκευή εφαρμογής ασύρματου δικτύου με σκοπό την αναγνώριση της ανθρώπινης δραστηριότητας και τον εντοπισμό της πτώσης σε πραγματικό χρόνο, καθώς επίσης και την παρακολούθηση των αποτελεσμάτων από απομακρυσμένη τοποθεσία. Στη συγκεκριμένη εργασία σκοπός είναι η αναγνώριση των τεσσάρων βασικών καταστάσεων της ανθρώπινης φυσικής δραστηριότητας (κάθομαι, ξαπλώνω, στέκομαι, κινούμαι) και ο εντοπισμός της πτώσης με χρήση της των επιταχυνσιομέτρων που προσφέρει η πλατφόρμα SunSpot καθώς και η σύνδεση της διάταξης με το διαδίκτυο για την παροχή πληροφορίας σχετικά με την κατάσταση του κατόχου του συστήματος σε απομακρυσμένη τοποθεσία. Πραγματοποιήθηκε μελέτη σχετικά με διάφορες διατάξεις των αισθητήρων ,την συχνότητα δειγματοληψίας, τους αλγορίθμους κατάταξης καθώς και για τις μεθόδους διάθεσης της πληροφορίας στο διαδίκτυο. Για την αναγνώριση των καταστάσεων και τον εντοπισμό της πτώσης χρησιμοποιήθηκαν δυο πλατφόρμες αισθητήρων SunSPOT, μια στο στήθος (master) και μια στο δεξιό τετρακέφαλο (slave) / A wearable wireless sensor network application performing human activity recognition and fall detection using the Naïve Bayesian Classifier algorithm in the SunSpot Platform accompanied by a web application in the Google App Engine platform to be able to monitor the classification results from a remote location and to automatically notify via e-mail in case of emergency.

L'objectif de cette thèse est de concevoir une approche transparente originale apte à détecter en temps-réel l'occurrence d'une action, dans un flot non segmenté et idéalement le plus tôt possible. Ces travaux s'inscrivent dans une collaboration entre deux équipes de l'IRISA-lnria de Rennes, à savoir lntuidoc et MimeTIC. En profitant de la complémentarité des savoir-faire des deux équipes de recherche, nous proposons de reconsidérer les besoins et les difficultés rencontrées pour modéliser, reconnaître et détecter une action 30 en proposant de nouvelles solutions à la lumière des avancées réalisées en termes de modélisation de gestes manuscrits 20. Les contributions de cette thèse sont regroupées en trois parties principales. Dans la première partie, nous proposons une nouvelle approche pour modéliser et reconnaître une action pré­segmentée. Dans la deuxième partie, nous introduisons une approche permettant de reconnaître une action dans un flot non segmenté. Enfin, dans la troisième partie, nous étendons cette dernière approche pour la caractérisation précoce d'une action avec très peu de d'information. Pour chacune de ces trois problématiques, nous avons identifié explicitement les difficultés à considérer afin d'en effectuer une description complète pour permettre de concevoir des solutions ciblées pour chacune d'elles. Les résultats expérimentaux obtenus sur différents benchmarks d'actions attestent de la validité de notre démarche. En outre, à travers des coopérations ayant eu lieu au cours de la thèse, les approches développées ont été déployées dans trois applications, dont des applications en animation et en reconnaissance de gestes dynamiques de la main. / The objective of this thesis is to design an original transparent approach able to detect in real time the occurrence of an action, in an unsegmented flow and ideally as early as possible. This work is part of a collaboration between two IRISA-lnria teams in Rennes, namely lntuidoc and Mime TIC. By taking advantage of the complementary expertise of the two research teams, we propose to reconsider the needs and difficulties encountered to model, recognize and detect a 30 action by proposing new solutions in the light of the advances made in terms of 20 handwriting modeling. The contributions of this thesis are grouped into three main parts. In the first part, we propose a new approach to model and recognize a pre-segmented action. Indeed, it is first necessary to develop a representation able to characterize as finely as possible a given action to facilitate recognition. In the second part, we introduce an approach to recognize an action in an unsegmented flow. Finally, in the third part, we extend this last approach for the early characterization of an action with very little information. For each of these three issues, we have explicitly identified the difficulties to be considered in order to make a complete description of them so that we can design targeted solutions for each of them. The experimental results obtained on different benchmarks of actions attest to the validity of our approach. In addition, through collaborations that took place during the thesis, the developed approaches were deployed in three applications, including applications in animation and in dynamic hand gestures recognition.

abstract: The human motion is defined as an amalgamation of several physical traits such as bipedal locomotion, posture and manual dexterity, and mental expectation. In addition to the “positive” body form defined by these traits, casting light on the body produces a “negative” of the body: its shadow. We often interchangeably use with silhouettes in the place of shadow to emphasize indifference to interior features. In a manner of speaking, the shadow is an alter ego that imitates the individual. The principal value of shadow is its non-invasive behaviour of reflecting precisely the actions of the individual it is attached to. Nonetheless we can still think of the body’s shadow not as the body but its alter ego. Based on this premise, my thesis creates an experiential system that extracts the data related to the contour of your human shape and gives it a texture and life of its own, so as to emulate your movements and postures, and to be your extension. In technical terms, my thesis extracts abstraction from a pre-indexed database that could be generated from an offline data set or in real time to complement these actions of a user in front of a low-cost optical motion capture device like the Microsoft Kinect. This notion could be the system’s interpretation of the action which creates modularized art through the abstraction’s ‘similarity’ to the live action. Through my research, I have developed a stable system that tackles various connotations associated with shadows and the need to determine the ideal features that contribute to the relevance of the actions performed. The implication of Factor Oracle [3] pattern interpretation is tested with a feature bin of videos. The system also is flexible towards several methods of Nearest Neighbours searches and a machine learning module to derive the same output. The overall purpose is to establish this in real time and provide a constant feedback to the user. This can be expanded to handle larger dynamic data. In addition to estimating human actions, my thesis best tries to test various Nearest Neighbour search methods in real time depending upon the data stream. This provides a basis to understand varying parameters that complement human activity recognition and feature matching in real time. / Dissertation/Thesis / Masters Thesis Electrical Engineering 2016

Human-machine interaction could support many daily activities in making it more convenient. The development of smart devices has flourished the underlying smart systems that process smart and personalized control of devices. The first step in controlling any device is observation; through understanding the surrounding environment and human activity, a smart system can physically control a device. Human activity recognition (HAR) is essential in many smart applications such as self-driving cars, human-robot interaction, and automatic systems such as infrared (IR) taps. For human-centric systems, there are some requirements to perform a physical task in real-time. For human-machine interactions, the anticipation of human actions is essential. IR taps have delay limitations because of the proximity sensor that signals the solenoid valve only when the user’s hands are exactly below the tap. The hardware and electronics delay causes inconvenience in use and water waste. In this thesis, an alternative control based on deep learning action anticipation is proposed. Humans interact with taps for various tasks such as washing hands, face, brushing teeth, just to name a few. We focus on a small subset of these activities. Specifically, we focus on the activities carried out sequentially during an Islamic cleansing ritual called Wudu. Skeleton modality is widely used in HAR because of having abstract information that is scale-invariant and robust against imagery variances. We used depth cameras to obtain accurate 3D human skeletons of users performing Wudu. The sequences were manually annotated with ten atomic action classes. This thesis investigated the use of different Deep Learning networks with architectures optimized for real-time action anticipation. The proposed methods were mainly based on the Spatial-Temporal Graph Convolutional Network. With further improvements, we proposed a Gated Recurrent Unit (GRU) model with Spatial-Temporal Graph Convolution Network (ST-GCN) backbone to extract local temporal features. The GRU process the local temporal latent features sequentially to predict future actions. The proposed models scored 94.14% recall on binary classification to turn on and off the water tap. And higher than 81.58-89.08% recall on multiclass classification.

No description available.