abstract: Real-world environments are characterized by non-stationary and continuously evolving data. Learning a classification model on this data would require a framework that is able to adapt itself to newer circumstances. Under such circumstances, transfer learning has come to be a dependable methodology for improving classification performance with reduced training costs and without the need for explicit relearning from scratch. In this thesis, a novel instance transfer technique that adapts a "Cost-sensitive" variation of AdaBoost is presented. The method capitalizes on the theoretical and functional properties of AdaBoost to selectively reuse outdated training instances obtained from a "source" domain to effectively classify unseen instances occurring in a different, but related "target" domain. The algorithm is evaluated on real-world classification problems namely accelerometer based 3D gesture recognition, smart home activity recognition and text categorization. The performance on these datasets is analyzed and evaluated against popular boosting-based instance transfer techniques. In addition, supporting empirical studies, that investigate some of the less explored bottlenecks of boosting based instance transfer methods, are presented, to understand the suitability and effectiveness of this form of knowledge transfer. / Dissertation/Thesis / M.S. Computer Science 2011

abstract: In this thesis, we consider the problem of fast and efficient indexing techniques for time sequences which evolve on manifold-valued spaces. Using manifolds is a convenient way to work with complex features that often do not live in Euclidean spaces. However, computing standard notions of geodesic distance, mean etc. can get very involved due to the underlying non-linearity associated with the space. As a result a complex task such as manifold sequence matching would require very large number of computations making it hard to use in practice. We believe that one can device smart approximation algorithms for several classes of such problems which take into account the geometry of the manifold and maintain the favorable properties of the exact approach. This problem has several applications in areas of human activity discovery and recognition, where several features and representations are naturally studied in a non-Euclidean setting. We propose a novel solution to the problem of indexing manifold-valued sequences by proposing an intrinsic approach to map sequences to a symbolic representation. This is shown to enable the deployment of fast and accurate algorithms for activity recognition, motif discovery, and anomaly detection. Toward this end, we present generalizations of key concepts of piece-wise aggregation and symbolic approximation for the case of non-Euclidean manifolds. Experiments show that one can replace expensive geodesic computations with much faster symbolic computations with little loss of accuracy in activity recognition and discovery applications. The proposed methods are ideally suited for real-time systems and resource constrained scenarios. / Dissertation/Thesis / M.S. Electrical Engineering 2012

L'objectif de cette thèse est de concevoir une approche transparente originale apte à détecter en temps-réel l'occurrence d'une action, dans un flot non segmenté et idéalement le plus tôt possible. Ces travaux s'inscrivent dans une collaboration entre deux équipes de l'IRISA-lnria de Rennes, à savoir lntuidoc et MimeTIC. En profitant de la complémentarité des savoir-faire des deux équipes de recherche, nous proposons de reconsidérer les besoins et les difficultés rencontrées pour modéliser, reconnaître et détecter une action 30 en proposant de nouvelles solutions à la lumière des avancées réalisées en termes de modélisation de gestes manuscrits 20. Les contributions de cette thèse sont regroupées en trois parties principales. Dans la première partie, nous proposons une nouvelle approche pour modéliser et reconnaître une action pré­segmentée. Dans la deuxième partie, nous introduisons une approche permettant de reconnaître une action dans un flot non segmenté. Enfin, dans la troisième partie, nous étendons cette dernière approche pour la caractérisation précoce d'une action avec très peu de d'information. Pour chacune de ces trois problématiques, nous avons identifié explicitement les difficultés à considérer afin d'en effectuer une description complète pour permettre de concevoir des solutions ciblées pour chacune d'elles. Les résultats expérimentaux obtenus sur différents benchmarks d'actions attestent de la validité de notre démarche. En outre, à travers des coopérations ayant eu lieu au cours de la thèse, les approches développées ont été déployées dans trois applications, dont des applications en animation et en reconnaissance de gestes dynamiques de la main. / The objective of this thesis is to design an original transparent approach able to detect in real time the occurrence of an action, in an unsegmented flow and ideally as early as possible. This work is part of a collaboration between two IRISA-lnria teams in Rennes, namely lntuidoc and Mime TIC. By taking advantage of the complementary expertise of the two research teams, we propose to reconsider the needs and difficulties encountered to model, recognize and detect a 30 action by proposing new solutions in the light of the advances made in terms of 20 handwriting modeling. The contributions of this thesis are grouped into three main parts. In the first part, we propose a new approach to model and recognize a pre-segmented action. Indeed, it is first necessary to develop a representation able to characterize as finely as possible a given action to facilitate recognition. In the second part, we introduce an approach to recognize an action in an unsegmented flow. Finally, in the third part, we extend this last approach for the early characterization of an action with very little information. For each of these three issues, we have explicitly identified the difficulties to be considered in order to make a complete description of them so that we can design targeted solutions for each of them. The experimental results obtained on different benchmarks of actions attest to the validity of our approach. In addition, through collaborations that took place during the thesis, the developed approaches were deployed in three applications, including applications in animation and in dynamic hand gestures recognition.

O reconhecimento de atividaes é vital no contexto dos ambientes inteligentes. Mesmo com a facilidade de acesso a sensores móveis baratos, reconhecer atividades continua sendo um problema difícil devido à incerteza nas leituras dos sensores e à complexidade das atividades. A teoria da evidência provê um modelo de reconhecimento de atividades que detecta atividades mesmo na presença de incerteza nas leituras dos sensores, mas ainda não é capaz de modelar atividades complexas ou mudanças na configuração dos sensores ou do ambiente. Este trabalho propõe combinar abordagens baseadas em modelagem de conhecimento com a teoria da evidência, melhorando assim a construção dos modelos da última trazendo a reusabilidade, flexibilidade e semântica rica da primeira. / Activity recognition is a vital need in the field of ambient intelligence. It is essential for many internet of things applications including energy management, healthcare systems and home automation. But, even with the many cheap mobile sensors envisioned by the internet of things, activity recognition remains a hard problem. This is due to uncertainty in sensor readings and the complexity of activities themselves. Evidence theory models provide activity recognition even in the presence of uncertain sensor readings, but cannot yet model complex activities or dynamic changes in sensor and environment configurations. This work proposes combining knowledge-based approaches with evidence theory, improving the construction of evidence theory models for activity recognition by bringing reusability, flexibility and rich semantics.

abstract: The human motion is defined as an amalgamation of several physical traits such as bipedal locomotion, posture and manual dexterity, and mental expectation. In addition to the “positive” body form defined by these traits, casting light on the body produces a “negative” of the body: its shadow. We often interchangeably use with silhouettes in the place of shadow to emphasize indifference to interior features. In a manner of speaking, the shadow is an alter ego that imitates the individual. The principal value of shadow is its non-invasive behaviour of reflecting precisely the actions of the individual it is attached to. Nonetheless we can still think of the body’s shadow not as the body but its alter ego. Based on this premise, my thesis creates an experiential system that extracts the data related to the contour of your human shape and gives it a texture and life of its own, so as to emulate your movements and postures, and to be your extension. In technical terms, my thesis extracts abstraction from a pre-indexed database that could be generated from an offline data set or in real time to complement these actions of a user in front of a low-cost optical motion capture device like the Microsoft Kinect. This notion could be the system’s interpretation of the action which creates modularized art through the abstraction’s ‘similarity’ to the live action. Through my research, I have developed a stable system that tackles various connotations associated with shadows and the need to determine the ideal features that contribute to the relevance of the actions performed. The implication of Factor Oracle [3] pattern interpretation is tested with a feature bin of videos. The system also is flexible towards several methods of Nearest Neighbours searches and a machine learning module to derive the same output. The overall purpose is to establish this in real time and provide a constant feedback to the user. This can be expanded to handle larger dynamic data. In addition to estimating human actions, my thesis best tries to test various Nearest Neighbour search methods in real time depending upon the data stream. This provides a basis to understand varying parameters that complement human activity recognition and feature matching in real time. / Dissertation/Thesis / Masters Thesis Electrical Engineering 2016

O reconhecimento de atividaes é vital no contexto dos ambientes inteligentes. Mesmo com a facilidade de acesso a sensores móveis baratos, reconhecer atividades continua sendo um problema difícil devido à incerteza nas leituras dos sensores e à complexidade das atividades. A teoria da evidência provê um modelo de reconhecimento de atividades que detecta atividades mesmo na presença de incerteza nas leituras dos sensores, mas ainda não é capaz de modelar atividades complexas ou mudanças na configuração dos sensores ou do ambiente. Este trabalho propõe combinar abordagens baseadas em modelagem de conhecimento com a teoria da evidência, melhorando assim a construção dos modelos da última trazendo a reusabilidade, flexibilidade e semântica rica da primeira. / Activity recognition is a vital need in the field of ambient intelligence. It is essential for many internet of things applications including energy management, healthcare systems and home automation. But, even with the many cheap mobile sensors envisioned by the internet of things, activity recognition remains a hard problem. This is due to uncertainty in sensor readings and the complexity of activities themselves. Evidence theory models provide activity recognition even in the presence of uncertain sensor readings, but cannot yet model complex activities or dynamic changes in sensor and environment configurations. This work proposes combining knowledge-based approaches with evidence theory, improving the construction of evidence theory models for activity recognition by bringing reusability, flexibility and rich semantics.

O reconhecimento de atividaes é vital no contexto dos ambientes inteligentes. Mesmo com a facilidade de acesso a sensores móveis baratos, reconhecer atividades continua sendo um problema difícil devido à incerteza nas leituras dos sensores e à complexidade das atividades. A teoria da evidência provê um modelo de reconhecimento de atividades que detecta atividades mesmo na presença de incerteza nas leituras dos sensores, mas ainda não é capaz de modelar atividades complexas ou mudanças na configuração dos sensores ou do ambiente. Este trabalho propõe combinar abordagens baseadas em modelagem de conhecimento com a teoria da evidência, melhorando assim a construção dos modelos da última trazendo a reusabilidade, flexibilidade e semântica rica da primeira. / Activity recognition is a vital need in the field of ambient intelligence. It is essential for many internet of things applications including energy management, healthcare systems and home automation. But, even with the many cheap mobile sensors envisioned by the internet of things, activity recognition remains a hard problem. This is due to uncertainty in sensor readings and the complexity of activities themselves. Evidence theory models provide activity recognition even in the presence of uncertain sensor readings, but cannot yet model complex activities or dynamic changes in sensor and environment configurations. This work proposes combining knowledge-based approaches with evidence theory, improving the construction of evidence theory models for activity recognition by bringing reusability, flexibility and rich semantics.

Abstract Inertial sensors are devices that measure movement, and therefore, when they are attached to a body, they can be used to measure human movements. In this thesis, data from these sensors are studied to recognize human activities user-independently. This is possible if the following two hypotheses are valid: firstly, as human movements are dissimilar between activities, also inertial sensor data between activities is so different that this data can be used to recognize activities. Secondly, while movements and inertial data are dissimilar between activities, they are so similar when different persons are performing the same activity that they can be recognized as the same activity. In this thesis, pattern recognition -based solutions are applied to inertial data to find these dissimilarities and similarities, and therefore, to build models to recognize activities user-independently. Activity recognition within this thesis is studied in two contexts: daily activity recognition using mobile phones, and activity recognition in industrial context. Both of these contexts have special requirements and these are considered in the presented solutions. Mobile phones are optimal devices to measure daily activity: they include a wide range of useful sensors to detect activities, and people carry them with them most of the time. On the other hand, the usage of mobile phones in active recognition includes several challenges; for instance, a person can carry a phone in any orientation, and there are hundreds of smartphone models, and each of them have specific hardware and software. Moreover, as battery life is always as issue with smartphones, techniques to lighten the classification process are proposed. Industrial context is different from daily activity context: when daily activities are recognized, occasional misclassifications may disturb the user, but they do not cause any other type of harm. This is not the case when activities are recognized in industrial context and the purpose is to recognize if the assembly line worker has performed tasks correctly. In this case, false classifications may be much more harmful. Solutions to these challenges are presented in this thesis. The solutions introduced in this thesis are applied to activity recognition data sets. However, as the basic idea of the activity recognition problem is the same as in many other pattern recognition procedures, most of the solutions can be applied to any pattern recognition problem, especially to ones where time series data is studied. / Tiivistelmä Liikettä mittaavista antureista, kuten kiihtyvyysantureista, saatavaa tietoa voidaan käyttää ihmisten liikkeiden mittaamiseen kiinnittämällä ne johonkin kohtaan ihmisen kehoa. Väitöskirjassani tavoitteena on opettaa tähän tietoon perustuvia käyttäjäriippumattomia malleja, joiden avulla voidaan tunnistaa ihmisten toimia, kuten käveleminen ja juokseminen. Näiden mallien toimivuus perustuu seuraavaan kahteen oletukseen: (1) koska henkilöiden liikkeet eri toimissa ovat erilaisia, myös niistä mitattava anturitieto on erilaista, (2) useamman henkilön liikkeet samassa toimessa ovat niin samanlaisia, että liikkeistä mitatun anturitiedon perusteella nämä liikkeet voidaan päätellä kuvaavan samaa toimea. Tässä väitöskirjassa käyttäjäriippumaton ihmisten toimien tunnistus perustuu hahmontunnistusmenetelmiin ja tunnistusta on sovellettu kahteen eri asiayhteyteen: arkitoimien tunnistamiseen älypuhelimella sekä toimintojen tunnistamiseen teollisessa ympäristössä. Molemmilla sovellusalueilla on omat erityisvaatimuksensa ja -haasteensa. Älypuhelimien liikettä mittaavien antureihin perustuva tunnistus on haastavaa esimerkiksi siksi, että puhelimen asento ja paikka voivat vaihdella. Se voi olla esimerkiksi laukussa tai taskussa, lisäksi se voi olla missä tahansa asennossa. Myös puhelimen akun rajallinen kesto luo omat haasteensa. Tämän vuoksi tunnistus tulisi tehdä mahdollisimman kevyesti ja vähän virtaa kuluttavalla tavalla. Teollisessa ympäristössä haasteet ovat toisenlaisia. Kun tarkoituksena on tunnistaa esimerkiksi työvaiheiden oikea suoritusjärjestys kokoamislinjastolla, yksikin virheellinen tunnistus voi aiheuttaa suuren vahingon. Teollisessa ympäristössä tavoitteena onkin tunnistaa toimet mahdollisimman tarkasti välittämättä siitä kuinka paljon virtaa ja tehoa tunnistus vaatii. Väitöskirjassani kerrotaan kuinka nämä erityisvaatimukset ja -haasteet voidaan ottaa huomioon suunniteltaessa malleja ihmisten toimien tunnistamiseen. Väitöskirjassani esiteltyjä uusia menetelmiä on sovellettu ihmisten toimien tunnistamiseen. Samoja menetelmiä voidaan kuitenkin käyttää monissa muissa hahmontunnistukseen liittyvissä ongelmissa, erityisesti sellaisissa, joissa analysoitava tieto on aikasarjamuotoista.

Abstract The way how people interact with machines will change in the future. Long have been the traditional ways – mouse and keyboard – the primary interface between man and computer. Recently, the voice and gesture controlled interfaces have been introduced in many devices but they have not yet become very popular. One possible direction where human-computer interfaces can go is to be able to completely hide the interface from the user and allow him or her to interact with the machines in a way that is more natural to human. This thesis introduces a smart living space concept that is a small step towards that direction. The interfacing is assumed to be done unnoticeably to the user via a wireless sensor network that is monitoring the user and analysing his or her behaviour and also using a hand held mobile device which can be used to control the system. A system for human body part segmentation is presented. The system is applied in various applications related to person identification from one’s gait and unusual activity detection. The system is designed to work robustly when the data streams provided by the sensor network are noisy. This increases the usefulness of the system in home environments where the person using the interface is either occluded by the static objects in the room or is interacting with any movable objects. The second part of the proposed smart living space concept is the mobile device carried by the user. Two methods that can be used in a hand gesture-based UI are proposed. A database for training such methods is proposed. / Tiivistelmä Tapa jolla ihmiset käyttävät tietokonetta on muuttumassa. Hiiri ja näppäimistö ovat olleet jo pitkään yleisimmät tavat, joilla tietokoneita on ohjattu. Uusia tapoja ohjata tietokonetta on kehitetty, mutta ne eivät ole vielä syrjäyttäneet perinteisiä menetelmiä täysin. Yksi todennäköinen muutos tulevaisuudessa on se, että käyttöliittymät sulautetaan ympäristöön ja sen myötä tehdään käyttökokemuksesta luonnollisempi ihmiselle. Tässä väitöskirjassa esitellään järjestelmä, joka muuttaa ihmisen elinympäristön älykkääksi. Langaton kameraverkko analysoi automaattisesti huoneen tapahtumia ja käyttäjä kontrolloi järjestelmää eleohjatulla mobiililaitteella. Väitöskirjassa esitellään menetelmä ihmisen ruumiinosien tunnistukseen, jota sovelletaan myös ihmisen tunnistukseen kävelytyylistä ja epänormaalien aktiviteettien tunnistukseen. Menetelmää suunnitellessa on painotettu sitä, että se toimisi myös silloin, kun käytettävissä on vain huonolaatuista ja kohinaista videodataa. Kohinaa aiheuttaa kotiympäristöissä erityisesti huonekalut, jotka osittain peittävät näkymää ja tavarat, joita huoneessa oleskeleva ihminen saattaa siirrellä. Toinen osa väitöskirjaa käsittelee mobiililaitteen ohjausta käsielein ja esittelee kaksi menetelmää, joilla tällainen käyttöliittymä on mahdollista toteuttaa. Toisen menetelmän opetuksessa käytetty käsitietokanta ja tietokannan vertailutulokset julkaistaan.

Human-machine interaction could support many daily activities in making it more convenient. The development of smart devices has flourished the underlying smart systems that process smart and personalized control of devices. The first step in controlling any device is observation; through understanding the surrounding environment and human activity, a smart system can physically control a device. Human activity recognition (HAR) is essential in many smart applications such as self-driving cars, human-robot interaction, and automatic systems such as infrared (IR) taps. For human-centric systems, there are some requirements to perform a physical task in real-time. For human-machine interactions, the anticipation of human actions is essential. IR taps have delay limitations because of the proximity sensor that signals the solenoid valve only when the user’s hands are exactly below the tap. The hardware and electronics delay causes inconvenience in use and water waste. In this thesis, an alternative control based on deep learning action anticipation is proposed. Humans interact with taps for various tasks such as washing hands, face, brushing teeth, just to name a few. We focus on a small subset of these activities. Specifically, we focus on the activities carried out sequentially during an Islamic cleansing ritual called Wudu. Skeleton modality is widely used in HAR because of having abstract information that is scale-invariant and robust against imagery variances. We used depth cameras to obtain accurate 3D human skeletons of users performing Wudu. The sequences were manually annotated with ten atomic action classes. This thesis investigated the use of different Deep Learning networks with architectures optimized for real-time action anticipation. The proposed methods were mainly based on the Spatial-Temporal Graph Convolutional Network. With further improvements, we proposed a Gated Recurrent Unit (GRU) model with Spatial-Temporal Graph Convolution Network (ST-GCN) backbone to extract local temporal features. The GRU process the local temporal latent features sequentially to predict future actions. The proposed models scored 94.14% recall on binary classification to turn on and off the water tap. And higher than 81.58-89.08% recall on multiclass classification.