Technological advancements in sensor miniaturization, processing power and faster networks has broadened the scope of our contemporary compute-infrastructure to an extent that Context-Aware Intelligent Environment (CAIE)--physical spaces with computing systems embedded in it--are increasingly commonplace. With the widespread adoption of intelligent personal agents proliferating as close to us as our living rooms, there is a need to rethink the human-computer interface to accommodate some of their inherent properties such as multiple focus of interaction with a dynamic set of devices and limitations such as lack of a continuous coherent medium of interaction. A CAIE provides context-aware services to aid in achieving user's goals by inferring their instantaneous context. However, often due to lack of complete understanding of a user's context and goals, these services may be inappropriate or at times even pose hindrance in achieving user's goals. Determining service appropriateness is a critical step in implementing a reliable and robust CAIE. Explicitly querying the user to gather such feedback comes at the cost of user's cognitive resources in addition to defeating the purpose of designing a CAIE to provide automated services. The CAIE may, however, infer this appropriateness implicitly from the user, by observing and sensing various behavioral cues and affective reactions from the user, thereby seamlessly gathering such user-feedback. In this dissertation, we have studied the design space for incorporating user's affective reactions to the intelligent services, as a mode of implicit communication between the user and the CAIE. As a result, we have introduced a framework named CAfFEINE, acronym for Context-aware Affective Feedback in Engineering Intelligent Naturalistic Environments. The CAfFEINE framework encompasses models, methods and algorithms establishing the validity of the idea of using a physiological-signal based affective feedback loop in conveying service appropriateness in a CAIE. In doing so, we have identified methods of learning ground-truth about an individual user's affective reactions as well as introducing a novel algorithm of estimating a physiological signal based quality-metric for our inferences. To evaluate the models and methods presented in the CAfFEINE framework, we have designed a set of experiments in laboratory-mockups and virtual-reality setup, providing context aware services to the users, while collecting their physiological signals from wearable sensors. Our results provide empirical validation for our CAfFEINE framework, as well as point towards certain guidelines for conducting future research extending this novel idea. Overall, this dissertation contributes by highlighting the symbiotic nature of the subfields of Affective Computing and Context-aware Computing and by identifying models, proposing methods and designing algorithms that may help accentuate this relationship making future intelligent environments more human-centric. / Ph. D. / Physical spaces containing intelligent computing agents have become an increasingly commonplace concept. These systems when populating a physical space, provides intelligent services by inferring user’s immediate needs, they are called intelligent environments. With this widespread adoption of intelligent systems, there is a need to design computer interfaces that focuses on the human user’s responses. In order for this service-delivery interaction to feel natural, these interfaces need to sense a user’s disapproval of a wrong service, without the user actively indicating so. It is imperative that implicitly inferring a user’s disapproval of a service by observing and sensing various behavioral cues from the user, will help in making the computing system cognitively disappear into the background. In this dissertation, we have studied the design space for incorporating user’s affective reactions to the intelligent services, as a mode of implicit communication between the user and the intelligent system. As a result, we have introduced an interaction framework named CAfFEINE, acronym for Context-aware Affective Feedback in Engineering Intelligent Naturalistic Environments. The CAfFEINE framework encompasses models, methods and algorithms exploring the validity of the idea of using physiological signal based affective feedback in intelligent environments. To evaluate the models and algorithms, we have designed a set of experimental protocols and conducted user studies in virtual-reality setup. The results from these user studies demonstrate the feasibility of this novel idea, in addition to proposing new methods of evaluating the quality of underlying physiological signals. Overall, this dissertation contributes by highlighting the symbiotic nature of the subfields of Affective Computing and Context-aware Computing and by identifying models, proposing methods and designing algorithms that may help accentuate this relationship making future intelligent environments more human-centric.

This dissertation presents novel approaches to test context aware applications that suffer from a cost prohibitive number of context and GUI events and event combinations. The contributions of this work to test context aware applications under test include: (1) a real-world context events dataset from 82 Android users over a 30-day period, (2) applications of Markov models, Closed Sequential Pattern Mining (CloSPAN), Deep Neural Networks- Long Short Term Memory (LSTM) and Gated Recurrent Units (GRU), and Conditional Random Fields (CRF) applied to predict context patterns, (3) data driven test case generation techniques that insert events at the beginning of each test case in a round-robin manner, iterate through multiple context events at the beginning of each test case in a round-robin manner, and interleave real-world context event sequences and GUI events, and (4) systematically interleaving context with a combinatorial-based approach. The results of our empirical studies indicate (1) CRF outperforms other models thereby predicting context events with F1 score of about 60% for our dataset, (2) the ISFreqOne that iterates over context events at the beginning of each test case in a round-robin manner as well as interleaves real-world context event sequences and GUI events at an interval one achieves up to four times better code coverage than not including context, 0.06 times better coverage than RSContext that inserts random context events at the beginning of each test case, 0.05 times better coverage than ISContext that iterates over context events to insert at the beginning of each test case in a round-robin manner, and 0.04 times better coverage than ISFreqTwo that iterates over context events at the beginning of each test case in a round-robin manner as well as interleaves real-world context event sequences and GUI events at an interval two on an average across four subject applications and, (3) the PairwiseInterleaved technique that selects a different context event at the beginning of each test case by iterating through context covering array in a round-robin manner and systematically interleaves context with GUI events by prioritizing the execution of GUI events in new contexts achieves higher code coverage up to a factor of six when compared to Monkey, up to a factor of 1.3 when compared to a technique that generates test suites without context events, and similar code coverage when compared to ISContext that iterates over context events to insert at the beginning of each test case in a round-robin manner on an average across five subject applications.

Σε αυτή τη διδακτορική διατριβή μελετήσαμε τις δυνατότητες αξιοποίησης των οντολογιών στην αναπαράσταση γνώσης σε συστήματα απανταχού και κινητού υπολογίζειν. / In this thesis we studied the use of ontologies for knowledge representation in ubiquitous and mobile computing.

Aujourd'hui, les environnements ubiquitaires font partie de notre vie quotidienne. À la maison, au travail, dans les voitures, dans les hôtels, les supermarchés et autres espaces publiques, nous rencontrons des technologies qui visent à rendre notre vie plus simple et plus facile d'une façon transparente. Durant ces dernières années, le potentiel de ces environnements a été de plus en plus exploité, notamment avec l'avènement et l'utilisation généralisée des smartphones. Ce type de dispositifs permet l'exécution d'applications qui ont la capacité de s'adapter parfaitement à l'état courant de l'environnement. De telles applications, appelées "applications sensibles au contexte", bénéficient de l'information du contexte et des services qui sont présents dans leur environnement pour améliorer et changer automatiquement leur comportement. Toutefois, ces adaptations nécessitent une intégration d'informations qui doit être effectuée en tenant compte de l'hétérogénéité en termes de dispositifs, de plateformes d'exécution, et de protocoles de communication ainsi que la mobilité des applications de sorte que les différentes responsabilités de l'adaptation peuvent être distribuées. Pour faire face à ces défis, et compte tenu des limitations des solutions existantes, nous fournissons deux contributions majeures dans cette dissertation. Tout d'abord nous introduisons l'intergiciel SPACES comme une solution d'intégration des informations contextuelles et ensuite nous définissons le paradigme de "boucles de contrôle ubiquitaires" pour adapter les applications sensibles au contexte. En particulier, dans SPACES, nous définissons un méta-modèle inspiré du style architectural REST (REpresentational State Transfer) pour favoriser l'échange des informations contextuelles en tant que ressources, ce qui représente le fondement de notre approche. Ensuite, nous définissons les connecteurs SPACES pour modulariser les différents concepts et préoccupations identifiés par le méta-modèle. Ces connecteurs sont conçus en utilisant les principes de la programmation orientée composant et ils sont incorporés dans le modèle Service Component Architecture (SCA) pour être utilisés dans différents types d'applications, et ainsi indépendamment des applications sensibles au contexte. Grâce à la définition de SPACES, nous sommes en mesure d'élaborer la seconde contribution de la dissertation--i.e., les boucles de contrôle ubiquitaires. Inspiré par les concepts de l'informatique autonome, les boucles de contrôle offre la flexibilité nécessaire pour intégrer de nouveaux participants dans le processus d'adaptation (par exemple, des applications sensibles au contexte, des services et des systèmes existants) tout en fournissant un support pour la mobilité et l'intégration de nouveaux mécanismes de communication en cas de besoin. Dans le noyau des boucles de contrôle ubiquitaires--i.e., la prise de décision-- nous employons des techniques de programmation par contraintes pour optimiser la configuration courante de l'application en intégrant des critères qui garantissent une meilleure expérience à l'utilisateur final, tels que les coûts associés à l'adaptation, les ressources consommées ou encore la qualité de service offerte par la nouvelle configuration. Enfin, nous validons notre proposition avec trois études de cas: Tout d'abord une politique de Caching or Off-Loading, dans laquelle le comportement de l'application est modifiée lors de l'exécution, ensuite TRACK.ME, une plateforme pour effectuer des expérimentations scientifiques et enfin DIGIHOME, une plateforme pour la création des maisons intelligentes. Ces scénarios démontrent la pertinence de notre approche lorsque différents types de dispositifs, des protocoles et des technologies de mise en œuvre sont impliqués dans le processus d'adaptation.

The thesis examines how current mobile operating systems support context-aware applications and investigates the methods of mobile application debugging. The thesis points out what kind of problems need to be solved during debugging of context-aware applications. The primary goal of the thesis is to propose a debugging method which takes context information into account and to implement this method. The thesis contains a real world use case to demonstrate the proposed method.

En informatique ubiquitaire, l'observation du monde physique et de son "contexte" (une représentation haut niveau de la situation physique) est essentielle. Il existe de nombreux moyens pour observer le contexte. Typiquement, cela consiste en un traitement en plusieurs étapes commençant par la récupération de données brutes issues de capteurs. Diverses technologies de capteurs sont utilisées pour la récupération d'informations de bas niveau sur les activités physiques en cours. Ces données sont ensuite rassemblées, analysées et traitées ailleurs dans les systèmes d'information afin d'offrir une reconnaissance de contexte. Les applications déployées réagissent alors en fonction du contexte/de la situation détecté(e). Parmis les capteurs utilisés, les tags RFID, une technologie émergente, permettent de créer un lien virtuel direct entre les objets physiques et les systèmes d'information. En plus de stocker des identifiants, ils offrent un espace mémoire générique aux objets auxquels ils sont attachés, offrant de nouvelles possibilités d'architectures en informatique omniprésente. Dans cette thèse, nous proposons une approche originale tirant parti de l'espace mémoire offerts aux objets réels par les tags RFID. Dans notre approche, les objets supportent directement le système d'information. Ce type d'intégration permet de réduire les communications requises par le traitement à distance. Pour ce faire, des données sémantiques sont tout d'abord attachées aux objets afin de les rendre auto-descriptifs. Ainsi, les données pertinentes concernant une entité physique sont directement disponibles pour un traitement local. Les objets peuvent ensuite être liés virtuellement grâce à des structures de données dédiées ou ad hoc et distribuées sur les objets eux-mêmes. Ce faisant, le traitement des données peut se faire de façon directe. Par exemple, certaines propriétés peuvent être vérifiées localement sur un ensemble d'objets. Une relation physique peut être déduite directement de la structure de données, d'où le nom de "structures de données tangibles". Vis-à-vis des approches conventionnelles tirant parti des identifiants, notre approche offrent des avantages en termes de vie privée, de mise à l'échelle, d'autonomie et d'indépendance vis-à-vis des infrastructures. Le défi se situe au niveau de son expressivité limitée à cause du faible espace mémoire disponible sur les tags RFID. Les principes sont validés dans deux prototypes aux applications différentes. Le premier prototype est développé dans le domaine de la gestion de déchets afin d'aider le tri et d'améliorer le recyclage. Le deuxième offre des services supplémentaires, tels qu'une assistance lors du montage et de la vérification d'objets composés de plusieurs parties, grâce aux structures de données distribuées sur les différentes parties. / Pervasive computing or ambient computing aims to integrate information systems into the environment, in a manner as transparent as possible to the users. It allows the information systems to be tightly coupled with the physical activities within the environment. Everyday used objects, along with their environment, are made smarter with the use of embedded computing, sensors etc. and also have the ability to communicate among themselves. In pervasive computing, it is necessary to sense the real physical world and to perceive its “context” ; a high level representation of the physical situation. There are various ways to derive the context. Typically, the approach is a multi-step process which begins with sensing. Various sensing technologies are used to capture low level information of the physical activities, which are then aggregated, analyzed and computed elsewhere in the information systems, to become aware of the context. Deployed applications then react, depending on the context situation. Among sensors, RFID is an important emerging technology which allows a direct digital link between information systems and physical objects. Besides storing identification data, RFID also provides a general purpose storage space on objects, enabling new architectures for pervasive computing. In this thesis, we defend an original approach adopting the later use of RFID i.e. a digital memory integrated to real objects. The approach uses the principle where the objects self-support information systems. This way of integration reduces the need of communication for remote processing. The principle is realized in two ways. First, objects are piggybacked with semantic information, related to itself ; as self-describing objects. Hence, relevant information associated with the physical entities are readily available locally for processing. Second, group of related objects are digitally linked using dedicated or ad-hoc data structure, distributed over the objects. Hence, it would allow direct data processing - like validating some property involving the objects in proximity. This property of physical relation among objects can be interpreted digitally from the data structure ; this justifies the appellation “Tangible Data Structures”. Unlike the conventional method of using identifiers, our approach has arguments on its benefits in terms of privacy, scalability, autonomy and reduced dependency with respect to infrastructure. But its challenge lies in the expressivity due to limited memory space available in the tags. The principles are validated by prototyping in two different application domains. The first application is developed for waste management domain that helps in efficient sorting and better recycling. And the second, provides added services like assistance while assembling and verification for composite objects, using the distributed data structure across the individual pieces.

Research that involves human-beings depends on the data collection. As technology solutions become popular in the context of healthcare, researchers highlight the need for monitoring and caring patients in situ. Human Activity Recognition (HAR) is a research field that combines two areas: Ubiquitous Computing and Artificial Intelligence. HAR is daily applied in several service sectors including military, security (surveillance), health and entertainment. A HAR system aims to identify and recognize the activities and actions a user performs, in real time or not. Ambient sensors (e.g. cameras) and wearable devices (e.g. smartwatches) collect information about users and their context (e.g. localization, time, companions). This data is processed by machine learning algorithms that extract information and classify the corresponding activity. Although there are several works in the literature related to HAR systems, most studies focusing on elderly users are limited and do not use, as ground truth, data collected from elder volunteers. Databases and sensors reported in the literature are geared towards a generic audience, which leads to loss in accuracy and robustness when targeted at a specific audience. Considering this gap, this work presents a Human Activity Recognition system and corresponding database focusing on the elderly, raising requirements and guidelines for supportive HAR system and the selection of sensor devices. The system evaluation was carried out checking the accuracy of the activity recognition process, defining the best statistical features and classification algorithms for the Elderly Activity Recognition System (EARS). The results suggest that EARS is a promising supportive technology for the elderly, having an accuracy of 98.37% with KNN (k = 1). / Pesquisas e serviços no campo da saúde se valem da coleta, em tempo real ou não, de dados de ordem física, psicológica, sentimental, comportamental, entre outras, de pacientes ou participantes em experimentos: o objetivo é melhorar tratamentos e procedimentos. As soluções tecnológicas estão se tornando populares no contexto da saúde, pesquisadores da área de saúde destacam a necessidade de monitoramento e cuidado dos pacientes in situ. O campo de pesquisa de Reconhecimento de Atividade Humana (sigla em inglês HAR, Human Activity Recognition) envolve as áreas de computação ubíqua e de inteligência artificial, sendo aplicado nos mais diversos domínios. Com o uso de sensores como câmeras, microfones e acelerômetros, entre outros, um sistema HAR tem por tarefa identificar as atividades que uma pessoa realiza em um determinado momento. As informações coletadas pelos sensores e os dados do usuário são processados por algoritmos de aprendizado de máquina para identificar a atividade humana realizada. Apesar de existirem vários trabalhos na literatura de sistemas HAR, poucos são voltados para o público ancião. Bases de dados e sensores reportados em trabalhos relacionados são voltadas para um público genérico, perdendo precisão e robustez quando se trata de um público específico. Diante dessa lacuna, este trabalho propõe um sistema de Reconhecimento de Atividade Humana voltado para o idoso, levantando requisitos para o sistema HAR assistido e selecionando os dispositivos sensores. Um banco de dados HAR com dados coletados de voluntários mais velhos também é fornecido e disponibilizado. A avaliação do sistema foi realizada verificando a acurácia do processo de reconhecimento da atividade, definindo as melhores características estatísticas e algoritmos de classificação para o sistema de reconhecimento de atividades do idoso. Os resultados sugerem que esse sistema é uma tecnologia de suporte promissora para idosos, tendo uma acurácia de 98.37% com KNN (k = 1).

O trabalho apresentado nesta dissertação trata do desenvolvimento de um mecanismo para adaptação automática de ví?deo MPEG-4 ao vivo, de modo a atender as necessidades ou capacidades atuais de usuários e do sistema. Um dos desafios dessa área é capturar e representar as informações necessárias para realizar a adaptação. Assim, utilizando técnicas da área de computação ciente de contexto, foi desenvolvido um modelo extensível para representação de dispositivos. Também foram desenvolvidos métodos automáticos e semi-automáticos para capturar as informações necessárias. Neste trabalho foi adotado o modelo de recodificação de vídeo, o qual pode gerar atrasos que inviabilizam a adaptação de vídeo ao vivo em aplicações interativas. Assim, este trabalho realizou uma avaliação do impacto causado pela recodificação no atraso total, fim-a-fim, percebido pelo usuário. / This work presents the development of a mechanism to automatically adapt MPEG-4 live video, in a way to response the actual necessities or capacities of users or systems. One of the challanges in this area is to capture and represent the information needed to adapting content. Thus, using context aware computing techniques, an extensible model has been developed, which can be used to represent devices. It has also been developed automatic and semi-automatic methods to capture the needed information. In this work, the transcoding model has been adopted, which may generate latency, making difficult to use transcoding with interactive applications. In this way, this work has evaluated the impact caused by the transcoding when compared to the end-to-end total delay perceived by the user.

No description available.

A população idosa mundial está crescendo e, com ela, o número de diagnósticos de doenças relacionadas à velhice como, por exemplo, declínios cognitivos também. Essas doenças costumam afetar a autonomia do idoso no seu lar, especialmente no que se refere à realização de atividades diárias. Com isso em vista, é preciso empregar cuidadores e serviços de saúde que acabam por implicar em altos custos. Nesse sentido, surge a necessidade de conceber sistemas robustos, automatizados, utilizáveis e de baixo custo para a assistência pessoal. A concepção desses sistemas faz menção à área de Ambientes de Vivência Assistida. Portanto, esta dissertação propõe uma abordagem que beneficia os sistemas para Ambientes de Vivência Assistida com a capacidade de prever ações humanas para a facilitação de atividades diárias, particularmente quando declínios cognitivos relacionados à elas ocorrerem. Nesse sentido, foi concebido um meta-modelo semântico para a geração de modelos conceituais de contexto e de comportamento, compostos pelas ações humanas. A partir disso, a previsão de ações (informação de suporte) é realizada por um mecanismo de predição e inferência composto por um modelo semântico probabilístico. A abordagem é demonstrada através de um estudo de caso cujo cenário representa uma situação de declínio cognitivo, enfrentada por um usuário, que impede a condução de uma atividade diária. Então, o mecanismo de predição e inferência, utilizando o modelo semântico probabilístico, prevê qual a ação mais adequada que facilite a conclusão da atividade. Essa previsão é avaliada para aferir o quão bem um usuário seria auxiliado, isto é, se a operação prevista foi por ele realizada. Para isso, foi utilizado um dataset relacionado ao cenário do estudo de caso e medidas de desempenho como a precisão, a revocação e a medida-F. Os resultados dessa avaliação se mostraram promissores sendo, em média, 69,5% para a precisão, 100% para a revocação e 81% para a medida-F. As principais contribuições deste trabalho dizem respeito ao meta-modelo semântico a partir do qual pesquisas na área deste trabalho podem utilizar para gerar modelos de comportamento, e ao modelo semântico probabilístico que realiza predição através de raciocínio incerto sobre os modelos de comportamento, propiciando decisões mais precisas para auxiliar usuários com declínio cognitivo. / The world’s elderly population is growing and, with it, the number of diagnoses of diseases related to old age, such as cognitive declines as well. These diseases usually affect the autonomy of the elderly in their home, especially when it comes to performing daily activities. With this in mind, it is necessary to employ caregivers and health services that end up implying high costs. In this sense, the need arises to design robust, automated, usable and low-cost systems for personal assistance. The design of these systems makes reference to the area of Ambient Assisted Living. Therefore, this dissertation proposes an approach that benefits the Ambient Assisted Living systems with the ability to predict human actions for the facilitation of daily activities, particularly when cognitive declines related to them occur. In this sense, a semantic meta-model was conceived for the generation of conceptual models of context and behavior, composed by human actions. From this, the prediction of actions (information of support) is realized by a mechanism of prediction and inference composed by a probabilistic semantic model. The approach is demonstrated through a case study whose scenario represents a situation of cognitive decline, faced by a user, that prevents the conduct of a daily activity. Then, the prediction and inference mechanism, using the probabilistic semantic model, predicts the most appropriate action that facilitates the conclusion of the activity. This forecast is evaluated to gauge how well a user would be assisted, that is, if the intended operation was performed by him. For this, a dataset related to the case study scenario and performance measures such as precision, recall, and F-measure were used. The results of this evaluation are promising, averaging 69.5% for precision, 100% for recall and 81% for F-measure. The main contributions of this work are related to the semantic meta-model from which research in the area of this work can be used to generate behavioral models, and to the probabilistic semantic model that performs prediction through uncertain reasoning over behavior models, providing better decisions to help users with cognitive decline.