With the Internet of Things paradigm, the data generated by the rapidly increasing number of connected devices lead to new possibilities, such as using machine learning for activity recognition in smart environments. However, it also introduces several challenges. The sensors of different devices might be of different types, making the fusion of data non-trivial. Moreover, the devices are often mobile, resulting in that data from a particular sensor is not always available, i.e. there is a need to handle data from a dynamic set of sensors. From a machine learning perspective, the data from the sensors arrives in a streaming fashion, i.e., online learning, as compared to many learning problems where a static dataset is assumed. Machine learning is in many cases a good approach for classification problems, but the performance is often linked to the quality of the data. Having a good data set to train a model can be an issue in general, due to the often costly process of annotating the data. With dynamic and heterogeneous data, annotation can be even more problematic, because of the ever-changing environment. This means that there might not be any, or a very small amount of, annotated data to train the model on at the start of learning, often referred to as the cold start problem. To be able to handle these issues, adaptive systems are needed. With adaptive we mean that the model is not static over time, but is updated if there for instance is a change in the environment. By including human-in-the-loop during the learning process, which we refer to as interactive machine learning, the input from users can be utilized to build the model. The type of input used is typically annotations of the data, i.e. user input in the form of correctly labelled data points. Generally, it is assumed that the user always provides correct labels in accordance with the chosen interactive learning strategy. In many real-world applications these assumptions are not realistic however, as users might provide incorrect labels or not provide labels at all in line with the chosen strategy. In this thesis we explore which interactive learning strategies are possible in the given scenario and how they affect performance, as well as the effect of machine learning algorithms on performance. We also study how a user who is not always reliable, i.e. that does not always provide a correct label when expected to, can affect performance. We propose a taxonomy of interactive online machine learning strategies and test how the different strategies affect performance through experiments on multiple datasets. The findings show that the overall best performing interactive learning strategy is one where the user provides labels when previous estimations have been incorrect, but that the best performing machine learning algorithm depends on the problem scenario. The experiments also show that a decreased reliability of the user leads to decreased performance, especially when there is a limited amount of labelled data.

Large collections of historical document images have been collected by companies and government institutions for decades. More recently, these collections have been made available to a larger public via the Internet. However, to make accessing them truly useful, the contained images need to be made readable and searchable. One step in that direction is document image binarization, the separation of text foreground from page background. This separation makes the text shown in the document images easier to process by humans and other image processing algorithms alike. While reasonably well working binarization algorithms exist, it is not sufficient to just being able to perform the separation of foreground and background well. This separation also has to be achieved in an efficient manner, in terms of execution time, but also in terms of training data used by machine learning based methods. This is necessary to make binarization not only theoretically possible, but also practically viable. In this thesis, we explore different ways to achieve efficient binarization in terms of execution time by improving the implementation and the algorithm of a state-of-the-art binarization method. We find that parameter prediction, as well as mapping the algorithm onto the graphics processing unit (GPU) help to improve its execution performance. Furthermore, we propose a binarization algorithm based on recurrent neural networks and evaluate the choice of its design parameters with respect to their impact on execution time and binarization quality. Here, we identify a trade-off between binarization quality and execution performance based on the algorithm’s footprint size and show that dynamically weighted training loss tends to improve the binarization quality. Lastly, we address the problem of training data efficiency by evaluating the use of interactive machine learning for reducing the required amount of training data for our recurrent neural network based method. We show that user feedback can help to achieve better binarization quality with less training data and that visualized uncertainty helps to guide users to give more relevant feedback. / Scalable resource-efficient systems for big data analytics

<div>Real-time social media data can provide useful information on evolving events and situations. In addition, various domain users are increasingly leveraging real-time social media data to gain rapid situational awareness. Informed by discussions with first responders and government officials, we focus on two major barriers limiting the widespread adoption of social media for situational awareness: the lack of geotagged data and the deluge of irrelevant information during events. Geotags are naturally useful, as they indicate the location of origin and provide geographic context. Only a small portion of social media is geotagged, however, limiting its practical use for situational awareness. The deluge of irrelevant data provides equal difficulties, impeding the effective identification of semantically relevant information. Existing methods for short text relevance classification fail to incorporate users' knowledge into the classification process. Therefore, classifiers cannot be interactively retrained for specific events or user-dependent needs in real-time, limiting situational awareness. In this work, we first adapt, improve, and evaluate a state-of-the-art deep learning model for city-level geolocation prediction, and integrate it with a visual analytics system tailored for real-time situational awareness. We then present a novel interactive learning framework in which users rapidly identify relevant data by iteratively correcting the relevance classification of tweets in real-time. We integrate our framework with the extended Social Media Analytics and Reporting Toolkit (SMART) 2.0 system, allowing the use of our interactive learning framework within a visual analytics system adapted for real-time situational awareness.</div>

Every day, machine learning (ML) and artificial intelligence (AI) embeds itself further into domestic and industrial technologies. Interaction de- signers have historically struggled to engage directly with the subject, facing a shortage of appropriate methods and abstractions. There is a need to find ways though which interaction design practitioners might integrate ML into their work, in order to democratize and diversify the field. This thesis proposes a mode of inquiry that considers the inter- active qualities of what machine learning does, as opposed the tech- nical specifications of what machine learning is. A shift in focus from the technicality of ML to the artifacts it creates allows the interaction designer to situate its existing skill set, affording it to engage with ma- chine learning as a design material. A Research-through-Design pro- cess explores different methodological adaptions, evaluated through user feedback and an-in depth case analysis. An elaborated design experiment, Multiverse, examines the novel, non-anthropomorphic aesthetic qualities of generative literature. It prototypes interactions with bidirectional literature and studies how these transform the reader into a cybertextual “user-reader”. The thesis ends with a discussion on the implications of machine written literature and proposes a number of future investigations into the research space unfolded through the prototype.

[pt] A fisioterapia visa melhorar a funcionalidade física das pessoas, procurando atenuar as incapacidades causadas por alguma lesão, distúrbio ou doença. Nesse contexto, diversas tecnologias computacionais têm sido desenvolvidas com o intuito de apoiar o processo de reabilitação, como as tecnologias adaptáveis para o usuário final. Essas tecnologias possibilitam ao fisioterapeuta adequar aplicações e criarem atividades com características personalizadas de acordo com as preferências e necessidades de cada paciente. Nesta tese é proposta uma abordagem de baixo custo baseada no aprendizado de máquina interativo (iML - Interactive Machine Learning) que visa auxiliar os fisioterapeutas a criarem atividades personalizadas para seus pacientes de forma fácil e sem a necessidade de codificação de software, a partir de apenas alguns exemplos em vídeo RGB (capturadas por uma câmera de vídeo digital) Para tal, aproveitamos a estimativa de pose baseada em aprendizado profundo para rastrear, em tempo real, as articulações-chave do corpo humano a partir de dados da imagem. Esses dados são processados como séries temporais por meio do algoritmo Dynamic Time Warping em conjunto com com o algoritmo K-Nearest Neighbors para criar um modelo de aprendizado de máquina. Adicionalmente, usamos um algoritmo de detecção de anomalias com o intuito de avaliar automaticamente os movimentos. A arquitetura de nossa abordagem possui dois módulos: um para o fisioterapeuta apresentar exemplos personalizados a partir dos quais o sistema cria um modelo para reconhecer esses movimentos; outro para o paciente executar os movimentos personalizados enquanto o sistema avalia o paciente. Avaliamos a usabilidade de nosso sistema com fisioterapeutas de cinco clínicas de reabilitação. Além disso, especialistas avaliaram clinicamente nosso modelo de aprendizado de máquina. Os resultados indicam que a nossa abordagem contribui para avaliar automaticamente os movimentos dos pacientes sem monitoramento direto do fisioterapeuta, além de reduzir o tempo necessário do especialista para treinar um sistema adaptável. / [en] Physiotherapy aims to improve the physical functionality of people, seeking to mitigate the disabilities caused by any injury, disorder or disease. In this context, several computational technologies have been developed in order to support the rehabilitation process, such as the end-user adaptable technologies. These technologies allow the physiotherapist to adapt applications and create activities with personalized characteristics according to the preferences and needs of each patient. This thesis proposes a low-cost approach based on interactive machine learning (iML) that aims to help physiotherapists to create personalized activities for their patients easily and without the need for software coding, from just a few examples in RGB video (captured by a digital video camera). To this end, we take advantage of pose estimation based on deep learning to track, in real time, the key joints of the human body from image data. This data is processed as time series using the Dynamic Time Warping algorithm in conjunction with the K-Nearest Neighbors algorithm to create a machine learning model. Additionally, we use an anomaly detection algorithm in order to automatically assess movements. The architecture of our approach has two modules: one for the physiotherapist to present personalized examples from which the system creates a model to recognize these movements; another to the patient performs personalized movements while the system evaluates the patient. We assessed the usability of our system with physiotherapists from five rehabilitation clinics. In addition, experts have clinically evaluated our machine learning model. The results indicate that our approach contributes to automatically assessing patients movements without direct monitoring by the physiotherapist, in addition to reducing the specialist s time required to train an adaptable system.