Interactive machine learning (IML) is a promising approach for personalising physiotherapy training. This thesis uses a research- through-design and reflective approach to explore how IML can be ecologically integrated within in-clinic physiotherapy. Domain expert interviews and observations with physiotherapists were conducted to gain a broader understanding of the physiotherapy context, the role of feedback provided to patients, and how technology could be integrated into this context. Three design artefacts were proposed to participants to provoke a discussion on the implications and current practices. Due to time constraints, the findings suggest incorporation within the consultation may be difficult. The clinic’s gym or administration time revealed to be promising alternatives. Furthermore, results highlight the importance of IML supporting richer interaction forms and the implicitness and flexibility needed to describe movement and feedback, which define physiotherapy’s hands-on approach. Togetherness was a reoccurring theme, suggesting that the input and guidance of the IML system could be something done with the patient. Finally, a reflection on the results and the study opens up a discussion of the fitness of IML in physiotherapy.

With the Internet of Things paradigm, the data generated by the rapidly increasing number of connected devices lead to new possibilities, such as using machine learning for activity recognition in smart environments. However, it also introduces several challenges. The sensors of different devices might be of different types, making the fusion of data non-trivial. Moreover, the devices are often mobile, resulting in that data from a particular sensor is not always available, i.e. there is a need to handle data from a dynamic set of sensors. From a machine learning perspective, the data from the sensors arrives in a streaming fashion, i.e., online learning, as compared to many learning problems where a static dataset is assumed. Machine learning is in many cases a good approach for classification problems, but the performance is often linked to the quality of the data. Having a good data set to train a model can be an issue in general, due to the often costly process of annotating the data. With dynamic and heterogeneous data, annotation can be even more problematic, because of the ever-changing environment. This means that there might not be any, or a very small amount of, annotated data to train the model on at the start of learning, often referred to as the cold start problem. To be able to handle these issues, adaptive systems are needed. With adaptive we mean that the model is not static over time, but is updated if there for instance is a change in the environment. By including human-in-the-loop during the learning process, which we refer to as interactive machine learning, the input from users can be utilized to build the model. The type of input used is typically annotations of the data, i.e. user input in the form of correctly labelled data points. Generally, it is assumed that the user always provides correct labels in accordance with the chosen interactive learning strategy. In many real-world applications these assumptions are not realistic however, as users might provide incorrect labels or not provide labels at all in line with the chosen strategy. In this thesis we explore which interactive learning strategies are possible in the given scenario and how they affect performance, as well as the effect of machine learning algorithms on performance. We also study how a user who is not always reliable, i.e. that does not always provide a correct label when expected to, can affect performance. We propose a taxonomy of interactive online machine learning strategies and test how the different strategies affect performance through experiments on multiple datasets. The findings show that the overall best performing interactive learning strategy is one where the user provides labels when previous estimations have been incorrect, but that the best performing machine learning algorithm depends on the problem scenario. The experiments also show that a decreased reliability of the user leads to decreased performance, especially when there is a limited amount of labelled data.

Large collections of historical document images have been collected by companies and government institutions for decades. More recently, these collections have been made available to a larger public via the Internet. However, to make accessing them truly useful, the contained images need to be made readable and searchable. One step in that direction is document image binarization, the separation of text foreground from page background. This separation makes the text shown in the document images easier to process by humans and other image processing algorithms alike. While reasonably well working binarization algorithms exist, it is not sufficient to just being able to perform the separation of foreground and background well. This separation also has to be achieved in an efficient manner, in terms of execution time, but also in terms of training data used by machine learning based methods. This is necessary to make binarization not only theoretically possible, but also practically viable. In this thesis, we explore different ways to achieve efficient binarization in terms of execution time by improving the implementation and the algorithm of a state-of-the-art binarization method. We find that parameter prediction, as well as mapping the algorithm onto the graphics processing unit (GPU) help to improve its execution performance. Furthermore, we propose a binarization algorithm based on recurrent neural networks and evaluate the choice of its design parameters with respect to their impact on execution time and binarization quality. Here, we identify a trade-off between binarization quality and execution performance based on the algorithm’s footprint size and show that dynamically weighted training loss tends to improve the binarization quality. Lastly, we address the problem of training data efficiency by evaluating the use of interactive machine learning for reducing the required amount of training data for our recurrent neural network based method. We show that user feedback can help to achieve better binarization quality with less training data and that visualized uncertainty helps to guide users to give more relevant feedback. / Scalable resource-efficient systems for big data analytics

<div>Real-time social media data can provide useful information on evolving events and situations. In addition, various domain users are increasingly leveraging real-time social media data to gain rapid situational awareness. Informed by discussions with first responders and government officials, we focus on two major barriers limiting the widespread adoption of social media for situational awareness: the lack of geotagged data and the deluge of irrelevant information during events. Geotags are naturally useful, as they indicate the location of origin and provide geographic context. Only a small portion of social media is geotagged, however, limiting its practical use for situational awareness. The deluge of irrelevant data provides equal difficulties, impeding the effective identification of semantically relevant information. Existing methods for short text relevance classification fail to incorporate users' knowledge into the classification process. Therefore, classifiers cannot be interactively retrained for specific events or user-dependent needs in real-time, limiting situational awareness. In this work, we first adapt, improve, and evaluate a state-of-the-art deep learning model for city-level geolocation prediction, and integrate it with a visual analytics system tailored for real-time situational awareness. We then present a novel interactive learning framework in which users rapidly identify relevant data by iteratively correcting the relevance classification of tweets in real-time. We integrate our framework with the extended Social Media Analytics and Reporting Toolkit (SMART) 2.0 system, allowing the use of our interactive learning framework within a visual analytics system adapted for real-time situational awareness.</div>

Every day, machine learning (ML) and artificial intelligence (AI) embeds itself further into domestic and industrial technologies. Interaction de- signers have historically struggled to engage directly with the subject, facing a shortage of appropriate methods and abstractions. There is a need to find ways though which interaction design practitioners might integrate ML into their work, in order to democratize and diversify the field. This thesis proposes a mode of inquiry that considers the inter- active qualities of what machine learning does, as opposed the tech- nical specifications of what machine learning is. A shift in focus from the technicality of ML to the artifacts it creates allows the interaction designer to situate its existing skill set, affording it to engage with ma- chine learning as a design material. A Research-through-Design pro- cess explores different methodological adaptions, evaluated through user feedback and an-in depth case analysis. An elaborated design experiment, Multiverse, examines the novel, non-anthropomorphic aesthetic qualities of generative literature. It prototypes interactions with bidirectional literature and studies how these transform the reader into a cybertextual “user-reader”. The thesis ends with a discussion on the implications of machine written literature and proposes a number of future investigations into the research space unfolded through the prototype.

In the domain of cyber security, machine learning promises advanced threat detection. However, the volume of available unlabeled data poses challenges for efficient data management. This study investigates the potential for active learning, a subset of interactive machine learning, to reduce the effort required for manual data labelling. Through different query strategies, the most informative unlabeled data points were selected for manual labelling. The performance of different query strategies was assessed by testing a transformer model’s ability to accurately distinguish tweets mentioning names of advanced persistent threats. The findings suggest that the K-means diversity-based query strategy outperformed both the uncertainty-based approach and the random data point selection, when the amount of labelled training data was limited. This study also evaluated the cost-effective active learning approach, which incorporates high-confidence data points into the training dataset. However, this was shown to be the least effective strategy. Lastly, the study acknowledges that the computational time taken for each query strategy varies significantly between strategies. Hence, an optimal query strategy selection requires a balanced consideration of F-score performance taken together with time efficiency. / Maskininlärning skulle kunna användas för avancerad hotdetektion i cyberdomänen. Dock utgör behovet av träningsdata tillsammans med den stora tillgången till oannoterad data en utmaning. Detta arbete undersöker huruvida aktiv inlärning, en delmängd av interaktiv maskininlärning, kan minska behovet av annoterad data. Genom olika frågestrategier valdes de mest informativa datapunkterna ut för mänsklig annotering. Resultaten för de olika frågestrategierna utvärderades sedan genom att testa en maskininlärningsmodells förmåga att korrekt urskilja tweets som innehåller namn på cyberhotsaktörer. Resultaten tyder på att när mängden annoterad data var begränsad, presterade den diversifieringsbaserade strategin K-means bättre än både den osäkerhetsbaserade frågestrategin och strategin som väljer ut datapunkter slumpmässigt. Denna studie utvärderade också kostnadseffektiv aktiv inlärning som lägger till datapunkter som modellen redan är relativt säker på till träningsdatamängden. Denna metod visade sig dock vara den minst effektiva strategin. Slutligen visar arbetet att beräkningstiden som krävs för varje frågestrategi varierar avsevärt. För att utse den mest optimala frågestrategin krävs därför ett övervägande av både prestanda och tidsåtgång.

[pt] A fisioterapia visa melhorar a funcionalidade física das pessoas, procurando atenuar as incapacidades causadas por alguma lesão, distúrbio ou doença. Nesse contexto, diversas tecnologias computacionais têm sido desenvolvidas com o intuito de apoiar o processo de reabilitação, como as tecnologias adaptáveis para o usuário final. Essas tecnologias possibilitam ao fisioterapeuta adequar aplicações e criarem atividades com características personalizadas de acordo com as preferências e necessidades de cada paciente. Nesta tese é proposta uma abordagem de baixo custo baseada no aprendizado de máquina interativo (iML - Interactive Machine Learning) que visa auxiliar os fisioterapeutas a criarem atividades personalizadas para seus pacientes de forma fácil e sem a necessidade de codificação de software, a partir de apenas alguns exemplos em vídeo RGB (capturadas por uma câmera de vídeo digital) Para tal, aproveitamos a estimativa de pose baseada em aprendizado profundo para rastrear, em tempo real, as articulações-chave do corpo humano a partir de dados da imagem. Esses dados são processados como séries temporais por meio do algoritmo Dynamic Time Warping em conjunto com com o algoritmo K-Nearest Neighbors para criar um modelo de aprendizado de máquina. Adicionalmente, usamos um algoritmo de detecção de anomalias com o intuito de avaliar automaticamente os movimentos. A arquitetura de nossa abordagem possui dois módulos: um para o fisioterapeuta apresentar exemplos personalizados a partir dos quais o sistema cria um modelo para reconhecer esses movimentos; outro para o paciente executar os movimentos personalizados enquanto o sistema avalia o paciente. Avaliamos a usabilidade de nosso sistema com fisioterapeutas de cinco clínicas de reabilitação. Além disso, especialistas avaliaram clinicamente nosso modelo de aprendizado de máquina. Os resultados indicam que a nossa abordagem contribui para avaliar automaticamente os movimentos dos pacientes sem monitoramento direto do fisioterapeuta, além de reduzir o tempo necessário do especialista para treinar um sistema adaptável. / [en] Physiotherapy aims to improve the physical functionality of people, seeking to mitigate the disabilities caused by any injury, disorder or disease. In this context, several computational technologies have been developed in order to support the rehabilitation process, such as the end-user adaptable technologies. These technologies allow the physiotherapist to adapt applications and create activities with personalized characteristics according to the preferences and needs of each patient. This thesis proposes a low-cost approach based on interactive machine learning (iML) that aims to help physiotherapists to create personalized activities for their patients easily and without the need for software coding, from just a few examples in RGB video (captured by a digital video camera). To this end, we take advantage of pose estimation based on deep learning to track, in real time, the key joints of the human body from image data. This data is processed as time series using the Dynamic Time Warping algorithm in conjunction with the K-Nearest Neighbors algorithm to create a machine learning model. Additionally, we use an anomaly detection algorithm in order to automatically assess movements. The architecture of our approach has two modules: one for the physiotherapist to present personalized examples from which the system creates a model to recognize these movements; another to the patient performs personalized movements while the system evaluates the patient. We assessed the usability of our system with physiotherapists from five rehabilitation clinics. In addition, experts have clinically evaluated our machine learning model. The results indicate that our approach contributes to automatically assessing patients movements without direct monitoring by the physiotherapist, in addition to reducing the specialist s time required to train an adaptable system.

Advancements in creative artificial intelligence (AI) are leading to systems that can actively work together with designers in tasks such as ideation, i.e. the creation, development, and communication of ideas. In human group work, making suggestions and explaining the reasoning behind them as well as comprehending other group member’s explanations aids reflection, trust, alignment of goals and inspiration through diverse perspectives. Despite their ability to inspire through independent suggestions, state-of-the-art creative AI systems do not leverage these advantages of group work due to missing or one-sided explanations. For other use cases, AI systems that explain their reasoning are already gathering wide research interest. However, there is a knowledge gap on the effects of explanations on creativity. Furthermore, it is unknown whether a user can benefit from also explaining their contributions to an AI system. This thesis investigates whether reciprocal explanations, a novel technique which combines explanations from and to an AI system, improve the designers’ and AI’s joint exploration of ideas. I integrated reciprocal explanations into an AI aided tool for mood board design, a common method for ideation. In our implementation, the AI system uses text to explain which features of its suggestions match or complement the current mood board. Occasionally, it asks for user explanations providing several options for answers that it reacts to by aligning its strategy. A study was conducted with 16 professional designers who used the tool to create mood boards followed by presentations and semi-structured interviews. The study emphasized a need for explanations that make the principles of the system transparent and showed that alignment of goals motivated participants to provide explanations to the system. Also, enabling users to explain their contributions to the AI system facilitated reflection on their own reasons. / Framsteg inom kreativ artificiell intelligens (AI) har lett till system som aktivt kan samarbeta med designers under idéutformningsprocessen, dvs vid skapande, utveckling och kommunikation av idéer. I grupparbete är det viktigt att kunna göra förslag och förklara resonemanget bakom dem, samt förstå de andra gruppmedlemmarnas resonemang. Detta ökar reflektionsförmågan och förtroende hos medlemmarna, samt underlättar sammanjämkning av mål och ger inspiration genom att höra olika perspektiv. Trots att system, baserade på kreativ artificiell intelligens, har förmågan att inspirera genom sina oberoende förslag, utnyttjar de allra senaste kreativa AI-systemen inte dessa fördelar för att facilitera grupparbete. Detta är på grund av AI-systemens bristfälliga förmåga att resonera över sina förslag. Resonemangen är ofta ensidiga, eller saknas totalt. AI-system som kan förklara sina resonemang är redan ett stort forskningsintresse inom många användningsområden. Dock finns det brist på kunskap om AI-systemens påverkan på den kreativa processen. Dessutom är det okänt om en användare verkligen kan dra nytta av möjligheten att kunna förklara sina designbeslut till ett AI-system. Denna avhandling undersöker om ömsesidiga förklaringar, en ny teknik som kombinerar förklaringar från och till ett AI system, kan förbättra designerns och AI:s samarbete under utforskningen av idéer. Jag integrerade ömsesidiga förklaringar i ett AI-hjälpmedel som underlättar skapandet av stämningsplank (eng. mood board), som är en vanlig metod för konceptutveckling. I vår implementering använder AI-systemet textbeskrivningar för att förklara vilka delar av dess förslag som matchar eller kompletterar det nuvarande stämningsplanket. Ibland ber den användaren ge förklaringar, så den kan anpassa sin förslagsstrategi efter användarens önskemål. Vi genomförde en studie med 16 professionella designers som använde verktyget för att skapa stämningsplank. Feedback samlades genom presentationer och semistrukturerade intervjuer. Studien betonade behovet av förklaringar och resonemang som gör principerna bakom AI-systemet transparenta för användaren. Höjd sammanjämkning mellan användarens och systemets mål motiverade deltagarna att ge förklaringar till systemet. Genom att göra det möjligt för användare att förklara sina designbeslut för AI-systemet, förbättrades också användarens reflektionsförmåga över sina val.