[pt] Este trabalho considera o problema de Regressão com Tempo Limitado. Dados um dataset, um algoritmo de aprendizado (Learner) a ser treinado e um tempo limitado, não sabemos se seria possível treinar o modelo com todo o dataset dentro deste tempo. Queremos então elaborar a estratégia que extraia o melhor modelo possível deste algoritmo de aprendizado respeitando o limite de tempo. Uma estratégia consiste em interagir com o Learner de duas formas: enviando exemplos para o Learner treinar e enviando exemplos para o Learner rotular. Nós definimos o que é o problema de Regressão com Tempo Limitado, decompomos o problema de elaborar uma estratégia em subproblemas mais simples e bem definidos, elaboramos uma estratégia natural baseada em escolha aleatória de exemplos e finalmente apresentamos uma estratégia, TW+BH, que supera a estratégia natural em experimentos que realizamos com diversos datasets reais. / [en] This work considers the Time-Limited Regression problem. Given a dataset, a learning algorithm (Learner) to be trained and a limited time, we do not know if it s going to be possible to train the model with the entire dataset within this time constraint. We then want to elaborate the strategy that extracts the best possible model from this learning algorithm respecting the time limit. A strategy consists of a series of interactions with the Learner, in two possible ways: sending labeled examples for the Learner to train and sending unlabeled examples for the Learner to classify. We define what the Time-Limited Regression problem is, we decompose the problem of elaborating a strategy into simpler and more well-defined sub-problems, we elaborate a natural strategy based on random choice of examples and finally we present a strategy, TW+BH, that performs better than the natural strategy in experiments we have done with several real datasets.

[pt] Com a crescente quantidade de dados sendo gerados e coletados, torna-se mais comum cenários em que se dispõe de dados rotulados em larga escala, mas com recursos computacionais limitados, de modo que não seja possível treinar modelos preditivos utilizando todas as amostras disponíveis. Diante dessa realidade, adotamos o paradigma de Machine Teaching como uma alternativa para obter modelos eficazes utilizando um subconjunto representativo dos dados disponíveis. Inicialmente, consideramos um problema central da área de Machine Teaching que consiste em encontrar o menor conjunto de amostras necessário para obter uma dada hipótese alvo h(asterisco). Adotamos o modelo de ensino black-box learner introduzido em (DASGUPTA et al., 2019), em que o ensino é feito interativamente sem qualquer conhecimento sobre o algoritmo do learner e sua classe de hipóteses, exceto que ela contém a hipótese alvo h(asterisco). Refinamos alguns resultados existentes para esse modelo e estudamos variantes dele. Em particular, estendemos um resultado de (DASGUPTA et al., 2019) para o cenário mais realista em que h(asterisco) pode não estar contido na classe de hipóteses do learner e, portanto, o objetivo do teacher é fazer o learner convergir para a melhor aproximação disponível de h(asterisco). Também consideramos o cenário com black-box learners não adversários e mostramos que podemos obter melhores resultados para o tipo de learner que se move para a próxima hipótese de maneira suave, preferindo hipóteses que são mais próximas da hipótese atual. Em seguida, definimos e abordamos o problema de Aprendizado com Restrição de Tempo considerando um cenário em que temos um enorme conjunto de dados e um limite de tempo para treinar um dado learner usando esse conjunto. Propomos o método TCT, um algoritmo para essa tarefa, desenvolvido com base nos princípios de Machine Teaching. Apresentamos um estudo experimental envolvendo 5 diferentes learners e 20 datasets no qual mostramos que TCT supera métodos alternativos considerados. Finalmente, provamos garantias de aproximação para uma versão simplificada do TCT. / [en] With the growing amount of data being generated and collected, it becomes increasingly common to have scenarios where there are large-scale labeled data but limited computational resources, making it impossible to train predictive models using all available samples. Faced with this reality, we adopt the Machine Teaching paradigm as an alternative to obtain effective models using a representative subset of available data. Initially, we consider a central problem of the Machine Teaching area which consists of finding the smallest set of samples necessary to obtain a given target hypothesis h(asterisk). We adopt the black-box learner teaching model introduced in (DASGUPTA et al., 2019), where teaching is done interactively without any knowledge about the learner s algorithm and its hypothesis class, except that it contains the target hypothesis h(asterisk). We refine some existing results for this model and study its variants. In particular, we extend a result from (DASGUPTA et al., 2019) to the more realistic scenario where h(asterisk) may not be contained in the learner s hypothesis class, and therefore, the teacher s objective is to make the learner converge to the best available approximation of h(asterisk). We also consider the scenario with non-adversarial black-box learners and show that we can obtain better results for the type of learner that moves to the next hypothesis smoothly, preferring hypotheses that are closer to the current hypothesis. Next, we address the Time-Constrained Learning problem, considering a scenario where we have a huge dataset and a time limit to train a given learner using this dataset. We propose the TCT method, an algorithm for this task, developed based on Machine Teaching principles. We present an experimental study involving 5 different learners and 20 datasets in which we show that TCT outperforms alternative methods considered. Finally, we prove approximation guarantees for a simplified version of TCT.

I en framtid där förutsägelser och beslut som tas av maskininlärningssystem överträffar människors förmåga behöver systemen att vara tolkbara för att vi skall kunna lita på och förstå dem. Vår studie utforskar världen av tolkbar maskininlärning genom att designa och undersöka artefakter. Vi genomför experiment för att utforska förklarbarhet, tolkbarhet samt tekniska utmaningar att skapa maskininlärningsmodeller för att identifiera liknande men unika objekt. Slutligen genomför vi ett användartest för att utvärdera toppmoderna förklaringsverktyg i ett direkt mänskligt sammanhang. Med insikter från dessa experiment diskuterar vi den potentiella framtiden för detta fält / In a future where predictions and decisions made by machine learning systems outperform humans we need the systems to be interpretable in order for us to trust and understand them. Our study explore the realm of interpretable machine learning through designing artifacts. We conduct experiments to explore explainability, interpretability as well as technical challenges of creating machine learning models to identify objects that appear similar to humans. Lastly, we conduct a user test to evaluate current state-of-the-art visual explanatory tools in a human setting. From these insights, we discuss the potential future of this field.

With the Internet of Things paradigm, the data generated by the rapidly increasing number of connected devices lead to new possibilities, such as using machine learning for activity recognition in smart environments. However, it also introduces several challenges. The sensors of different devices might be of different types, making the fusion of data non-trivial. Moreover, the devices are often mobile, resulting in that data from a particular sensor is not always available, i.e. there is a need to handle data from a dynamic set of sensors. From a machine learning perspective, the data from the sensors arrives in a streaming fashion, i.e., online learning, as compared to many learning problems where a static dataset is assumed. Machine learning is in many cases a good approach for classification problems, but the performance is often linked to the quality of the data. Having a good data set to train a model can be an issue in general, due to the often costly process of annotating the data. With dynamic and heterogeneous data, annotation can be even more problematic, because of the ever-changing environment. This means that there might not be any, or a very small amount of, annotated data to train the model on at the start of learning, often referred to as the cold start problem. To be able to handle these issues, adaptive systems are needed. With adaptive we mean that the model is not static over time, but is updated if there for instance is a change in the environment. By including human-in-the-loop during the learning process, which we refer to as interactive machine learning, the input from users can be utilized to build the model. The type of input used is typically annotations of the data, i.e. user input in the form of correctly labelled data points. Generally, it is assumed that the user always provides correct labels in accordance with the chosen interactive learning strategy. In many real-world applications these assumptions are not realistic however, as users might provide incorrect labels or not provide labels at all in line with the chosen strategy. In this thesis we explore which interactive learning strategies are possible in the given scenario and how they affect performance, as well as the effect of machine learning algorithms on performance. We also study how a user who is not always reliable, i.e. that does not always provide a correct label when expected to, can affect performance. We propose a taxonomy of interactive online machine learning strategies and test how the different strategies affect performance through experiments on multiple datasets. The findings show that the overall best performing interactive learning strategy is one where the user provides labels when previous estimations have been incorrect, but that the best performing machine learning algorithm depends on the problem scenario. The experiments also show that a decreased reliability of the user leads to decreased performance, especially when there is a limited amount of labelled data.

Conversational agents and virtual reality are two emerging technologies that are increasingly being explored in mental health research. Although the combination of these technologies could provide easily accessible and cost-effective treatment for a wide range of health behaviors, use of this opportunity in healthcare has not yet been undertaken. Inherent difficulties of leveraging conversational agent solutions for non machine learning(ML) experts, and challenges - operational as well as legislative -regarding the exploitation of its capabilities, has posed a bottleneck for further adoption in the domain. This study applies interactive machine teaching principles, in the creation of a technical system enabling health-care clinicians (or any other non-ML experts) continuously develop and improve conversational agents deployed in VR applications. The methodology adopted first conducted a literature review providing background knowledge of the field, how conversational agents in VR could support clinicians and patients, methodological aspects and focus of the study. Secondly, a proof-of-concept VR system implementing a conversational agent for stress rehabilitation was developed in joint consultation with psychologists and later evaluated in an experimental study. This study presents a novel approach for deploying VR-based health interventions implementing conversational agents, and findings suggest the feasibility of such applications and the opportunities it has to support clinicians and patients in reaching their goals.