Comparative anatomical studies of diverse plant species are vital for the understanding of changes in gene functions such as those involved in solute transport and hormone signaling in plant roots. The state-of-the-art method for confocal image analysis called PlantSeg utilized U-Net for cell wall segmentation. U-Net is a neural network model that requires training with a large amount of manually labeled confocal images and lacks generalizability. In this research, we test a foundation model called the Segment Anything Model (SAM) to evaluate its zero-shot learning capability and whether prompt engineering can reduce the effort and time consumed in dataset annotation, facilitating a semi-automated training process. Our proposed method improved the detection rate of cells and reduced the error rate as compared to state-of-the-art segmentation tools. We also estimated the IoU scores between the proposed method and PlantSeg to reveal the trade-off between accuracy and detection rate for different quality of data. By addressing the challenges specific to confocal images, our approach offers a robust solution for studying plant structure. Our findings demonstrated the efficiency of SAM in confocal image segmentation, showcasing its adaptability and performance as compared to existing tools. Overall, our research highlights the potential of foundation models like SAM in specialized domains and underscores the importance of tailored approaches for achieving accurate semantic segmentation in confocal imaging. / Master of Science / Studying different plant species' anatomy is crucial for understanding how genes work, especially those related to moving substances and signaling in plant roots. Scientists often use advanced techniques like confocal microscopy to examine plant tissues in detail. Traditional techniques like PlantSeg in automatically segmenting plant cells require a lot of computational resources and manual effort in preparing the dataset and training the model. In this study, we develop a novel technique using Segment-Anything-Model that could learn to identify cells without needing as much training data. We found that SAM performed better than other methods, detecting cells more accurately and making fewer mistakes. By comparing SAM with PlantSeg, we could see how well they worked with different types of images. Our results show that SAM is a reliable option for studying plant structures using confocal imaging. This research highlights the importance of using tailored approaches like SAM to get accurate results from complex images, offering a promising solution for plant scientists.

Human-AI interaction can improve overall performance, exceeding the performance that either humans or AI could achieve separately, thus producing a whole greater than the sum of the parts. Visual analytics enables collaboration between humans and AI through interactive visual interfaces. Semantic interaction is a design methodology to enhance visual analytics systems for sensemaking tasks. It is widely applied for sensemaking in high-stakes domains such as intelligence analysis and academic research. However, existing semantic interaction systems support collaboration between humans and traditional machine learning models only; they do not apply state-of-the-art deep learning techniques. The contribution of this work is the effective integration of deep neural networks into visual analytics systems with semantic interaction. More specifically, I explore how to redesign the semantic interaction pipeline to enable collaboration between human and deep learning models for sensemaking tasks. First, I validate that semantic interaction systems with pre-trained deep learning better support sensemaking than existing semantic interaction systems with traditional machine learning. Second, I integrate interactive deep learning into the semantic interaction pipeline to enhance inference ability in capturing analysts' precise intents, thereby promoting sensemaking. Third, I add semantic explanation into the pipeline to interpret the interactively steered deep learning model. With a clear understanding of DL, analysts can make better decisions. Finally, I present a neural design of the semantic interaction pipeline to further boost collaboration between humans and deep learning for sensemaking. / Doctor of Philosophy / Human AI interaction can harness the separate strengths of human and machine intelligence to accomplish tasks neither can solve alone. Analysts are good at making high-level hypotheses and reasoning from their domain knowledge. AI models are better at data computation based on low-level input features. Successful human-AI interactions can perform real-world, high-stakes tasks, such as issuing medical diagnoses, making credit assessments, and determining cases of discrimination. Semantic interaction is a visual methodology providing intuitive communications between analysts and traditional machine learning models. It is commonly utilized to enhance visual analytics systems for sensemaking tasks, such as intelligence analysis and scientific research. The contribution of this work is to explore how to use semantic interaction to achieve collaboration between humans and state-of-the-art deep learning models for complex sensemaking tasks. To do this, I first evaluate the straightforward solution of integrating the pretrained deep learning model into the traditional semantic interaction pipeline. Results show that the deep learning representation matches human cognition better than hand engineering features via semantic interaction. Next, I look at methods for supporting semantic interaction systems with interactive and interpretable deep learning. The new pipeline provides effective communication between human and deep learning models. Interactive deep learning enables the system to better capture users' intents. Interpretable deep learning lets users have a clear understanding of models. Finally, I improve the pipeline to better support collaboration using a neural design. I hope this work can contribute to future designs for the human-in-the-loop analysis with deep learning and visual analytics techniques.

The premise for AI systems like personal assistants to provide guidance and suggestions to an end-user is to understand, at any moment in time, the personal context that the user is in. The context – where the user is, what she is doing and with whom – allows the machine to represent the world in user’s terms. The context must be inferred from a stream of sensor readings generated by smart wearables such as smartphones and smartwatches, and the labels are acquired from the user directly. To perform robust context prediction in this real-world scenario, the machine must handle the egocentric nature of the context, adapt to the changing world and user, and maintain a bidirectional interaction with the user to ensure the user-machine alignment of world representations. To this end, the machine must learn incrementally on the input stream of sensor readings and user supervision. In this work, we: (i) introduce interactive classification in the wild and present knowledge drift (KD), a special form of concept drift, occurring due to world and user changes; (ii) develop simple and robust ML methods to tackle these scenarios; (iii) showcase the advantages of each of these methods in empirical evaluations on controlled synthetic and real-world data sets; (iv) design a flexible and modular architecture that combines the methods above to support context recognition in the wild; (v) present an evaluation with real users in a concrete social science use case.

Philosophiae Doctor - PhD / This research study investigates optimization of IRS to individual information needs in order of relevance. The research addressed development of algorithms that optimize the ranking of documents retrieved from IRS. In this thesis, we present two aspects of context-awareness in IR. Firstly, the design of context of information. The context of a query determines retrieved information relevance. Thus, executing the same query in diverse contexts often leads to diverse result rankings. Secondly, the relevant context aspects should be incorporated in a way that supports the knowledge domain representing users’ interests. In this thesis, the use of evolutionary algorithms is incorporated to improve the effectiveness of IRS. A context-based information retrieval system is developed whose retrieval effectiveness is evaluated using precision and recall metrics. The results demonstrate how to use attributes from user interaction behaviour to improve the IR effectiveness

As complex systems become more available, the possibility to leverage human intelligence to continuously train these systems is becoming increasingly valuable. Collecting and incorporating feedback from end-users into the system development processes could hold great potential for future development of autonomous systems, but it is not without difficulties A literature review was conducted with the aim to review and help categorize the different dynamics relevant to the act of collecting and implementing user feedback in system development processes. Practical examples of such system are commonly found in active and interactive learning systems, which were studied with a particular interest towards possible novel applications in the industrial sector. This review was complimented by an exploratory experiment, aimed at testing how system accuracy affected the feedback provided by users for a simulated people recognition system. The findings from these studies indicate that when and how feedback is given along with the context of use is of importance for the interplay between system and user. The findings are discussed in relation to current directions in machine learning and interactive learning systems. The study concludes that factors such as system criticality, the phase in which feedback is given, how feedback is given, and the user’s understanding of the learning process all have a large impact on the interactions and outcomes of the user-automation interplay. Suggestions of how to design feedback collection for increased user engagement and increased data assimilation are given.

In this thesis I present explorative work that shows how sounds beyond speak can be used on the input side in the design of interactive experiences and natural interfaces. By engagingin explorative approaches with a material view on sound and interactive machine learning, I’ve shown how these two counterparts may be combined with a goal to envision new possibilities and perspectives on sonic natural interfaces beyond speech. This exploration has been guided with a theoretical background ofdesign materials, machine learning, sonic interaction design and with a research through design driven process, I’ve used iterative prototyping and workshops with participants to conduct knowledge and guide the explorative process. My design work has resultedin new prototyping tools for designers to work with sound and interactive machine learning as well as a prototypes concept for kids that aims to manifest the material ndings around sound and interactive machine learning that I’ve done in this project.By evaluating my design work in contextual settings with participants, I’ve conducted both analytical and productive investigations than can construct new perspectives on how sound based interfaces beyond speech can be designed to support new interactive experiences with artefacts. Here my focus has been to engage with sound as a design material from both contextual and individual perspectives, and how this can be explored by end-users empowered by interactive machine learning to foster new forms of creative engagement with our physical world.

Antagning.se and universityadmissions.se are two websites that enables people to apply for higher education. These websites are developed and maintained by ITS, which is a department at Umeå University. Antagning.se and universityadmissions.se allows applicants to add documents such as grades through a document upload function. Recently, There has been some experimentation with machine learning as a way to read documents that are uploaded. This study explores the possibility to use machine-learning in the user interface of the document upload function in a way that assists the users to upload the documents correctly. The objective is to determine whether doing this affects the users confidence that they uploaded a document successfully. The Double diamond method was used to design a lo-fi, a mid-fi and a hi-fi prototype. The lo-fi prototype were developed during an innovation sprint at ITS, with the purpose to develop a user interface for a Swedish folk high school validation system. The mid-fi prototype were tested using a qualitative user test to find issues that had to be addressed in the hi-fi prototype. A quantitative user test were conducted in order to determine if it affected the users confidence that they completed a task successfully versus if the same task were performed on the current system in use by Antagning and University Admissions. The results from the user testing of the hi-fi prototype were analyzed. Using hypothesis testing, it could not be determined that there were a significant difference in user confidence between the hi-fi prototype and the current system. / Antagning.se och universityadmissions.se är två webbplatser vars syften är att möjliggöra ansökan till högre utbildning i Sverige. ITS, som är en del av Umeå Universitet, är ansvariga för att driva och utveckla dessa webbplatser. Som en del av ansökningsprocessen så krävs det i vissa fall kompletterande information, exempelvis betyg eller andra typer av dokument. Personen som söker kan då scanna in dessa dokument och ladda upp den genom antagning.se eller universityadmissions.se. ITS har experimenterat med maskinlärning för att läsa in dessa inscannade dokument. Detta arbete utforskar möjligheten att använda maskinlärning i användargränssnittet till denna dokumentuppladdningsfunktion. Detta som ett sätt att assistera användaren i att se till att dokumenten laddas upp korrekt. Syftet med detta avgöra om detta har någon inverkan på hur säker användaren är att hen har genomfört uppladningen av dokumentet på ett korrekt sätt. Double diamond-metoden användes för att utveckla en lo-fi, en mid-fi och en hi-fi-prototyp. Lo-fi-prototypen togs fram under en innovationssprint på ITS, där syftet var att utveckla ett användargränssnitt till ett valideringssystem för folkhögskoledokument som maskinlärning. Mid-fi-prototypen testades med en kvalitativ metod för att hitta problem med gränssnittets design. Dessa problem togs i åtanke när hi-fi-prototypen togs fram. En kvantitativ användarstudie genomfördes för att avgöra om användaren upplevde någon skillnad i hur säkra de var att de laddat upp ett dokument korrekt. Detta jämfört med det befintliga systemet som antagning.se idag använder för att ladda upp dokument. Resultatet av studien var att det inte gick att påvisa någon skillnad mellan det befintliga systemet och hi-fi-prototypen gällande hur säker användaren var att dokumentet laddats upp korrekt.

Interactive machine learning (IML) is a promising approach for personalising physiotherapy training. This thesis uses a research- through-design and reflective approach to explore how IML can be ecologically integrated within in-clinic physiotherapy. Domain expert interviews and observations with physiotherapists were conducted to gain a broader understanding of the physiotherapy context, the role of feedback provided to patients, and how technology could be integrated into this context. Three design artefacts were proposed to participants to provoke a discussion on the implications and current practices. Due to time constraints, the findings suggest incorporation within the consultation may be difficult. The clinic’s gym or administration time revealed to be promising alternatives. Furthermore, results highlight the importance of IML supporting richer interaction forms and the implicitness and flexibility needed to describe movement and feedback, which define physiotherapy’s hands-on approach. Togetherness was a reoccurring theme, suggesting that the input and guidance of the IML system could be something done with the patient. Finally, a reflection on the results and the study opens up a discussion of the fitness of IML in physiotherapy.

With the Internet of Things paradigm, the data generated by the rapidly increasing number of connected devices lead to new possibilities, such as using machine learning for activity recognition in smart environments. However, it also introduces several challenges. The sensors of different devices might be of different types, making the fusion of data non-trivial. Moreover, the devices are often mobile, resulting in that data from a particular sensor is not always available, i.e. there is a need to handle data from a dynamic set of sensors. From a machine learning perspective, the data from the sensors arrives in a streaming fashion, i.e., online learning, as compared to many learning problems where a static dataset is assumed. Machine learning is in many cases a good approach for classification problems, but the performance is often linked to the quality of the data. Having a good data set to train a model can be an issue in general, due to the often costly process of annotating the data. With dynamic and heterogeneous data, annotation can be even more problematic, because of the ever-changing environment. This means that there might not be any, or a very small amount of, annotated data to train the model on at the start of learning, often referred to as the cold start problem. To be able to handle these issues, adaptive systems are needed. With adaptive we mean that the model is not static over time, but is updated if there for instance is a change in the environment. By including human-in-the-loop during the learning process, which we refer to as interactive machine learning, the input from users can be utilized to build the model. The type of input used is typically annotations of the data, i.e. user input in the form of correctly labelled data points. Generally, it is assumed that the user always provides correct labels in accordance with the chosen interactive learning strategy. In many real-world applications these assumptions are not realistic however, as users might provide incorrect labels or not provide labels at all in line with the chosen strategy. In this thesis we explore which interactive learning strategies are possible in the given scenario and how they affect performance, as well as the effect of machine learning algorithms on performance. We also study how a user who is not always reliable, i.e. that does not always provide a correct label when expected to, can affect performance. We propose a taxonomy of interactive online machine learning strategies and test how the different strategies affect performance through experiments on multiple datasets. The findings show that the overall best performing interactive learning strategy is one where the user provides labels when previous estimations have been incorrect, but that the best performing machine learning algorithm depends on the problem scenario. The experiments also show that a decreased reliability of the user leads to decreased performance, especially when there is a limited amount of labelled data.

Nowadays the Learning Management System (LMS) platforms provide limited bidirectional, interactive mechanisms that they are competent to handle personal or small-scale distance learning systems. These mechanisms are designed for one to many online tutorials, and the technology utilizes single-input by single-output video stream technology, the video and audio data need to be coupled with one or many Multipoint Control Units (MCU) to mix or convert them into a single output media stream. In this platform MCU is critical to LMS, however, such system is expensive, lack of capacity and difficult to be massively deployed. Access-Grid (AG), an Open Source program, offers users capability to watch online multimedia audio-video contents from all the interconnected nodes of LMS through Multicast protocol, and supports groups-to-group high quality interactive distance learning. It requires all the networks to support the Multicast protocol. The MBONE (Multicast Backbone) can be used to connect different Multicast groups via Unicast communication. However, if the number of groups involving in the distance learning are large, the host computers or routers of the network will be heavily loaded because they need to handle the delivering of the media packets. To use a QuickBridge for aggregating and delivering packages is an alternative of LMS and requires (N-1) *N *BW bandwidth . For example, if there is a 15 nodes online conference and each node uses 800kbps data rate to transmit audio-video contents, then the demanded bandwidth of the aggregation is 168 Mbps. The way of dispersing and controlling the data flow becomes important factors and will greatly affect the quality of the AG online conference. This thesis modifies the procedure of AG and QuickBridge, and allows all the AG Clients to be able to transmit Unicast and Multicast packets in the online conference. It offers a Meeting Management Server to dynamically adjust topology and hub points, and achieves better elasticity to the system. By modifying VIC and RAT procedure, the system controls the outbound audio-video data flow from each nodes of online conference, and reduces the demand of bandwidth. The system can directly provide end-to-end conferencing, using Unicast communication to connect the nodes in different Multicast groups, or using the Multicast on the backbone and then using unicast communication to the local nodes. The functionality of the LMS can be improved and capable of supporting multi-windows to multi-user interactive online conference for the users. The results of this thesis can be applied upon real-time interactive distance learning, online video conferencing and interactive online TV. It also helps to lower the cost of the system and reduce the requirement of network bandwidth.