We introduce Domain Aware Continual Zero-Shot Learning (DACZSL), the task of visually recognizing images of unseen categories in unseen domains sequentially. We created DACZSL on top of the DomainNet dataset by dividing it into a sequence of tasks, where classes are incrementally provided on seen domains during training and evaluation is conducted on unseen domains for both seen and unseen classes. We also proposed a novel Domain-Invariant CZSL Network (DIN), which outperforms state-of-the-art baseline models that we adapted to DACZSL setting. We adopt a structure-based approach to alleviate forgetting knowledge from previous tasks with a small per-task private network in addition to a global shared network. To encourage the private network to capture the domain and task-specific representation, we train our model with a novel adversarial knowledge disentanglement setting to make our global network task-invariant and domain-invariant over all the tasks. Our method also learns a class-wise learnable prompt to obtain better class-level text representation, which is used to represent side information to enable zero-shot prediction of future unseen classes. Our code and benchmarks are made available at https://zero-shot-learning.github.io/daczsl.

Over the last decade, the integration of robots into various applications has seen significant advancements fueled by Machine Learning (ML) algorithms, particularly in autonomous and independent operations. While robots have become increasingly proficient in various tasks, object instance recognition, a fundamental component of real-world robotic interactions, has witnessed remarkable improvements in accuracy and robustness. Nevertheless, most existing approaches heavily rely on prior information, limiting their adaptability in unfamiliar environments. To address this constraint, this thesis introduces the Segment and Learn Semantics (SaLS) framework, which combines video object segmentation with Continual Learning (CL) methods to enable semantic understanding in robotic applications. The research focuses on the potential application of SaLS in mobile robotics, with specific emphasis on the TORO robot developed at the Deutsches Zentrum für Luft- und Raumfahrt (DLR). Evaluation of the proposed method is conducted using a diverse dataset comprising various terrains and objects encountered by the TORO robot during its walking sessions. The results demonstrate the effectiveness of SaLS in classifying both known and previously unseen objects, achieving an average accuracy of 78.86% and 70.78% in the CL experiments. When running the whole method in the image sequences collected with TORO, the accuracy scores were of 75.54% and 84.75%, for known and unknown objects respectively. Notably, SaLS exhibited resilience against catastrophic forgetting, with only minor accuracy decreases observed in specific cases. Computational resource usage was also explored, indicating that the method is feasible for practical mobile robotic systems, with GPU memory usage being a potential limiting factor. In conclusion, the SaLS framework represents a significant step forward in enabling robots to autonomously understand and interact with their surroundings. This research contributes to the ongoing development of robotic systems that can operate effectively in unstructured environments, paving the way for more versatile and capable autonomous robots.

In data science, machine learning methods, such as deep learning and other AI algorithms, have been widely used in many applications. These machine learning methods often have complicated model structures with a large number of model parameters and a set of hyperparameters. Moreover, these machine learning methods are data-driven in nature. Thus, it is not easy to provide a comprehensive evaluation on the performance of these machine learning methods with respect to the data quality and hyper-parameters of the algorithms. In the statistical literature, design of experiments (DoE) is a set of systematical methods to effectively investigate the effects of input factors for the complex systems. There are few works focusing on the use of DoE methodology for evaluating the quality assurance of AI algorithms, while an AI algorithm is naturally a complex system. An understanding of the quality of Artificial Intelligence (AI) algorithms is important for confidently deploying them in real applications such as cybersecurity, healthcare, and autonomous driving. In this proposal, I aim to develop a set of novel methods on the interface between experimental design and machine learning, providing a systematical framework of using DoE methodology for AI algorithms. This proposal contains six chapters. Chapter 1 provides a general introduction of design of experiments, machine learning, and surrogate modeling. Chapter 2 focuses on investigating the robustness of AI classification algorithms by conducting a comprehensive set of mixture experiments. Chapter 3 proposes a so-called Do-AIQ framework of using DoE for evaluating the AI algorithm’s quality assurance. I establish a design-of-experiment framework to construct an efficient space-filling design in a high-dimensional constraint space and develop an effective surrogate model using additive Gaussian process to enable the quality assessment of AI algorithms. Chapter 4 introduces a framework to generate continual learning (CL) datsets for cybersecurity applications. Chapter 5 presents a variable selection method under cumulative exposure model for time-to-event data with time-varying covariates. Chapter 6 provides the summary of the entire dissertation. / Doctor of Philosophy / Artificial intelligence (AI) techniques, including machine learning and deep learning algorithms, are widely used in various applications in the era of big data. While these algorithms have impressed the public with their remarkable performance, their underlying mechanisms are often highly complex and difficult to interpret. As a result, it becomes challenging to comprehensively evaluate the overall performance and quality of these algorithms. The Design of Experiments (DoE) offers a valuable set of tools for studying and understanding the underlying mechanisms of complex systems, thereby facilitating improvements. DoE has been successfully applied in diverse areas such as manufacturing, agriculture, and healthcare. The use of DoE has played a crucial role in enhancing processes and ensuring high quality. However, there are few works focusing on the use of DoE methodology for evaluating the quality assurance of AI algorithms, where an AI algorithm can be naturally considered as a complex system. This dissertation aims to develop innovative methodologies on the interface between experimental design and machine learning. The research conducted in this dissertation can serve as practical tools to use DoE methodology in the context of AI algorithms.

This thesis addresses the crucial problem of knowledge transfer and retention in deep neural networks. The ability to transfer knowledge from previously learned tasks and retain it for future use is essential for machine learning models to continually adapt to new tasks and improve their overall performance. In principle, knowledge can be transferred between any type of task, but we believe it to be particularly challenging in the field of computer vision, where the size and diversity of visual data often result in high compute requirements and the need for large, complex models. Hence, we analyze transfer and retention learning between unsupervised and supervised visual tasks, which form the main focus of this thesis. We categorize our efforts into several knowledge transfer and retention paradigms, and we tackle them with several contributions for the scientific community. The thesis proposes settings and methods based on knowledge distillation and self-supervised learning techniques. In particular, we devise two novel continual learning settings and seven new methods for knowledge transfer and retention, setting new state-of-the-art in a wide range of tasks. In conclusion, this thesis provides a valuable contribution to the field of computer vision and machine learning and sets a foundation for future work in this area.

Humans are able to develop a solid knowledge of the world around them: they can leverage information coming from different sources (e.g., language, vision), focus on the most relevant information from the input they receive in a given life situation, and exploit what they have learned before without forgetting it. In the field of Artificial Intelligence and Computational Linguistics, replicating these human abilities in artificial models is a major challenge. Recently, models based on pre-training and on attention mechanisms, namely pre-trained multimodal Transformers, have been developed. They seem to perform tasks surprisingly well compared to other computational models in multiple contexts. They simulate a human-like cognition in that they supposedly rely on previously acquired knowledge (transfer learning) and focus on the most important information (attention mechanisms) of the input. Nevertheless, we still do not know whether these models can deal with multimodal tasks that require merging different types of information simultaneously to be solved, as humans would do. This thesis attempts to fill this crucial gap in our knowledge of multimodal models by investigating the ability of pre-trained Transformers to encode multimodal information; and the ability of attention-based models to remember how to deal with previously-solved tasks. With regards to pre-trained Transformers, we focused on their ability to rely on pre-training and on attention while dealing with tasks requiring to merge information coming from language and vision. More precisely, we investigate if pre-trained multimodal Transformers are able to understand the internal structure of a dialogue (e.g., organization of the turns); to effectively solve complex spatial questions requiring to process different spatial elements (e.g., regions of the image, proximity between elements, etc.); and to make predictions based on complementary multimodal cues (e.g., guessing the most plausible action by leveraging the content of a sentence and of an image). The results of this thesis indicate that pre-trained Transformers outperform other models. Indeed, they are able to some extent to integrate complementary multimodal information; they manage to pinpoint both the relevant turns in a dialogue and the most important regions in an image. These results suggest that pre-training and attention play a key role in pre-trained Transformers’ encoding. Nevertheless, their way of processing information cannot be considered as human-like. Indeed, when compared to humans, they struggle (as non-pre-trained models do) to understand negative answers, to merge spatial information in difficult questions, and to predict actions based on complementary linguistic and visual cues. With regards to attention-based models, we found out that these kinds of models tend to forget what they have learned in previously-solved tasks. However, training these models on easy tasks before more complex ones seems to mitigate this catastrophic forgetting phenomenon. These results indicate that, at least in this context, attention-based models (and, supposedly, pre-trained Transformers too) are sensitive to tasks’ order. A better control of this variable may therefore help multimodal models learn sequentially and continuously as humans do.

Bakgrund: En kund köper inte varor och tjänster som inte skapar något värde, vilket är något företagen på marknaden måste ta hänsyn till. Utöver det är företagens mål att nå finansiella samt icke-finansiella vinningar från sin produkt. Det är stor konkurrens på marknaden vilket innebär att konsumenten kan välja och vraka något som företagen måste anpassa sig till. I tjänstesektorn är det kunskap som är den viktigaste faktorn för att locka till sig kunder och är på så sätt något tjänsteföretagen ständigt måste arbeta med om de skall överleva på marknaden. För att undersöka detta kommer företagens syn på värdeskapande, kontinuerligt lärande och de egna resurserna studeras i uppsatsen. Syfte: Att undersöka hur tjänsteföretag arbetar med värdeskapande, kontinuerligt lärande samt dess nedlagda resurser inom ramen för internt utvecklingsarbete. Vidare är syftet att jämföra tjänsteföretagens mognadsgrader med varandra för att därigenom se vilka skillnader samt var potential till förbättring finns. Metod: Uppsatsen grundar sig i den kvalitativa ansatsen med ett heremeneutiskt förhållningssätt. Den sekundärdata som samlades in var genom vetenskapliga artiklar, internetkällor och böcker. Fyra företag och fem personer deltog i uppsatsen vilka intervjuades genom en semi-strukturerad intervjuguide. Datainsamlingen analyserades med hjälp av ett organisatoriskt IQ-test. Resultat och slutsats: Företagen har en liknande grundsyn i sättet de tänker på värdeskapande och kontinuerligt lärande. Dock visar empirin att företagen skildje sig i konflikthanteringen samt att några av företagen hade svårigheter att förändra sig på lokal nivå eftersom de agerade på en global marknad. För att uppnå ett värdeskapande och kontinuerligt lärande lägger samtliga företag ner mycket resurser för att erbjuda en tjänst som möter kundens behov och krav. Hur mognadsgraden för värdeskapande och kontinuerligt lärande ser ut skiljer sig mellan företagen eftersom företagen var bra på olika saker. Det är även inom dessa områden potential till förbättring finns. Uppsatsens bidrag: I sin helhet ger uppsatsen en ökad kunskap om hur tjänsteföretag i den privata tjänstesektorn arbetar med värdeskapande och kontinuerligt lärande. Därutöver bidrar uppsatsen med ytterligare kunskap inom vilka områden tjänsteföretagen utmärker sig, deras mognadsgrad i värdeskapande och kontinuerligt lärande samt var det finns potential till förbättring. / Background: Customer do not intend to purchase a product or service that does not provide them with value and this is something that companies have to consider. At the same time companies aim to achieve financial or non-financial profits by selling their products or services. The market is very competitive and ultimately it is the consumers who have the power to pick and choose whatever alternative that fits them best. Hence, this is something companies have to take into consideration. The private sector provides services rather than products and it is this which is used to attract customers and therefore something companies continually have to work with in order to survive in the market. In order to investigate this, different companies’ view of value creation, continual learning and resources have been studied. Aim: To investigate how service companies work with value creation, continual learning and utilized resources (within the range of internal development work). The aim is also to compare company’s level of maturity and suggest where improvements can be made. Methodology: The study is qualitative with a hermeneutic approach. Data were collected from scientific articles, internet sources and books. Four companies and five persons in total participated in the study and were interviewed using a semi-structured interview form. The data was later analyzed with the help of an organizational IQ-test. Results and conclusion: Companies often have a similar ethos in the way they think about value creation and continuous learning. However, the empirical data shows that companies differ in regard to conflict management, as well as some of the companies having difficulties in making changes at the local level because they act in a global marked. In order to achieve value creation and continuous learning all the companies use a great deal of resources to meet the needs and demands from customers. However, the maturity level of value creation and continuous learning look different between various companies because the companies are good at different activities. It is in these areas there lies the potential for improvement. Contribution of the thesis: It provides increased knowledge in how the private service sector works with value creation and continual learning. It also provides further knowledge of how the service sector excels, its maturity grade in value creation and continual learning, and how they can improve.

<p>This dissertation explore two major goals in Deep Learning algorithm design: efficiency and trustworthiness. We motivate these concerns in Chapter 1 and give relevant background in Chapter 2. We then discuss six works to target these two goals. </p> <p>The first of these discusses how to make the model compression methodology more efficient, so it can be done in a single shot. This allows us to create models with reduced size and layers, so we can have faster and more efficient inference, and is covered in Chapter 3. We then extend this to target efficiency in continual learning in Chapter 4, while mitigating the problem of catastrophic forgetting. The method discussed also allows us to circumvent the potential for data leakage by avoiding the need to store any data from the past tasks. Next, we consider brain-inspired computing as an alternative to traditional neural networks to improve compute efficiency of networks. The spiking neural networks discussed however have large inference latency due to the need for accumulating spikes over many timesteps. We tackle this by introducing a new scheme that distributes an image over time by breaking it down into a sum of its ranked sinusoidal bases in Chapter 5. This results in networks that are faster and more efficient to deploy. Chapter 6 targets mitigating both the communication expense and potential for data leakage in federated learning, by distilling the gradients to be communicated in a small number of images that resemble noise. Communicating these images is more efficient, and circumvents the potential for data leakage as they resemble noise. We then explore the applications of studying curvature of loss with respect to input data points in the last two chapters. We first utilize curvature to create performant coresets to reduce the size of datasets, to make training more efficient in Chapter 7. In Chapter 8, we use curvature as a metric for overfitting and use it to expose dataset integrity issues arising from memorization.</p>

Le temps de mélange de la chaîne de Markov induite par une politique limite ses performances dans les scénarios réels d'apprentissage continu. Pourtant, l'effet des temps de mélange sur l'apprentissage dans l'apprentissage par renforcement (RL) continu reste peu exploré. Dans cet article, nous caractérisons des problèmes qui sont d'un intérêt à long terme pour le développement de l'apprentissage continu, que nous appelons processus de décision markoviens (MDP) « extensibles » (scalable), à travers le prisme des temps de mélange. En particulier, nous établissons théoriquement que les MDP extensibles ont des temps de mélange qui varient de façon polynomiale avec la taille du problème. Nous démontrons ensuite que les temps de mélange polynomiaux présentent des difficultés importantes pour les approches existantes, qui souffrent d'un biais myope et d'estimations à base de ré-échantillonnage avec remise ensembliste (bootstrapping) périmées. Pour valider notre théorie, nous étudions la complexité des temps de mélange en fonction du nombre de tâches et de la durée des tâches pour des politiques très performantes déployées sur plusieurs jeux Atari. Notre analyse démontre à la fois que des temps de mélange polynomiaux apparaissent en pratique et que leur existence peut conduire à un comportement d'apprentissage instable, comme l'oubli catastrophique dans des contextes d'apprentissage continu. / The mixing time of the Markov chain induced by a policy limits performance in real-world continual learning scenarios. Yet, the effect of mixing times on learning in continual reinforcement learning (RL) remains underexplored. In this paper, we characterize problems that are of long-term interest to the development of continual RL, which we call scalable MDPs, through the lens of mixing times. In particular, we theoretically establish that scalable MDPs have mixing times that scale polynomially with the size of the problem. We go on to demonstrate that polynomial mixing times present significant difficulties for existing approaches, which suffer from myopic bias and stale bootstrapped estimates. To validate our theory, we study the empirical scaling behavior of mixing times with respect to the number of tasks and task duration for high performing policies deployed across multiple Atari games. Our analysis demonstrates both that polynomial mixing times do emerge in practice and how their existence may lead to unstable learning behavior like catastrophic forgetting in continual learning settings.

While deep learning has proved to be useful in a large variety of tasks, a limitation remains of needing all classes and samples to be present at the training stage in supervised problems. This is a major issue in the field of biomedical imaging since keeping samples in the training sets consistently is often a liability. Furthermore, this issue prevents the simple updating of older models with only the new data when it is introduced, and prevents collaboration between companies. In this work, we examine an array of Continual Learning approaches to try to improve upon the baseline of the naive finetuning approach when retraining on new tasks, and achieve accuracy levels similar to the ones seen when all the data is available at the same time. Continual learning approaches with which we attempt to mitigate the problem are EWC, UCB, EWC Online, SI, MAS, CN-DPM. We explore some complex scenarios with varied classes being included in the tasks, as well as close to ideal scenarios where the sample size is balanced among the tasks. Overall, we focus on X-ray images, since they encompass a large variety of diseases, with new diseases requiring retraining. In the preferred setting, where classes are relatively balanced, we get an accuracy of 63.30 versus a baseline of 53.92 and the target score of 66.83. For the continued training on the same classes, we get an accuracy of 35.52 versus a baseline of 27.73. We also examine whether learning rate adjustments at task level improve accuracy, with some improvements for EWC Online. The preliminary results indicate that CL approaches such as EWC Online and SI could be integrated into radiography data learning pipelines to reduce catastrophic forgetting in situations where some level of sequential training ability justifies the significant computational overhead. / Även om djupinlärning har visat sig vara användbart i en mängd olika uppgifter, kvarstår en begränsning av att behöva alla klasser och prover som finns på utbildningsstadiet i övervakade problem. Detta är en viktig fråga inom området biomedicinsk avbildning eftersom det ofta är en belastning att hålla prover i träningsuppsättningarna. Dessutom förhindrar det här problemet enkel uppdatering av äldre modeller med endast nya data när de introduceras och förhindrar samarbete mellan företag. I det här arbetet undersöker vi en rad kontinuerliga inlärningsmetoder för att försöka förbättra baslinjen för den naiva finjusteringsmetoden vid omskolning på nya uppgifter och närma sig noggrannhetsnivåer som de som ses när alla data är tillgängliga samtidigt. Kontinuerliga inlärningsmetoder som vi försöker mildra problemet med inkluderar bland annat EWC, UCB, EWC Online, SI. Vi utforskar några komplexa scenarier med olika klasser som ingår i uppgifterna, samt nära idealiska scenarier där exempelstorleken balanseras mellan uppgifterna. Sammantaget fokuserar vi på röntgenbilder, eftersom de omfattar ett stort antal sjukdomar, med nya sjukdomar som kräver omskolning. I den föredragna inställningen får vi en noggrannhet på 63,30 jämfört med en baslinje på 53,92 och målpoängen på 66,83. Medan vi för den utökade träningen på samma klasser får en noggrannhet på 35,52 jämfört med en baslinje på 27,73. Vi undersöker också om justeringar av inlärningsfrekvensen på uppgiftsnivå förbättrar noggrannheten, med vissa förbättringar för EWC Online. De preliminära resultaten tyder på att CL-metoder som EWC Online och SI kan integreras i rörledningar för röntgendatainlärning för att minska katastrofal glömska i situationer där en viss nivå av sekventiell utbildningsförmåga motiverar den betydande beräkningskostnaden.

The field of machine learning currently draws massive attention due to ad- vancements and successful applications announced in the last few years. One of these applications is self-driving vehicles. A machine learning model can learn to drive through behavior cloning. Behavior cloning uses an expert’s behavioral traces as training data. However, the model’s steering predictions influence the succeeding input to the model and thus the model’s input data will vary depending on earlier predictions. Eventually the vehicle may de- viate from the expert’s behavioral traces and fail due to encountering data it has not been trained on. This is the problem of sequential predictions. DAG- GER and its improvement SafeDAGGER are algorithms that enable training models in the sequential prediction domain. Both algorithms iteratively col- lect new data, aggregate new and old data and retrain models on all data to avoid catastrophically forgetting previous knowledge. The aggregation of data leads to problems with increasing model training times, memory requirements and requires that previous data is maintained forever. This thesis’s purpose is investigate whether or not SafeDAGGER can be improved with continual learning to create a more scalable and flexible algorithm. This thesis presents an improved algorithm called EWC-SD that uses the continual learning algo- rithm EWC to protect a model’s previous knowledge and thereby only train on new data. Training only on new data allows EWC-SD to have lower training times, memory requirements and avoid storing old data forever compared to the original SafeDAGGER. The different algorithms are evaluated in the con- text of self-driving vehicles on three tracks in the VBS3 simulator. The results show EWC-SD when trained on new data only does not reach the performance of SafeDAGGER. Adding a rehearsal buffer containing only 23 training exam- ples to EWC-SD allows it to outperform SafeDAGGER by reaching the same performance in half as many iterations. The conclusion is that EWC-SD with rehearsal solves the problems of increasing model training times, memory re- quirements and requiring access to all previous data imposed by data aggre- gation. / Fältet för maskininlärning drar för närvarande massiv uppmärksamhet på grund av framsteg och framgångsrika applikationer som meddelats under de senaste åren. En av dessa applikationer är självkörande fordon. En maskininlärningsmodell kan lära sig att köra ett fordon genom beteendekloning. Beteendekloning använder en experts beteendespår som träningsdata. En modells styrförutsägelser påverkar emellertid efterföljande indata till modellen och således varierar modellens indata utifrån tidigare förutsägelser. Så småningom kan fordonet avvika från expertens beteendespår och misslyckas på grund av att modellen stöter på indata som den inte har tränats på. Det här är problemet med sekventiella förutsägelser. DAGGER och dess förbättring SafeDAGGER är algoritmer som möjliggör att träna modeller i domänen sekventiella förutsägelser. Båda algoritmerna samlar iterativt nya data, aggregerar nya och gamla data och tränar om modeller på alla data för att undvika att katastrofalt glömma tidigare kunskaper. Aggregeringen av data leder till problem med ökande träningstider, ökande minneskrav och kräver att man behåller åtkomst till all tidigare data för alltid. Avhandlingens syfte är att undersöka om SafeDAGGER kan förbättras med stegvis inlärning för att skapa en mer skalbar och flexibel algoritm. Avhandlingen presenterar en förbättrad algoritm som heter EWC-SD, som använder stegvis inlärningsalgoritmen EWC för att skydda en modells tidigare kunskaper och därigenom enbart träna på nya data. Att endast träna på nya data gör det möjligt för EWC-SD att ha lägre träningstider, ökande minneskrav och undvika att lagra gamla data för evigt jämfört med den ursprungliga SafeDAGGER. De olika algoritmerna utvärderas i kontexten självkörande fordon på tre banor i VBS3-simulatorn. Resultaten visar att EWC-SD tränad enbart på nya data inte uppnår prestanda likvärdig SafeDAGGER. Ifall en lägger till en repeteringsbuffert som innehåller enbart 23 träningsexemplar till EWC-SD kan den överträffa SafeDAGGER genom att uppnå likvärdig prestanda i hälften så många iterationer. Slutsatsen är att EWC-SD med repeteringsbuffert löser problemen med ökande träningstider, ökande minneskrav samt kravet att alla tidigare data ständigt är tillgängliga som påtvingas av dataaggregering.