Cette étude s’inscrit dans le domaine de l’intelligence artificielle, plus précisément au croisement des deux domaines que sont la planification autonome en environnement probabiliste et la vérification formelle probabiliste. Dans ce contexte, elle pose la question de la maîtrise de la complexité face à l’intégration de nouvelles technologies dans les systèmes critiques : comment garantir que l’ajout d’une intelligence à un système, sous la forme d’une autonomie, ne se fasse pas au détriment de la sécurité ? Pour répondre à cette problématique, cette étude a pour enjeu de développer un processus outillé, permettant de concevoir des systèmes auto-adaptatifs critiques, ce qui met en œuvre à la fois des méthodes de modélisation formelle des connaissances d’ingénierie, ainsi que des algorithmes de planification sûre et optimale des décisions du système. / This study takes place in the broad field of Artificial Intelligence, specifically at the intersection of two domains : Automated Planning and Formal Verification in probabilistic environment. In this context, it raises the question of the integration of new technologies in critical systems, and the complexity it entails : How to ensure that adding intelligence to a system, in the form of autonomy, is not done at the expense of safety ? To address this issue, this study aims to develop a tool-supported process for designing critical, self-adaptive systems. Throughout this document, innovations are therefore proposed in methods of formal modeling and in algorithms for safe and optimal planning.

Au cours de la dernière décennie, la complexité des applications a considérablement évolué afin de répondre aux besoins métiers émergeants. Leur conception implique une composition distribuée de composants logiciels. Ces applications fournissent des services à travers les interactions métiers maintenues par leurs composants. De telles applications sont intrinsèquement en évolution dynamique en raison de la dynamicité de leurs contextes. En effet, elles évoluent dans des environnements qui changent tout en présentant des conditions très dynamiques durant leur cycle de vie d’exécution. De tels contextes représentent une lourde charge pour les développeurs aussi bien pour leurs tâches de conception que de gestion. Cela a motivé́ le besoin de renforcer l’autonomie de gestion des applications pour les rendre moins dépendantes de l’intervention humaine en utilisant les principes de l’Informatique Autonomique. Les Systèmes Informatiques Autonomes (SIA) impliquent l’utilisation des boucles autonomiques, dédiées aux systèmes afin de les aider à accomplir leurs tâches de gestion. Ces boucles ont pour objectif d’adapter leurs systèmes à la dynamicité de leurs contextes, en se basant sur une logique d’adaptation intégrée. Cette logique est souvent donnée par des règles statiques codées manuellement. La construction de ces règles demande beaucoup de temps tout en exigeant une bonne expertise. En fait, elles nécessitent une compréhension approfondie de la dynamicité du système afin de prédire les adaptations précises à apporter à celui-ci. Par ailleurs, une telle logique ne peut envisager tous les scénarios d’adaptation possibles, donc, ne sera pas en mesure de prendre en compte des adaptations pour des situations précédemment inconnues. Les SIA devraient donc être assez sophistiqués afin de pouvoir faire face à la nature dynamique de leurs contextes et de pouvoir apprendre par eux-mêmes afin d’agir correctement dans des situations inconnues. Les SIA devraient également être capables d’apprendre de leur propre expérience passée afin de modifier leur logique d’adaptation en fonction de la dynamicité de leurs contextes. Dans ce manuscrit, nous abordons les lacunes décrites en utilisant les techniques d’Apprentissage par Renforcement (AR) afin de construire notre logique d’adaptation. Cependant, les approches fondées sur l’AR sont connues pour leur mauvaise performance lors des premières phases d’apprentissage. Cette mauvaise performance entrave leur utilisation dans le monde réel des systèmes déployés. Par conséquent, nous avons amélioré cette logique d’adaptation avec des capacités d’apprentissage plus performantes avec une approche AR en multi-pas. Notre objectif est d’optimiser la performance de l’apprentissage et de le rendre plus efficace et plus rapide, en particulier durant les premières phases d’apprentissage. Nous avons aussi proposé́ un cadriciel générique visant à aider les développeurs dans la construction d’applications auto-adaptatives. Nous avons donc proposé de transformer des applications existantes en ajoutant des capacités d’autonomie et d’apprentissage à leurs composants. La transformation consiste en l’encapsulation des composants dans des conteneurs autonomiques pour les doter du comportement auto-adaptatif nécessaire. Notre objectif est d’alléger la charge des tâches de gestion des développeurs et de leur permettre de se concentrer plus sur la logique métier de leurs applications. Les solutions proposées sont destinées à être génériques, granulaires et basées sur un standard connu, à savoir l’Architecture de Composant de Service. Enfin, nos propositions ont été évaluées et validées avec des résultats expérimentaux. Ils ont démontré leur efficacité en montrant un ajustement dynamique des applications transformées face aux dynamicités de leurs contextes en un temps beaucoup plus court comparé aux approches existantes / During the past decade, the complexity of applications has significantly scaled to satisfy the emerging business needs. Their design entails a composition of distributed and interacting software components. They provide services by means of the business interactions maintained by their components. Such applications are inherently in a dynamic evolution due to their context dynamics. Indeed, they evolve in changing environments while exhibiting highly dynamic conditions during their execution life-cycle (e.g., their load, availability, performance, etc.). Such contexts have burdened the applications developers with their design and management tasks. Subsequently, motivated the need to enforce the autonomy of their management to be less dependent on human interventions with the Autonomic Computing principles. Autonomic Computing Systems (ACS) implies the usage of autonomic loops, dedicated to help the system to achieve its management tasks. These loops main role is to adapt their associated systems to the dynamic of their contexts by acting upon an embedded adaptation logic. Most of time, this logic is given by static hand-coded rules, often concern-specific and potentially error-prone. It is undoubtedly time and effort-consuming while demanding a costly expertise. Actually, it requires a thorough understanding of the system design and dynamics to predict the accurate adaptations to bring to the system. Furthermore, such logic cannot envisage all the possible adaptation scenarios, hence, not able to take appropriate adaptations for previously unknown situations. ACS should be sophisticated enough to cope with the dynamic nature of their contexts and be able to learn on their own to properly act in unknown situations. They should also be able to learn from their past experiences and modify their adaptation logic according to their context dynamics. In this thesis manuscript, we address the described shortcomings by using Reinforcement Learning (RL) techniques to build our adaptation logic. Nevertheless, RL-based approaches are known for their poor performance during the early stages of learning. This poor performance hinders their usage in real-world deployed systems. Accordingly, we enhanced the adaptation logic with sophisticated and better-performing learning abilities with a multi-step RL approach. Our main objective is to optimize the learning performance and render it timely-efficient which considerably improves the ACS performance even during the beginning of learning phase. Thereafter, we pushed further our work by proposing a generic framework aimed to support the application developers in building self-adaptive applications. We proposed to transform existing applications by dynamically adding autonomic and learning abilities to their components. The transformation entails the encapsulation of components into autonomic containers to provide them with the needed self-adaptive behavior. The objective is to alleviate the burden of management tasks on the developers and let them focus on the business logic of their applications. The proposed solutions are intended to be generic, granular and based on a well known standard (i.e., Service Component Architecture). Finally, our proposals were evaluated and validated with experimental results. They demonstrated their effectiveness by showing a dynamic adjustment to the transformed application to its context changes in a shorter time as compared to existing approaches

Autonomy is the most demanded yet hard-to-achieve feature of recent and future software systems. Self-driving cars, mail-delivering drones, automated guided vehicles in production sites, and housekeeping robots need to decide autonomously during most of their operation time. As soon as human intervention becomes necessary, the cost of ownership increases, and this must be avoided. Although the algorithms controlling autonomous systems become more and more intelligent, their hardest opponent is their inflexibility. The more environmental situations such a system is confronted with, the more complexity the control of the autonomous system will have to master. To cope with this challenge, engineers have approached a system design, which adopts feedback loops from nature. The resulting architectural principle, which they call self-adaptive systems, follows the idea of iteratively gathering sensor data, analyzing it, planning new adaptations of the system, and finally executing the plan. Often, adaptation means to alter the system setup, re-wire components, or even exchange control algorithms to keep meeting goals and requirements in the newly appeared situation. Although self-adaptivity helps engineers to organize the vast amount of information in a self-deciding system, it remains hard to deal with the variety of contexts, which involve both environmental influences and knowledge about the system\'s internals. This challenge not only holds for the construction phase but also for verification and validation, including software test. To assure sufficient quality of a system, it must be tested under an enormous and, thus, unmanageable, number of different contextual situations and manual test-cases. This thesis proposes a novel set of methods and model types, which help test engineers to specify precisely what they expect from a self-adaptive system under test. The formal nature of the introduced artifacts allows for automatically generating test-suites or running simulations in the loop so that a qualitative verdict on the system\'s correctness can be gained. Additional to these conceptional contributions, the thesis describes a model-based adaptivity test environment, which test engineers can use for testing actual self-adaptive systems. The implementation includes comprehensive tooling for creating the introduced types of models, generating test-cases, simulating them in the loop, automating tests, and reporting. Composing all enabling components for these tasks constitutes a reference architecture of integrated test environments for self-adaptive systems. We demonstrate the completeness and accuracy of the technical approach together with the underlying concepts by evaluating them in an experimental case study where an autonomous robot interacts with human co-workers. In summary, this thesis proposes concepts for automatically and, thus, efficiently testing self-adaptive systems. The quality, which is fostered by this novel approach, is resilience: the ability of a system to maintain its promises while facing changing environments.:1 Introduction 1 1.1 Problem Description 1 1.2 Overview of Adopted Methods 3 1.3 Hypothesis and Main Contributions 4 1.4 Organization of This Thesis 5 I Foundations 7 2 Background 9 2.1 Self-adaptive Software and Autonomic Computing 9 2.1.1 Common Principles and Components of SAS 10 2.1.2 Concrete Implementations and Applications of SAS 12 2.2 Model-based Testing 13 2.2.1 Testing for Dependability 14 2.2.2 The Basics of Testing 15 2.2.3 Automated Test Design 18 2.3 Dynamic Variability Management 22 2.3.1 Software Product Lines 23 2.3.2 Dynamic Software Product Lines 25 3 Related Work: Existing Research on Testing Self-Adaptive Systems 29 3.1 Testing Context-Aware Applications 30 3.2 The SimSOTA Project 31 3.3 Dynamic Variability in Complex Adaptive Systems (DiVA) 33 3.4 Other Early-Stage Research 34 3.5 Taxonomy of Requirements of Model-based SAS Testing 36 II Methods 39 4 Model-driven SAS Testing 41 4.1 Problem/Solution Fit 41 4.2 Example: Surveillance Drone 43 4.3 Concepts and Models for Testing Self-Adaptive Systems 44 4.3.1 Test Case Generation vs. Simulation in the Loop 44 4.3.2 Incremental Modeling Process 45 4.3.3 Basic Representation Format: Petri Nets 46 4.3.4 Context Variation 50 4.3.5 Modeling Adaptive Behavior 53 4.3.6 Dynamic Context Change 57 4.3.7 Interfacing Context from Behavioral Representation 62 4.3.8 Adaptation Mode Variation 64 4.3.9 Context-Dependent Recon guration 67 4.4 Adequacy Criteria for SAS Test Models 71 4.5 Discussion on the Viability of the Employed Models 71 4.6 Comparison to Related Work 73 4.7 Summary and Discussion 74 5 Model-based Adaptivity Test Environment 75 5.1 Technological Foundation 76 5.2 MATE Base Components 77 5.3 Metamodel Implementation 78 5.3.1 Feature-based Variability Model 79 5.3.2 Abstract and Concrete Syntax for Textual Notations 80 5.3.3 Adaptive Petri Nets 86 5.3.4 Stimulus and Recon guration Automata 87 5.3.5 Test Suite and Report Model 87 5.4 Test Generation Framework 87 5.5 Test Automation Framework 91 5.6 MATE Tooling and the SAS Test Process 93 5.6.1 Test Modeling 94 5.6.2 Test Case Generation 95 5.6.3 Test Case Execution and Test Reporting 96 5.6.4 Interactive Simulation Frontend 96 5.7 Summary and Discussion 97 III Evaluation 99 6 Experimental Study: Self-Adaptive Co-Working Robots 101 6.1 Robot Teaching and Co-Working with WEIR 103 6.1.1 WEIR Hardware Components 104 6.1.2 WEIR Software Infrastructure 105 6.1.3 KUKA LBR iiwa as WEIR Manipulator 106 6.1.4 Self-Adaptation Capabilities of WEIR 107 6.2 Cinderella as Testable Co-Working Application 109 6.2.1 Cinderella Setup and Basic Functionality 109 6.2.2 Co-Working with Cinderella 110 6.3 Testing Cinderella with MATE 112 6.3.1 Automating Test Execution 112 6.3.2 Modeling Cinderella in MATE 113 6.3.3 Testing Cinderella in the Loop 121 6.4 Evaluation Verdict and Summary 123 7 Summary and Discussion 125 7.1 Summary of Contributions 126 7.2 Open Research Questions 127 Bibliography 129 Appendices 137 Appendix Cinderella De nitions 139 1 Cinderella Adaptation Bounds 139 2 Cinderella Self-adaptive Workflow 140

[pt] Nos últimos anos, foi desenvolvido um número significativo de sistemas autoadaptativos (i.e.: sistemas capazes de saber o que está acontecendo sobre si mesmo e que, consequentemente, implementam parcialmente a qualidade de consciência). A literatura tem pesquisado extensivamente o uso da engenharia de requisitos orientada a metas e o uso da arquitetura de referência MAPE (Monitor-Analyze-Plan-Execute) para o desenvolvimento de sistemas autoadaptativos. Entretanto, construir tais sistemas com base em estratégias de referência não é trivial, podendo resultar em problemas estruturais que impactam negativamente alguns atributos de qualidade do produto final (e.g.: reusabilidade, modularidade, modificabilidade e entendibilidade). Neste contexto, estratégias de reengenharia para a reorganização de tais sistemas são pouco exploradas, limitando-se a recuperar e a reestruturar a lógica da adaptação em modelos de baixo nível. Esta prática mantém a dificuldade do tratamento da qualidade de consciência como um requisito não funcional (RNF) de primeira classe, impactando diretamente na seleção da arquite-tura e implementação do sistema. Nossa pesquisa visa mitigar esse problema atra-vés de uma estratégia de reengenharia de sistemas autoadaptativos, centrada no RNF de consciência de software, com vistas a auxiliar na remoção de alguns problemas recorrentes na implementação do MAPE conforme a literatura. A estratégia de reengenharia está organizada em quatro subprocessos: (A) recuperar a intencio-nalidade do sistema com ênfase em suas metas de consciência, gerando um modelo de metas AS-IS; (B) especificar o modelo de metas TO-BE reutilizando um conjunto de SRconstructs para operacionalizar o RNF de consciência de software conforme o padrão MAPE; (C) redesenhar o sistema revisando as operacionalizações de consciência e selecionando as tecnologias para implementar o MAPE, e; (D) finalmente, reimplementar o sistema conforme nova estrutura, adicionando metainformações de código para manter a rastreabilidade para o mecanismo de autoadaptação visando facilitar novas evoluções. O escopo da nossa pesquisa são sistemas autoadaptativos orientados a objetos (OO), utilizando o framework i como linguagem para os modelos orientados a metas. Nossos resultados de avaliações em sistemas auto-adaptativos OO desenvolvidos em Java para dispositivos móveis com Android demonstram que a estratégia auxilia no realinhamento do sistema com as boas práticas recomendadas pela literatura facilitando futuras evoluções. / [en] In recent years, a significant number of self-adaptive systems (i.e.: systems capable of knowing what is happening about themselves, and consequently partially implementing the quality of awareness) have been developed. The literature has extensively researched the use of goal oriented requirements engineering and the use of the MAPE (Monitor-Analyze-Plan-Execute) reference architecture for the development of self-adaptive systems. However, building such systems based on reference strategies is not trivial, it can result in structural problems that negatively impact some quality attributes of the final product (e.g.: reusability, modularity, modifiability and understandability). In this context, reengineering strategies for the reorganization of such systems are poor explored, and they are limited to recovering and restructuring the logic of adaptation in low-level models. This approach keeps the difficulty of treating the awareness quality as a first-class non-functional re-quirement (NFR) directly affecting architecture selection and implementation of the system. Our research aims to mitigate this problem through a strategy of reengi-neering self-adaptive systems, centered on software awareness as an NFR. This strategy will assist in the removal of some recurring problems in the implementation of MAPE according to the literature. The reengineering strategy is organized into four sub-processes: (A) recover the intentionality of the system with an emphasis on its awareness goals, generating an AS-IS goal model; (B) specify the TO-BE goal model by reusing a set of SRconstructs to operationalize the software awareness NFR according to the MAPE standard; (C) redesign the system by reviewing the operationalizations of awareness and selecting the technologies to implement the MAPE, and; (D) finally, reimplement the system according to a new structure, add-ing code metadata to maintain traceability for the self-adaptation mechanism in or-der to facilitate new evolutions. The scope of our research is object-oriented (OO) self-adaptive systems using the i framework as a language for goal-oriented models. Our results of evaluations, for OO self-adaptive systems developed in Java for mobile devices with Android, show that the strategy helps in realigning the system with the best practices recommended by the, facilitating future developments.

Depuis son introduction en 1982, la carte auto-organisatrice de Kohonen (Self-Organizing Map : SOM) a prouvé ses capacités de classification et visualisation des données multidimensionnelles dans différents domaines d’application. Les implémentations matérielles de la carte SOM, en exploitant le taux de parallélisme élevé de l’algorithme de Kohonen, permettent d’augmenter les performances de ce modèle neuronal souvent au détriment de la flexibilité. D’autre part, la flexibilité est offerte par les implémentations logicielles qui quant à elles ne sont pas adaptées pour les applications temps réel à cause de leurs performances temporelles limitées. Dans cette thèse nous avons proposé une architecture matérielle distribuée, adaptable, flexible et extensible de la carte SOM à base de NoC dédiée pour une implantation matérielle sur FPGA. A base de cette approche, nous avons également proposé une architecture matérielle innovante d’une carte SOM à structure croissante au cours de la phase d’apprentissage / Since its introduction in 1982, Kohonen’s Self-Organizing Map (SOM) showed its ability to classify and visualize multidimensional data in various application fields. Hardware implementations of SOM, by exploiting the inherent parallelism of the Kohonen algorithm, allow to increase the overall performances of this neuronal network, often at the expense of the flexibility. On the other hand, the flexibility is offered by software implementations which on their side are not suited for real-time applications due to the limited time performances. In this thesis we proposed a distributed, adaptable, flexible and scalable hardware architecture of SOM based on Network-on-Chip (NoC) designed for FPGA implementation. Moreover, based on this approach we also proposed a novel hardware architecture of a growing SOM able to evolve its own structure during the learning phase

A (self-) adaptive system is a system that can dynamically adapt its behavior or structure during execution to "adapt" to changes to its environment or the system itself. From a security standpoint, there has been some research pertaining to (self-) adaptive systems in general but not enough care has been shown towards the adaptation itself. Security of systems can be reasoned about using threat models to discover security issues in the system. Essentially that entails abstracting away details not relevant to the security of the system in order to focus on the important aspects related to security. Threat models often enable us to reason about the security of a system quantitatively using security metrics. The structural adaptation process of a (self-) adaptive system occurs based on a reconfiguration plan, a set of steps to follow from the initial state (configuration) to the final state. Usually, the reconfiguration plan consists of multiple strategies for the structural adaptation process and each strategy consists of several steps steps with each step representing a specific configuration of the (self-) adaptive system. Different reconfiguration strategies have different security levels as each strategy consists of a different sequence configuration with different security levels. To the best of our knowledge, there exist no approaches which aim to guide the reconfiguration process in order to select the most secure available reconfiguration strategy, and the explicit security of the issues associated with the structural reconfiguration process itself has not been studied. In this work, based on an in-depth literature survey, we aim to propose several metrics to measure the security of configurations, reconfiguration strategies and reconfiguration plans based on graph-based threat models. Additionally, we have implemented a prototype to demonstrate our approach and automate the process. Finally, we have evaluated our approach based on a case study of our making. The preliminary results tend to expose certain security issues during the structural adaptation process and exhibit the effectiveness of our proposed metrics.

Adaptation enables a reaction to a changing environment. For traditional software development, that means changing the design and implementation of the software in a potentially complex and expensive process. If requirements are not known until the runtime of a software system, this system must be able to cope with changes during its runtime. For this, self-adaptive systems (SAS) were created. They have internal knowledge about themselves and their environment to reason about changes and take appropriate actions. Many approaches aiming to build such systems have been published since the start of the research area at the beginning of the 21st century. However, it is difficult to find an appropriate approach, even when all requirements of a scenario the system should be built for are known. If no suitable approach can be found, software developers have to built a new system leading to high development costs and potentially inefficient solutions due to the complexity of the system. This thesis follows two goals: (1) To make approaches building SAS more comparable through a feature model describing features of SAS, and (2) to provide a novel way of specifying SAS concisely using reference attribute grammars (RAGs) providing efficient systems. RAGs originate from the research field of compiler construction and enable the concise description of parts of the internal knowledge mentioned above as well as of the computation of the actions to cope with recognised changes. To make RAGs fully usable, this thesis presents two extensions: Relational RAGs enable the efficient handling of relations required for knowledge graphs, and Connected RAGs let RAG-based system communicate with other external systems to both recognise changes and execute actions. To evaluate the novel approaches, a classification of 30 approaches for the feature model and several case studies in the areas smart home, robotics, and system orchestration were conducted. It can be shown, that significantly less code is required to specify SAS. To specify the computation, 14.5 % to 28.7 % less code was required, whereas in another case study only 6.3 % of the total code was manually written and the rest was generated. The efficiency is similar to the best comparable approaches for graph queries. Furthermore, using additional optimizations (incremental evaluation), the execution time can be shown to be faster by a factor of 167.88 less albeit being sometimes by 50.0 % slower for very small workloads and specific queries. In a more realistic, extrapolated experiment, using incremental evaluation creates speed-up factors between 6.63 and 44.93. With the contributions in this thesis, existing approaches can be selected more precisely, new approaches can classify themselves within the research area, and the development of self-adaptive systems is possible using RAG-based systems.

Self-adaptive systems have limited time to adjust their configurations whenever their adaptation goals, i.e., quality requirements, are violated due to some runtime uncertainties. Within the available time, they need to analyze their adaptation space, i.e., a set of configurations, to find the best adaptation option, i.e., configuration, that can achieve their adaptation goals. Existing formal analysis approaches find the best adaptation option by analyzing the entire adaptation space. However, exhaustive analysis requires time and resources and is therefore only efficient when the adaptation space is small. The size of the adaptation space is often in hundreds or thousands, which makes formal analysis approaches inefficient in large-scale self-adaptive systems. In this thesis, we tackle this problem by presenting an online learning approach that enables formal analysis approaches to analyze large adaptation spaces efficiently. The approach integrates with the standard feedback loop and reduces the adaptation space to a subset of adaptation options that are relevant to the current runtime uncertainties. The subset is then analyzed by the formal analysis approaches, which allows them to complete the analysis faster and efficiently within the available time. We evaluate our approach on two different instances of an Internet of Things application. The evaluation shows that our approach dramatically reduces the adaptation space and analysis time without compromising the adaptation goals.