Monitoring information technology (IT) systems during operation is one of the few methods that help administrators track the health of the monitored system, predict and detect faults, and assist in system repair and error prevention. However, current implementations impose architectural and functional constraints on monitored systems that result in less flexibility in deployment and operation. While excellent monitoring systems exist for some use cases, others are not adequately supported, having no monitoring system available at all for very specific use cases. In addition, most monitoring software specializes in specific data formats, protocols, data collection mechanisms, etc., further limiting its flexibility. As a result, individuals and organizations struggle to find the right combination of features to support their monitoring needs in a single monitoring system, forcing them to use multiple monitoring systems instead in order to support all of their use cases. The role-based approach to software modeling and implementation promises an intuitive way to increase flexibility in modeling and implementing IT systems. In conjunction with technology from the field of self-adaptive systems, this thesis describes a framework for context-sensitive control loops with roles that can be used to overcome these limitations. We present a novel approach to building a flexible role-based monitoring system based on that framework. Our approach allows for context-specific implementation of monitoring capabilities to support a variety of application domains, while maintaining a derived architecture of well-defined roleplaying components that inherently support distribution and scalability. To this end, important background knowledge from the areas of self-adaptive systems, control loops, the role concept, as well as role-based modeling and implementation is first presented. In addition, important related work from the areas of flexible system design and monitoring systems is presented. Then, a framework for context-sensitive control loops with roles is introduced and applied to the monitoring application domain in modeling and implementation. Based on a common use case for monitoring systems (i.e., monitoring and autoscaling of a web service infrastructure), the resulting Role-based Monitoring Approach (RBMA) is compared to two state-of-the-art monitoring toolkits. This is followed by a qualitative and quantitative evaluation of RBMA, showing that it is more flexible and, at the same time, provides reasonable performance at no additional cost compared to the state-of-the-art tools. Finally, it is explained how this thesis’ contributions can be applied to another monitoring use case (i.e., network device monitoring) as well as to another application domain (i.e., embedded systems monitoring) and its extension (i.e., the Internet of Things domain). This thesis concludes with a summary of the contributions and a presentation of important topics for future work.:Preface iv Statement of Authorship . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . v Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vi Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . vii Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . viii The RoSI Research Training Group . . . . . . . . . . . . . . . . . . . . . . . . . . x 1 Introduction 1 1.1 Thesis Topic . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1 1.2 Thesis Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1.3 Research Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 2 Background 5 2.1 Principles of Self-adaptation . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 2.1.1 The MAPE-K Control Loop . . . . . . . . . . . . . . . . . . . . . . . 7 2.1.2 MAPE-K Patterns for Distributed Self-adaptive Systems . . . . . . 12 2.1.3 MAPE-K Control Loop in Monitoring Systems . . . . . . . . . . . 16 2.2 The Notion of Roles . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 2.3 The Compartment Role Object Meta-Model . . . . . . . . . . . . . . . . . . 24 2.4 The ObjectTeams Java Programming Model . . . . . . . . . . . . . . . . . . 26 2.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 3 Related Work 31 3.1 Design Patterns for Flexibility in Software . . . . . . . . . . . . . . . . . . . 31 3.1.1 Strategy Pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 3.1.2 Template Method Pattern . . . . . . . . . . . . . . . . . . . . . . . . 34 3.1.3 Using Delegation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3.1.4 Role-object Pattern . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 3.2 Classifying Flexibility in Monitoring Systems . . . . . . . . . . . . . . . . . 39 3.2.1 Criteria for Flexibility in Monitoring Systems . . . . . . . . . . . . 40 3.2.2 Classification of Flexibility in Monitoring Systems . . . . . . . . . 44 3.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 4 The Role-based Monitoring Approach 47 4.1 Framework and Model for Context-sensitive Control Loops with Roles . 48 4.2 Evaluation Scenario: Autoscaling of Web Service Infrastructures . . . . . 54 4.2.1 Version 1: Role-based Monitoring Approach . . . . . . . . . . . . . 59 4.2.2 Version 2: Prometheus with Alertmanager . . . . . . . . . . . . . . 70 4.2.3 Version 3: Elasticsearch with Kibana . . . . . . . . . . . . . . . . . . 73 iii Contents 4.3 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76 5 Evaluation 77 5.1 Quantitative Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 5.1.1 First Experiment (Correct Functionality) . . . . . . . . . . . . . . . 78 5.1.2 Second Experiment (Idle Performance) . . . . . . . . . . . . . . . . 80 5.1.3 Third Experiment (Performance under Load) . . . . . . . . . . . . 80 5.2 Qualitative Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 5.3 Additional Use Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.3.1 Monitoring Network Devices . . . . . . . . . . . . . . . . . . . . . . 87 5.3.2 Flexible Embedded Systems Management . . . . . . . . . . . . . . 90 5.3.3 Managing Internet of Things Devices . . . . . . . . . . . . . . . . . 92 6 Conclusion and Future Work 95 6.1 Summary of Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.2 Topics for Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 Bibliography 99 List of Figures 107 List of Tables 109 List of Listings 110 List of Abbreviations 111 A Implementation, Compilation, and Execution of RBMA 113 A.1 Implementation of Base Classes . . . . . . . . . . . . . . . . . . . . . . . . . 113 A.2 Implementation of Team- and inner Role Classes . . . . . . . . . . . . . . . 121 A.3 Implementation of Auxiliary Classes . . . . . . . . . . . . . . . . . . . . . . 139 A.4 Compilation of RBMA with Eclipse OT/J . . . . . . . . . . . . . . . . . . . 144 A.5 Execution of RBMA . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 B Additional Information: Autoscaling of Web Service Infrastructures 145 B.1 Setup of the Slave-level Clusters (Versions 1, 2, and 3) . . . . . . . . . . . . 145 B.2 RBMA: Setup of the Master-level Cluster (Version 1) . . . . . . . . . . . . 156 B.3 Prometheus: Setup of Master-level Cluster (Version 2) . . . . . . . . . . . 160 B.4 Elastic Stack: Setup of the Master-level Cluster (Version 3) . . . . . . . . . 165 B.5 Auxiliary Tutorials . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 174 C Large Figures 179

Engineering contemporary distributed software applications is a challenging task due to the dynamic operating conditions in which these systems have to function. Examples are dynamic availability of resources, errors that are difficult to predict, and changing user requirements. These dynamics can affect a number of quality concerns of a system, such as robustness, openness, and performance. The challenges of engineering software systems with such dynamics have motivated the need for self-adaptation. Self-adaptation is based on the principle of separation of concerns, distinguishing two well defined systems: a managed system that deals with domain specific concerns and a managing system that deals with particular quality concerns of the managed system through adaptation with a feedback loop. State of the art in self- adaptation advocates the use of formal methods to specify and verify the system's behavior in order to provide evidence that the system's goals are satisfied. However, little work has been done on the consolidation of design knowledge to model and verify self-adaptation behaviors. To support designers, this thesis contributes with a set of formally specified templates for the specification and verification of self-adaptive behaviors of a family of distributed self-adaptive systems. The templates are based on the MAPE-K reference model (Monitor-Analyze-Plan-Execute plus Knowledge). The templates comprise: (1) behavior specification patterns for modeling the different MAPE components of a feedback loop, and (2) property specification patterns that support verification of the correctness of the adaptation behaviors. The target domain are distributed applications in which self-adaptation is used for managing resources for robustness and openness requirements. The templates are derived from expertise with developing several self-adaptive systems, including a collaborative mobile learning application in which we have applied self-adaptation to make the system robust to degrading GPS accuracy, and a robotic system in which we apply self-adaptation to support different types of openness requirements. We demonstrate the reusability of the templates in a number of case studies. / AMULETS

Communication network technologies have been evolving exponentially in the late decades. These innovations increase the network capabilities and open new horizons to creating novel and original services. The heterogeneity in equipment qualifications increases the level of complexity in the technological advancement. In such environment, service management has become an everyday challenge to service providers. Important efforts have been deployed to innovate in the exploitation of intelligent devices in the home and other private locations. In this Thesis, we propose a framework for self-configuration of devices within Hone Area Networks. We propose a device self-configuration architecture based on IBM Monitor-Analyze-Plan-Execute using Knowledge autonomic control loop. To prove the validity of our system architecture and support its applicability, we developed a prototype system that gives a general control loop implementation for device self-configuration using the CPE WAN Management Protocol. A video streaming scenario is implemented and used to evaluate validity our framework.

Communication network technologies have been evolving exponentially in the late decades. These innovations increase the network capabilities and open new horizons to creating novel and original services. The heterogeneity in equipment qualifications increases the level of complexity in the technological advancement. In such environment, service management has become an everyday challenge to service providers. Important efforts have been deployed to innovate in the exploitation of intelligent devices in the home and other private locations. In this Thesis, we propose a framework for self-configuration of devices within Hone Area Networks. We propose a device self-configuration architecture based on IBM Monitor-Analyze-Plan-Execute using Knowledge autonomic control loop. To prove the validity of our system architecture and support its applicability, we developed a prototype system that gives a general control loop implementation for device self-configuration using the CPE WAN Management Protocol. A video streaming scenario is implemented and used to evaluate validity our framework.

Communication network technologies have been evolving exponentially in the late decades. These innovations increase the network capabilities and open new horizons to creating novel and original services. The heterogeneity in equipment qualifications increases the level of complexity in the technological advancement. In such environment, service management has become an everyday challenge to service providers. Important efforts have been deployed to innovate in the exploitation of intelligent devices in the home and other private locations. In this Thesis, we propose a framework for self-configuration of devices within Hone Area Networks. We propose a device self-configuration architecture based on IBM Monitor-Analyze-Plan-Execute using Knowledge autonomic control loop. To prove the validity of our system architecture and support its applicability, we developed a prototype system that gives a general control loop implementation for device self-configuration using the CPE WAN Management Protocol. A video streaming scenario is implemented and used to evaluate validity our framework.

Communication network technologies have been evolving exponentially in the late decades. These innovations increase the network capabilities and open new horizons to creating novel and original services. The heterogeneity in equipment qualifications increases the level of complexity in the technological advancement. In such environment, service management has become an everyday challenge to service providers. Important efforts have been deployed to innovate in the exploitation of intelligent devices in the home and other private locations. In this Thesis, we propose a framework for self-configuration of devices within Hone Area Networks. We propose a device self-configuration architecture based on IBM Monitor-Analyze-Plan-Execute using Knowledge autonomic control loop. To prove the validity of our system architecture and support its applicability, we developed a prototype system that gives a general control loop implementation for device self-configuration using the CPE WAN Management Protocol. A video streaming scenario is implemented and used to evaluate validity our framework.

Les opérateurs de cloud IaaS (Infrastructure as a Service) proposent à leurs clients des ressources virtualisées (CPU, stockage et réseau) sous forme de machines virtuelles (VM). L’explosion du marché du cloud les a contraints à optimiser très finement l’utilisation de leurs centres de données afin de proposer des services attractifs à moindre coût. En plus des investissements liés à l’achat des infrastructures et de leur coût d’utilisation, la consommation énergétique apparaît comme un point de dépense important (2% de la consommation mondiale) et en constante augmentation. Sa maîtrise représente pour ces opérateurs un levier très intéressant à exploiter. D’un point de vue technique, le contrôle de la consommation énergétique s’appuie essentiellement sur les méthodes de consolidation. Or la plupart d'entre elles ne prennent en compte que l’utilisation CPU des machines physiques (PM) pour le placement de VM. En effet, des études récentes ont montré que les systèmes de stockage et les E/S disque constituent une part considérable de la consommation énergétique d’un centre de données (entre 14% et 40%). Dans cette thèse nous introduisons un nouveau modèle autonomique d’optimisation de placement de VM inspiré de MAPE-K (Monitor, Analyze, Plan, Execute, Knowledge), et prenant en compte en plus du CPU, les E/S des VM ainsi que les systèmes de stockage associés. Ainsi, notre première contribution est relative au développement d’un outil de trace des E/S de VM multi-niveaux. Les traces collectées alimentent, dans l’étape Analyze, un modèle de coût étendu dont l’originalité consiste à prendre en compte le profil d’accès des VM, les caractéristiques du système de stockage, ainsi que les contraintes économiques de l’environnement cloud. Nous analysons par ailleurs les caractéristiques des deux principales classes de stockage, pour aboutir à un modèle hybride exploitant au mieux les avantages de chacune. En effet, les disques durs magnétiques (HDD) sont des supports de stockage à la fois énergivores et peu performants comparés aux unités de calcul. Néanmoins, leur prix par gigaoctet et leur longévité peuvent jouer en leur faveur. Contrairement aux HDD, les disques SSD à base de mémoire flash sont plus performants et consomment peu d’énergie. Leur prix élevé par gigaoctet et leur courte durée de vie (comparés aux HDD) représentent leurs contraintes majeures. L’étape Plan a donné lieu, d’une part, à une extension de l'outil de simulation CloudSim pour la prise en compte des E/S des VM, du caractère hybride du système de stockage, ainsi que la mise en oeuvre du modèle de coût proposé dans l'étape Analyze. Nous avons proposé d’autre part, plusieurs heuristiques se basant sur notre modèle de coût et que nous avons intégrées dans CloudSim. Nous montrons finalement que notre approche permet d’améliorer d’un facteur trois le coût de placement de VM obtenu par les approches existantes. / IaaS cloud providers offer virtualized resources (CPU, storage, and network) as Virtual Machines(VM). The growth and highly competitive nature of this economy has compelled them to optimize the use of their data centers, in order to offer attractive services at a lower cost. In addition to investments related to infrastructure purchase and cost of use, energy efficiency is a major point of expenditure (2% of world consumption) and is constantly increasing. Its control represents a vital opportunity. From a technical point of view, the control of energy consumption is mainly based on consolidation approaches. These approaches, which exclusively take into account the CPU use of physical machines (PM) for the VM placement, present however many drawbacks. Indeed, recent studies have shown that storage systems and disk I/O represent a significant part of the data center energy consumption (between 14% and 40%).In this thesis we propose a new autonomic model for VM placement optimization based on MAPEK (Monitor, Analyze, Plan, Execute, Knowledge) whereby in addition to CPU, VM I/O and related storage systems are considered. Our first contribution proposes a multilevel VM I/O tracer which overcomes the limitations of existing I/O monitoring tools. In the Analyze step, the collected I/O traces are introduced in a cost model which takes into account the VM I/O profile, the storage system characteristics, and the cloud environment constraints. We also analyze the complementarity between the two main storage classes, resulting in a hybrid storage model exploiting the advantages of each. Indeed, Hard Disk Drives (HDD) represent energy-intensive and inefficient devices compared to compute units. However, their low cost per gigabyte and their long lifetime may constitute positive arguments. Unlike HDD, flash-based Solid-State Disks (SSD) are more efficient and consume less power, but their high cost per gigabyte and their short lifetime (compared to HDD) represent major constraints. The Plan phase has initially resulted in an extension of CloudSim to take into account VM I/O, the hybrid nature of the storage system, as well as the implementation of the previously proposed cost model. Secondly, we proposed several heuristics based on our cost model, integrated and evaluated using CloudSim. Finally, we showed that our contribution improves existing approaches of VM placement optimization by a factor of three.

Modern software systems are increasingly characterized by uncertainties in the operating context and user requirements. These uncertainties are difficult to predict at design time. Achieving the quality goals of such systems depends on the ability of the software to deal with these uncertainties at runtime. A self-adaptive system employs a feedback loop to continuously monitor and adapt itself to achieve particular quality goals (i.e., adaptation goals) regardless of uncertainties. Current research applies formal techniques to provide guarantees for adaptation goals, typically using exhaustive verification techniques. Although these techniques offer strong guarantees for the goals, they suffer from well-known state explosion problem. In this thesis, we take a broader perspective and focus on two types of guarantees: (1) functional correctness of the feedback loop, and (2) guaranteeing the adaptation goals in an efficient manner. To that end, we present ActivFORMS (Active FORmal Models for Self-adaptation), a formally founded model-driven approach for engineering self-adaptive systems with guarantees. ActivFORMS achieves functional correctness by direct execution of formally verified models of the feedback loop using a reusable virtual machine. To efficiently provide guarantees for the adaptation goals with a required level of confidence, ActivFORMS applies statistical model checking at runtime. ActivFORMS supports on the fly changes of adaptation goals and updates of the verified feedback loop models that meet the changed goals. To demonstrate the applicability and effectiveness of the approach, we applied ActivFORMS in several domains: warehouse transportation, oceanic surveillance, tele assistance, and IoT building security monitoring. / Marie Curie CIG, FP7-PEOPLE-2011-CIG, Project ID: 303791

Self-adaptive systems are capable of autonomously adjusting their behavior at runtime to accomplish particular adaptation goals. The most common way to realize self-adaption is using a feedback loop(s) which contains four actions: collect runtime data from the system and its environment, analyze the collected data, decide if an adaptation plan is required, and act according to the adaptation plan for achieving the adaptation goals. Existing approaches achieve the adaptation goals by using formal methods, and exhaustively verify all the available adaptation options, i.e., adaptation space. However, verifying the entire adaptation space is often not feasible since it requires time and resources. In this thesis, we present an approach which uses machine learning to reduce the adaptation space in self-adaptive systems. The approach integrates with the feedback loop and selects a subset of the adaptation options that are valid in the current situation. The approach is applied on the simulator of a self-adaptive Internet of Things application which is deployed in KU Leuven, Belgium. We compare our results with a formal model based self-adaptation approach called ActivFORMS. The results show that on average the adaptation space is reduced by 81.2% and the adaptation time by 85% compared to ActivFORMS while achieving the same quality guarantees.

Self-adaptive systems have limited time to adjust their configurations whenever their adaptation goals, i.e., quality requirements, are violated due to some runtime uncertainties. Within the available time, they need to analyze their adaptation space, i.e., a set of configurations, to find the best adaptation option, i.e., configuration, that can achieve their adaptation goals. Existing formal analysis approaches find the best adaptation option by analyzing the entire adaptation space. However, exhaustive analysis requires time and resources and is therefore only efficient when the adaptation space is small. The size of the adaptation space is often in hundreds or thousands, which makes formal analysis approaches inefficient in large-scale self-adaptive systems. In this thesis, we tackle this problem by presenting an online learning approach that enables formal analysis approaches to analyze large adaptation spaces efficiently. The approach integrates with the standard feedback loop and reduces the adaptation space to a subset of adaptation options that are relevant to the current runtime uncertainties. The subset is then analyzed by the formal analysis approaches, which allows them to complete the analysis faster and efficiently within the available time. We evaluate our approach on two different instances of an Internet of Things application. The evaluation shows that our approach dramatically reduces the adaptation space and analysis time without compromising the adaptation goals.