Autonomy is the most demanded yet hard-to-achieve feature of recent and future software systems. Self-driving cars, mail-delivering drones, automated guided vehicles in production sites, and housekeeping robots need to decide autonomously during most of their operation time. As soon as human intervention becomes necessary, the cost of ownership increases, and this must be avoided. Although the algorithms controlling autonomous systems become more and more intelligent, their hardest opponent is their inflexibility. The more environmental situations such a system is confronted with, the more complexity the control of the autonomous system will have to master. To cope with this challenge, engineers have approached a system design, which adopts feedback loops from nature. The resulting architectural principle, which they call self-adaptive systems, follows the idea of iteratively gathering sensor data, analyzing it, planning new adaptations of the system, and finally executing the plan. Often, adaptation means to alter the system setup, re-wire components, or even exchange control algorithms to keep meeting goals and requirements in the newly appeared situation. Although self-adaptivity helps engineers to organize the vast amount of information in a self-deciding system, it remains hard to deal with the variety of contexts, which involve both environmental influences and knowledge about the system\'s internals. This challenge not only holds for the construction phase but also for verification and validation, including software test. To assure sufficient quality of a system, it must be tested under an enormous and, thus, unmanageable, number of different contextual situations and manual test-cases. This thesis proposes a novel set of methods and model types, which help test engineers to specify precisely what they expect from a self-adaptive system under test. The formal nature of the introduced artifacts allows for automatically generating test-suites or running simulations in the loop so that a qualitative verdict on the system\'s correctness can be gained. Additional to these conceptional contributions, the thesis describes a model-based adaptivity test environment, which test engineers can use for testing actual self-adaptive systems. The implementation includes comprehensive tooling for creating the introduced types of models, generating test-cases, simulating them in the loop, automating tests, and reporting. Composing all enabling components for these tasks constitutes a reference architecture of integrated test environments for self-adaptive systems. We demonstrate the completeness and accuracy of the technical approach together with the underlying concepts by evaluating them in an experimental case study where an autonomous robot interacts with human co-workers. In summary, this thesis proposes concepts for automatically and, thus, efficiently testing self-adaptive systems. The quality, which is fostered by this novel approach, is resilience: the ability of a system to maintain its promises while facing changing environments.:1 Introduction 1 1.1 Problem Description 1 1.2 Overview of Adopted Methods 3 1.3 Hypothesis and Main Contributions 4 1.4 Organization of This Thesis 5 I Foundations 7 2 Background 9 2.1 Self-adaptive Software and Autonomic Computing 9 2.1.1 Common Principles and Components of SAS 10 2.1.2 Concrete Implementations and Applications of SAS 12 2.2 Model-based Testing 13 2.2.1 Testing for Dependability 14 2.2.2 The Basics of Testing 15 2.2.3 Automated Test Design 18 2.3 Dynamic Variability Management 22 2.3.1 Software Product Lines 23 2.3.2 Dynamic Software Product Lines 25 3 Related Work: Existing Research on Testing Self-Adaptive Systems 29 3.1 Testing Context-Aware Applications 30 3.2 The SimSOTA Project 31 3.3 Dynamic Variability in Complex Adaptive Systems (DiVA) 33 3.4 Other Early-Stage Research 34 3.5 Taxonomy of Requirements of Model-based SAS Testing 36 II Methods 39 4 Model-driven SAS Testing 41 4.1 Problem/Solution Fit 41 4.2 Example: Surveillance Drone 43 4.3 Concepts and Models for Testing Self-Adaptive Systems 44 4.3.1 Test Case Generation vs. Simulation in the Loop 44 4.3.2 Incremental Modeling Process 45 4.3.3 Basic Representation Format: Petri Nets 46 4.3.4 Context Variation 50 4.3.5 Modeling Adaptive Behavior 53 4.3.6 Dynamic Context Change 57 4.3.7 Interfacing Context from Behavioral Representation 62 4.3.8 Adaptation Mode Variation 64 4.3.9 Context-Dependent Recon guration 67 4.4 Adequacy Criteria for SAS Test Models 71 4.5 Discussion on the Viability of the Employed Models 71 4.6 Comparison to Related Work 73 4.7 Summary and Discussion 74 5 Model-based Adaptivity Test Environment 75 5.1 Technological Foundation 76 5.2 MATE Base Components 77 5.3 Metamodel Implementation 78 5.3.1 Feature-based Variability Model 79 5.3.2 Abstract and Concrete Syntax for Textual Notations 80 5.3.3 Adaptive Petri Nets 86 5.3.4 Stimulus and Recon guration Automata 87 5.3.5 Test Suite and Report Model 87 5.4 Test Generation Framework 87 5.5 Test Automation Framework 91 5.6 MATE Tooling and the SAS Test Process 93 5.6.1 Test Modeling 94 5.6.2 Test Case Generation 95 5.6.3 Test Case Execution and Test Reporting 96 5.6.4 Interactive Simulation Frontend 96 5.7 Summary and Discussion 97 III Evaluation 99 6 Experimental Study: Self-Adaptive Co-Working Robots 101 6.1 Robot Teaching and Co-Working with WEIR 103 6.1.1 WEIR Hardware Components 104 6.1.2 WEIR Software Infrastructure 105 6.1.3 KUKA LBR iiwa as WEIR Manipulator 106 6.1.4 Self-Adaptation Capabilities of WEIR 107 6.2 Cinderella as Testable Co-Working Application 109 6.2.1 Cinderella Setup and Basic Functionality 109 6.2.2 Co-Working with Cinderella 110 6.3 Testing Cinderella with MATE 112 6.3.1 Automating Test Execution 112 6.3.2 Modeling Cinderella in MATE 113 6.3.3 Testing Cinderella in the Loop 121 6.4 Evaluation Verdict and Summary 123 7 Summary and Discussion 125 7.1 Summary of Contributions 126 7.2 Open Research Questions 127 Bibliography 129 Appendices 137 Appendix Cinderella De nitions 139 1 Cinderella Adaptation Bounds 139 2 Cinderella Self-adaptive Workflow 140