Self-adaptive software is a closed-loop system that tries to manage, direct, or regulate its own behavior dynamically. Such a system aims at providing an automated and systematic approach to handling the increasing complexity of operation management. Mission-critical systems (e.g., e-business and telecommunication systems) are usually large, complex, and distributed. These systems must preserve their Quality of Service (QoS) at runtime under highly dynamic and non-deterministic conditions; therefore, they are suitable candidates for being equipped with self-adaptive capabilities. Although significant efforts have been devoted to modeling, designing, developing and deploying self-adaptive software since a decade ago, there is still a lack of well-established concrete processes for evaluating such systems. This dissertation proposes a systematic evaluation process for mission-critical self-adaptive software systems. The process is a well-defined testing approach that needs a post-mortem analysis, takes the quantified QoS requirements as inputs, and comprises two main phases: i) conducting system-level testing, and ii) evaluating QoS requirements satisfaction. The process uses Service Level Agreements (SLAs) as quantified QoS requirements, and consequently as the adaptation requirements of mission-critical systems. Adaptation requirements are specific types of requirements used to engineer self-adaptive software. Moreover, for the first phase, the dissertation discusses the uniqueness and necessity of conducting system-level load and stress testing on a self-adaptive software system, for collecting runtime QoS data. In the second phase, the process makes use of utility functions to generate a single value indicating the QoS satisfaction of the evaluated system. The dissertation mainly focuses on evaluating the performance, availability and reliability characteristics of QoS. An open source service-oriented Voice over IP (VoIP) application was selected as a case study. The VoIP application was transformed into a self-adaptive software system with various types of adaptation mechanisms. A set of empirical experiments was performed on the developed self-adaptive VoIP application, and the proposed process was adopted for evaluating the effectiveness of different adaptation mechanisms. To this end, the dissertation defines a sample SLA for the VoIP application, presents a report on the load and stress testing performed on the self-adaptive VoIP application, and presents a set of utility functions for evaluation. The experiments illustrate the validity, reliability, flexibility, and cost of the proposed evaluation process. In sum, this dissertation introduces a novel evaluation process for mission-critical self-adaptive software systems, and shows that the proposed process can help researchers to systematically evaluate their self-adaptive systems.

The thesis proposes a generic, configurable and enhanced goal-oriented decision-making model for self-adaptive software systems. The model has been designed to include feedback control loops as first class entities in the adaptation process whereby the decision-making processes can assess the impact of a previously executed decision, so that better decisions can be made in the future. Furthermore, the model provides the ability to detect and resolve conflicts amongst dependant adaptation requirements. The realization of the decision-model is extremely generic, flexible and extensible. It allows different voting algorithms to be specified for choosing a winner requirement for clusters of flexible adaptation requirements. Moreover, the implementation also allows for the specification of a wide variety of reinforcement learning algorithms to assess the impact of a previously executed decision. The implementation has been developed as a plug-in for a generic Java-based adaptation framework. It was tested using two case studies namely a News Web Application and an IP Telephony System. The aim of the conducted experiments was to assess the impact of the model on the systems goals and to determine the impact of feedback control loops as first class entities in the decision-making process. Based on the obtained results, it can be concluded that the model does improve the overall customer satisfaction level compared to a non-adaptive system. Moreover, it will be concluded that incorporating feedback loops as first class entities yields better results as compared to a decision-making model based solely on policies or goals.

Self-adaptive software is a closed-loop system that tries to manage, direct, or regulate its own behavior dynamically. Such a system aims at providing an automated and systematic approach to handling the increasing complexity of operation management. Mission-critical systems (e.g., e-business and telecommunication systems) are usually large, complex, and distributed. These systems must preserve their Quality of Service (QoS) at runtime under highly dynamic and non-deterministic conditions; therefore, they are suitable candidates for being equipped with self-adaptive capabilities. Although significant efforts have been devoted to modeling, designing, developing and deploying self-adaptive software since a decade ago, there is still a lack of well-established concrete processes for evaluating such systems. This dissertation proposes a systematic evaluation process for mission-critical self-adaptive software systems. The process is a well-defined testing approach that needs a post-mortem analysis, takes the quantified QoS requirements as inputs, and comprises two main phases: i) conducting system-level testing, and ii) evaluating QoS requirements satisfaction. The process uses Service Level Agreements (SLAs) as quantified QoS requirements, and consequently as the adaptation requirements of mission-critical systems. Adaptation requirements are specific types of requirements used to engineer self-adaptive software. Moreover, for the first phase, the dissertation discusses the uniqueness and necessity of conducting system-level load and stress testing on a self-adaptive software system, for collecting runtime QoS data. In the second phase, the process makes use of utility functions to generate a single value indicating the QoS satisfaction of the evaluated system. The dissertation mainly focuses on evaluating the performance, availability and reliability characteristics of QoS. An open source service-oriented Voice over IP (VoIP) application was selected as a case study. The VoIP application was transformed into a self-adaptive software system with various types of adaptation mechanisms. A set of empirical experiments was performed on the developed self-adaptive VoIP application, and the proposed process was adopted for evaluating the effectiveness of different adaptation mechanisms. To this end, the dissertation defines a sample SLA for the VoIP application, presents a report on the load and stress testing performed on the self-adaptive VoIP application, and presents a set of utility functions for evaluation. The experiments illustrate the validity, reliability, flexibility, and cost of the proposed evaluation process. In sum, this dissertation introduces a novel evaluation process for mission-critical self-adaptive software systems, and shows that the proposed process can help researchers to systematically evaluate their self-adaptive systems.

The thesis proposes a generic, configurable and enhanced goal-oriented decision-making model for self-adaptive software systems. The model has been designed to include feedback control loops as first class entities in the adaptation process whereby the decision-making processes can assess the impact of a previously executed decision, so that better decisions can be made in the future. Furthermore, the model provides the ability to detect and resolve conflicts amongst dependant adaptation requirements. The realization of the decision-model is extremely generic, flexible and extensible. It allows different voting algorithms to be specified for choosing a winner requirement for clusters of flexible adaptation requirements. Moreover, the implementation also allows for the specification of a wide variety of reinforcement learning algorithms to assess the impact of a previously executed decision. The implementation has been developed as a plug-in for a generic Java-based adaptation framework. It was tested using two case studies namely a News Web Application and an IP Telephony System. The aim of the conducted experiments was to assess the impact of the model on the systems goals and to determine the impact of feedback control loops as first class entities in the decision-making process. Based on the obtained results, it can be concluded that the model does improve the overall customer satisfaction level compared to a non-adaptive system. Moreover, it will be concluded that incorporating feedback loops as first class entities yields better results as compared to a decision-making model based solely on policies or goals.

The concepts that make self-adaptive software attractive also make it more difficult for users to gain confidence that these systems will consistently meet their goals under uncertain context. To improve user confidence in self-adaptive behavior, machine-readable conceptual models have been developed to instrument the adaption behavior of the target software system and primary feedback loop. By comparing these machine-readable models to the self-adaptive system, runtime verification and validation may be introduced as another method to increase confidence in self-adaptive systems; however, the existing conceptual models do not provide the semantics needed to institute this runtime verification or validation. This research confirms that the introduction of runtime verification and validation for self-adaptive systems requires the expansion of existing conceptual models with quality of service metrics, a hierarchy of goals, and states with temporal transitions. Based on this expanded semantics, runtime verification and validation was introduced as a second-level feedback loop to improve the performance of the primary feedback loop and quantitatively measure the quality of service achieved in a state-based, self-adaptive system. A web-based purchasing application running in a cloud-based environment was the focus of experimentation. In order to meet changing customer purchasing demand, the self-adaptive system monitored external context changes and increased or decreased available application servers. The runtime verification and validation system operated as a second-level feedback loop to monitor quality of service goals based on internal context, and corrected self-adaptive behavior when goals are violated. Two competing quality of service goals were introduced to maintain customer satisfaction while minimizing cost. The research demonstrated that the addition of a second-level runtime verification and validation feedback loop did quantitatively improve self-adaptive system performance even with simple, static monitoring rules.

The complexity of software systems is exploding, along with their use and application in new domains. Managing this complexity has become a focal point for research in Software Engineering. One direction for research in this area is developing techniques for designing adaptive software systems that self-optimize, self-repair, self-configure and self-protect, thereby reducing maintenance costs, while improving quality of service. This thesis presents a requirements-driven approach for developing adaptive and customizable systems. Requirements goal models are used as a basis for capturing problem variability, leading to software designs that support a space of possible behaviours – all delivering the same functionality. This space can be exploited at system deployment time to customize the system on the basis of user preferences. It can also be used at runtime to support system adaptation if the current behaviour of the running system is deemed to be unsatisfactory. The contributions of the thesis include a framework for systematically generating designs from high-variability goal models. Three complementary design views are generated: configurational view (feature model), behavioural view (statecharts) and an architectural view (parameterized architecture). The framework is also applied to the field of business process management for intuitive high-level process customization. In addition, the thesis proposes a modeling framework for capturing domain variability through contexts and applies it to goal models. A single goal model is used to capture requirements variations in different contexts. Models for particular contexts can then be automatically generated from this global requirements model. As well, the thesis proposes a new class of requirements-about-requirements called awareness requirements. Awareness requirements are naturally operationalized through feedback controllers – the core mechanisms of every adaptive system. The thesis presents an approach for systematically designing monitoring, analysis/diagnosis, and compensation components of a feedback controller, given a set of awareness requirements. Situations requiring adaptation are explicitly captured using contexts.

The complexity of software systems is exploding, along with their use and application in new domains. Managing this complexity has become a focal point for research in Software Engineering. One direction for research in this area is developing techniques for designing adaptive software systems that self-optimize, self-repair, self-configure and self-protect, thereby reducing maintenance costs, while improving quality of service. This thesis presents a requirements-driven approach for developing adaptive and customizable systems. Requirements goal models are used as a basis for capturing problem variability, leading to software designs that support a space of possible behaviours – all delivering the same functionality. This space can be exploited at system deployment time to customize the system on the basis of user preferences. It can also be used at runtime to support system adaptation if the current behaviour of the running system is deemed to be unsatisfactory. The contributions of the thesis include a framework for systematically generating designs from high-variability goal models. Three complementary design views are generated: configurational view (feature model), behavioural view (statecharts) and an architectural view (parameterized architecture). The framework is also applied to the field of business process management for intuitive high-level process customization. In addition, the thesis proposes a modeling framework for capturing domain variability through contexts and applies it to goal models. A single goal model is used to capture requirements variations in different contexts. Models for particular contexts can then be automatically generated from this global requirements model. As well, the thesis proposes a new class of requirements-about-requirements called awareness requirements. Awareness requirements are naturally operationalized through feedback controllers – the core mechanisms of every adaptive system. The thesis presents an approach for systematically designing monitoring, analysis/diagnosis, and compensation components of a feedback controller, given a set of awareness requirements. Situations requiring adaptation are explicitly captured using contexts.

Self-adaptive software systems are increasingly in demand. The driving forces are changes in the software “self” and “context”, particularly in distributed and pervasive applications. These systems provide self-* properties in order to keep requirements satisfied in different situations. Engineering self-adaptive software normally involves building the adaptable software and the adaptation manager. This PhD thesis focuses on the latter, especially on the design and implementation of the deciding process in an adaptation manager. For this purpose, a Quality-driven Framework for Engineering an Adaptation Manager (QFeam) is proposed, in which quality requirements play a key role as adaptation goals. Two major phases of QFeam are building the runtime adaptation model and designing the adaptation mechanism. The modeling phase investigates eliciting and specifying key entities of the adaptation problem space including goals, attributes, and actions. Three composition patterns are discussed to link these entities to build the adaptation model, namely: goal-centric, attribute-action-coupling, and hybrid patterns. In the second phase, the adaptation mechanism is designed according to the adopted pattern in the model. Therefore, three categories of mechanisms are discussed, in which the novel goal-ensemble mechanism is introduced. A concrete model and mechanism, the Goal-Attribute-Action Model (GAAM), is proposed based on the goal-centric pattern and the goal-ensemble mechanism. GAAM is implemented based on the StarMX framework for Java-based systems. Several considerations are taken into account in QFeam: i) the separation of adaptation knowledge from application knowledge, ii) highlighting the role of adaptation goals, and iii) modularity and reusability. Among these, emphasizing goals is the tenet of QFeam, especially in order to address the challenge of addressing several self- * properties in the adaptation manager. Furthermore, QFeam aims at embedding a model in the adaptation manager, particularly in the goal-centric and hybrid patterns. The proposed framework focuses on mission-critical systems including enterprise and service-oriented applications. Several empirical studies were conducted to put QFeam into practice, and also evaluate GAAM in comparison with other adaptation models and mechanisms. Three case studies were selected for this purpose: the TPC-W bookstore application, a news application, and the CC2 VoIP call controller. Several research questions were set for each case study, and findings indicate that the goal-ensemble mechanism and GAAM can outperform or work as well as a common rule-based approach. The notable difference is that the effort of building an adaptation manager based on a goal-centric pattern is less than building it using an attribute-action-coupling pattern. Moreover, representing goals explicitly leads to better scalability and understandability of the adaptation manager. Overall, the experience of working on these three systems show that QFeam improves the design and development process of the adaptation manager, particularly by highlighting the role of adaptation goals.

Self-adaptive software systems are increasingly in demand. The driving forces are changes in the software “self” and “context”, particularly in distributed and pervasive applications. These systems provide self-* properties in order to keep requirements satisfied in different situations. Engineering self-adaptive software normally involves building the adaptable software and the adaptation manager. This PhD thesis focuses on the latter, especially on the design and implementation of the deciding process in an adaptation manager. For this purpose, a Quality-driven Framework for Engineering an Adaptation Manager (QFeam) is proposed, in which quality requirements play a key role as adaptation goals. Two major phases of QFeam are building the runtime adaptation model and designing the adaptation mechanism. The modeling phase investigates eliciting and specifying key entities of the adaptation problem space including goals, attributes, and actions. Three composition patterns are discussed to link these entities to build the adaptation model, namely: goal-centric, attribute-action-coupling, and hybrid patterns. In the second phase, the adaptation mechanism is designed according to the adopted pattern in the model. Therefore, three categories of mechanisms are discussed, in which the novel goal-ensemble mechanism is introduced. A concrete model and mechanism, the Goal-Attribute-Action Model (GAAM), is proposed based on the goal-centric pattern and the goal-ensemble mechanism. GAAM is implemented based on the StarMX framework for Java-based systems. Several considerations are taken into account in QFeam: i) the separation of adaptation knowledge from application knowledge, ii) highlighting the role of adaptation goals, and iii) modularity and reusability. Among these, emphasizing goals is the tenet of QFeam, especially in order to address the challenge of addressing several self- * properties in the adaptation manager. Furthermore, QFeam aims at embedding a model in the adaptation manager, particularly in the goal-centric and hybrid patterns. The proposed framework focuses on mission-critical systems including enterprise and service-oriented applications. Several empirical studies were conducted to put QFeam into practice, and also evaluate GAAM in comparison with other adaptation models and mechanisms. Three case studies were selected for this purpose: the TPC-W bookstore application, a news application, and the CC2 VoIP call controller. Several research questions were set for each case study, and findings indicate that the goal-ensemble mechanism and GAAM can outperform or work as well as a common rule-based approach. The notable difference is that the effort of building an adaptation manager based on a goal-centric pattern is less than building it using an attribute-action-coupling pattern. Moreover, representing goals explicitly leads to better scalability and understandability of the adaptation manager. Overall, the experience of working on these three systems show that QFeam improves the design and development process of the adaptation manager, particularly by highlighting the role of adaptation goals.

It is known that to pursuit end-to-end QoS of a class-based traffic flow is inefficient in Differentiated Service Networks. Therefore in this thesis, we propose an effective throughput-recovery mechanism to allow high-priority traffic flow to receive suitable resource allocation, and hence the end-to-end QoS is guaranteed. The proposed throughput-recovery mechanism assures a predefined minimum departure rate of low-latency EF dataflow. It consists of two parts. The first part is referred to as the feedback self-adaptive mechanism, where the egress node measures and monitors throughput of EF dataflow to decide whether to send the control messages to ingress node. When ingress node receives the control messages, it will reallocate the resources to improve EF throughput. The second part is referred to as the dynamic weight adjustment mechanism, which can prevent EF packets from dropping when congestion occurs in the core routers. For the purpose of demonstration, we build a mathematical model and use NS-2 simulator. We have proved our throughput-recovery mechanism is effective in improving the throughput of EF traffic flow. Finally, we modify the traditional WRR such that it can adjust weight based on the delay requirements.