Discrete event simulation is a widely used technique for decision support. The results of the simulation must be reliable for critical decision making problems. Therefore, much research has concentrated on the verification and validation of simulations. In this thesis, we apply a well-known dynamic verification technique, assertion checking method, as a validation technique. Our aim is to validate the particular runs of the simulation model, rather than the model itself. As a case study, the operations of a manufacturing cell have been simulated. The cell, which is METUCIM Laboratory at the Mechanical Engineering Department of METU, has a robot and a conveyor to carry the materials, and two machines to manufacture the items, and a quality control to measure the correctness of the manufactured items. This simulation is monitored and checked by using the Monitoring and Checking (MaC) tool, a prototype developed at the University of Pennsylvania. The separation of low-level implementation details (pertaining to the code) from the high-level requirement specifications (pertaining to the simuland) helps keep monitoring and checking the simulations at an abstract level.

No description available.

In today's modern world, Cyber-Physical Systems (CPS) have gained widespread prevalence, offering tremendous benefits while also increasing society's dependence on them. Given the direct interaction of CPS with the physical environment, their malfunction or compromise can pose significant risks to human life, property, and the environment. However, as the complexity of CPS rises due to heightened expectations and expanded functional requirements, ensuring their trustworthy operation solely during the development process becomes increasingly challenging. This thesis introduces and delves into the novel concept of the 'Protective Shell' – a real-time safeguard actively monitoring CPS during their operational phases. The protective shell serves as a last line of defence, designed to detect abnormal behaviour, conduct thorough analyses, and initiate countermeasures promptly, thereby mitigating unforeseen risks in real-time. The primary objective of this research is to enhance the overall safety and security of CPS by refining, partly implementing, and evaluating the innovative protective shell concept. To provide context for collaborative systems working towards higher objectives — common within CPS as system-of-systems (SoS) — the thesis introduces the 'Emergence Matrix'. This matrix categorises outcomes of such collaboration into four quadrants based on their anticipated nature and desirability. Particularly concerning are outcomes that are both unexpected and undesirable, which frequently serve as the root cause of safety accidents and security incidents in CPS scenarios. The protective shell plays a critical role in mitigating these unfavourable outcomes, as conventional vulnerability elimination procedures during the CPS design phase prove insufficient due to their inability to proactively anticipate and address these unforeseen situations. Employing the design science research methodology, the thesis is structured around its iterative cycles and the research questions imposed, offering a systematic exploration of the topic. A detailed analysis of various safety accidents and security incidents involving CPS was conducted to retrieve vulnerabilities that led to dangerous outcomes. By developing specific protective shells for each affected CPS and assessing their effectiveness during these hazardous scenarios, a generic core for the protective shell concept could be retrieved, indicating general characteristics and its overall applicability. Furthermore, the research presents a generic protective shell architecture, integrating advanced anomaly detection techniques rooted in explainable artificial intelligence (XAI) and human machine teaming. While the implementation of protective shells demonstrate substantial positive impacts in ensuring CPS safety and security, the thesis also articulates potential risks associated with their deployment that require careful consideration. In conclusion, this thesis makes a significant contribution towards the safer and more secure integration of complex CPS into daily routines, critical infrastructures and other sectors by leveraging the capabilities of the generic protective shell framework.:1 Introduction 1.1 Background and Context 1.2 Research Problem 1.3 Purpose and Objectives 1.3.1 Thesis Vision 1.3.2 Thesis Mission 1.4 Thesis Outline and Structure 2 Design Science Research Methodology 2.1 Relevance-, Rigor- and Design Cycle 2.2 Research Questions 3 Cyber-Physical Systems 3.1 Explanation 3.2 Safety- and Security-Critical Aspects 3.3 Risk 3.3.1 Quantitative Risk Assessment 3.3.2 Qualitative Risk Assessment 3.3.3 Risk Reduction Mechanisms 3.3.4 Acceptable Residual Risk 3.4 Engineering Principles 3.4.1 Safety Principles 3.4.2 Security Principles 3.5 Cyber-Physical System of Systems (CPSoS) 3.5.1 Emergence 4 Protective Shell 4.1 Explanation 4.2 System Architecture 4.3 Run-Time Monitoring 4.4 Definition 4.5 Expectations / Goals 5 Specific Protective Shells 5.1 Boeing 737 Max MCAS 5.1.1 Introduction 5.1.2 Vulnerabilities within CPS 5.1.3 Specific Protective Shell Mitigation Mechanisms 5.1.4 Protective Shell Evaluation 5.2 Therac-25 5.2.1 Introduction 5.2.2 Vulnerabilities within CPS 5.2.3 Specific Protective Shell Mitigation Mechanisms 5.2.4 Protective Shell Evaluation 5.3 Stuxnet 5.3.1 Introduction 5.3.2 Exploited Vulnerabilities 5.3.3 Specific Protective Shell Mitigation Mechanisms 5.3.4 Protective Shell Evaluation 5.4 Toyota 'Unintended Acceleration' ETCS 5.4.1 Introduction 5.4.2 Vulnerabilities within CPS 5.4.3 Specific Protective Shell Mitigation Mechanisms 5.4.4 Protective Shell Evaluation 5.5 Jeep Cherokee Hack 5.5.1 Introduction 5.5.2 Vulnerabilities within CPS 5.5.3 Specific Protective Shell Mitigation Mechanisms 5.5.4 Protective Shell Evaluation 5.6 Ukrainian Power Grid Cyber-Attack 5.6.1 Introduction 5.6.2 Vulnerabilities in the critical Infrastructure 5.6.3 Specific Protective Shell Mitigation Mechanisms 5.6.4 Protective Shell Evaluation 5.7 Airbus A400M FADEC 5.7.1 Introduction 5.7.2 Vulnerabilities within CPS 5.7.3 Specific Protective Shell Mitigation Mechanisms 5.7.4 Protective Shell Evaluation 5.8 Similarities between Specific Protective Shells 5.8.1 Mitigation Mechanisms Categories 5.8.2 Explanation 5.8.3 Conclusion 6 AI 6.1 Explainable AI (XAI) for Anomaly Detection 6.1.1 Anomaly Detection 6.1.2 Explainable Artificial Intelligence 6.2 Intrinsic Explainable ML Models 6.2.1 Linear Regression 6.2.2 Decision Trees 6.2.3 K-Nearest Neighbours 6.3 Example Use Case - Predictive Maintenance 7 Generic Protective Shell 7.1 Architecture 7.1.1 MAPE-K 7.1.2 Human Machine Teaming 7.1.3 Protective Shell Plugin Catalogue 7.1.4 Architecture and Design Principles 7.1.5 Conclusion Architecture 7.2 Implementation Details 7.3 Evaluation 7.3.1 Additional Vulnerabilities introduced by the Protective Shell 7.3.2 Summary 8 Conclusion 8.1 Summary 8.2 Research Questions Evaluation 8.3 Contribution 8.4 Future Work 8.5 Recommendation

"Continual Validation of distributed software systems can facilitate their development and evolution and engender user trust. We present a monitoring architecture that is being developed collaboratively under DARPA's Dynamic Assembly for System Adaptability, Dependability, and Assurance program. The monitoring system includes a probing infrastructure that is injected into or wrapped around a target software system. Probes deliver events of interest to be processed by a monitoring infrastructure that consists of gauges for delivering information to system administrators. This thesis presents a classification of existing probing technologies and contains a full implementation of a probing infrastructure in Java."

Cars, trains, trucks, telecom networks and industrial robots are examples of products relying on complex embedded software systems, running on embedded computers. Such systems may consist of millions of lines of program code developed by hundreds of engineers over many years, often decades. Over the long life-cycle of such systems, the main part of the product development costs is typically not the initial development, but the software maintenance, i.e., improvements and corrections of defects, over the years. Of the maintenance costs, a major cost is the verification of the system after changes has been applied, which often requires a huge amount of testing. However, today's techniques are not sufficient, as defects often are found post-release, by the customers. This area is therefore of high relevance for industry. Complex embedded systems often control machinery where timing is crucial for accuracy and safety. Such systems therefore have important requirements on timing, such as maximum response times. However, when maintaining complex embedded software systems, it is difficult to predict how changes may impact the system's run-time behavior and timing, e.g., response times.Analytical and formal methods for timing analysis exist, but are often hard to apply in practice on complex embedded systems, for several reasons. As a result, the industrial practice in deciding the suitability of a proposed change, with respect to its run-time impact, is to rely on the subjective judgment of experienced developers and architects. This is a risky and inefficient, trial-and-error approach, which may waste large amounts of person-hours on implementing unsuitable software designs, with potential timing- or performance problems. This can generally not be detected at all until late stages of testing, when the updated software system can be tested on system level, under realistic conditions. Even then, it is easy to miss such problems. If products are released containing software with latent timing errors, it may cause huge costs, such as car recalls, or even accidents. Even when such problems are found using testing, they necessitate design changes late in the development project, which cause delays and increases the costs. This thesis presents an approach for impact analysis with respect to run-time behavior such as timing and performance for complex embedded systems. The impact analysis is performed through optimizing simulation, where the simulation models are automatically generated from the system implementation. This approach allows for predicting the consequences of proposed designs, for new or modified features, by prototyping the change in the simulation model on a high level of abstraction, e.g., by increasing the execution time for a particular task. Thereby, designs leading to timing-, performance-, or resource usage problems can be identified early, before implementation, and a late redesigns are thereby avoided, which improves development efficiency and predictability, as well as software quality. The contributions presented in this thesis is within four areas related to simulation-based analysis of complex embedded systems: (1) simulation and simulation optimization techniques, (2) automated model extraction of simulation models from source code, (3) methods for validation of such simulation models and (4) run-time recording techniques for model extraction, impact analysis and model validation purposes. Several tools has been developed during this work, of which two are in commercialization in the spin-off company Percepio AB. Note that the Katana approach, in area (2), is subject for a recent patent application - patent pending. / PROGRESS