In Model-Based Testing (MBT) the main goal is to test a system by designing models which describe the functionality of the system to test. Subsequently, test cases are obtained from the model, and these test cases can be executed automatically. Experience has shown that the learning curve for learning MBT can be steep - especially for people who do not have previous programming experience. This is because the language used to design models uses programming language concepts. In this thesis we describe a tool which automatically generates models, given an initial set of requirements. The advantage of this tool is that users do not need to learn a model-based testing language to design models, but instead they must learn to use a high-level of abstraction and a Graphical User Interface to specify their test cases. We demonstrate the value of the tool by using it to design models that generate test cases for telecommunications system, but show that this tool can be adapted for use in testing similar systems. The application of this tool can facilitate traditional phase-based software development methods, by saving a considerable amount of time and resources. In addition, when applied to agile software development, the reduced time required for testing because of the use of our tool helps shortening the feedback loops between designing and testing, thus increasing team efficiency within every iteration. / I Model-Based Testing (MBT) är det huvudsakliga målet att testa ett system genom modeller som beskriver systemets unktionalitet för att testa. Därefter erhålls testfall från modellen, och dessa testfall kan utföras automatiskt. Erfarenheten har visat att inlärningskurvan för lärande MBT kan vara branta - särskilt för personer som inte har tidigare erfarenhet av programmering. Detta beror på det språk som används för användning programmeringsspråk begrepp. I denna avhandling beskriver vi ett verktyg som automatiskt genererar modellerna, med tanke på en första uppsättning krav. Fördelen med detta verktyg är att användarna inte behöver lära sig en modellbaserad testning av språket att konstruera modeller, utan de måste lära sig att använda en hög nivåav abstraktion och ett grafiskt användargränssnitt för att ange sina testfall. Vi demonstrerar värdet av verktyget genom att använda den för att konstruera modeller som genererar testfall för telekommunikationssystem, men visar att detta verktyg kan anpassas för användning vid testning av liknande system. Tillämpningen av detta verktyg kan underlätta traditionella fas-baserade metoder mjukvaruutveckling, genom att spara en avsevärd tid och resurser. Dessutom, när det tillämpas på Agile Software utveckling, minskade tid som krävs för att testa på grund av användningen av vårt verktyg hjälper förkorta återkopplingar mellan design och testning, vilket ökar teamet effektiviteten inom varje iteration. / En Model-Based Testing (MBT), el objetivo principal es testear un sistema mediante el diseño de modelos que describan su funcionalidad. En consecuencia, estos modelos generan test cases que pueden ser ejecutados automáticamente en dicho sistema. La experiencia nos muestra que la curva de aprendizaje en el caso de MBT puede ser pronunciada, especialmente para aquellos sin ninguna experiencia previa en programación. Esto se debe a que los lenguajes usados para diseñar modelos usan conceptos intrínsecos a los lenguajes de programación. En este Proyecto Fin de Carrera, describimos una herramienta que genera automáticamente modelos, dado un conjunto de requisitos inicial. La ventaja que ofrece esta herramienta es que los usuarios no requieren el aprendizaje de ninguno lenguaje de modelado a la hora de diseñar modelos, sino que tan solo deben aprender a utilizar una Interfaz de Usuario Gráfica (GUI), a un alto nivel de abstracción, para especificar sus test cases. Demostramos el valor de esta herramienta mediante su aplicación en un nuevo sistema de telecomunicaciones en fase de pruebas de Ericsson, mostrando al mismo tiempo que puede ser utilizada en el testeo de sistemas similares. La aplicación de esta herramienta puede facilitar los métodos de desarrollo de software tradicionales mediante el ahorro de una cantidad considerable de tiempo y recursos. Además, aplicada a métodos de desarrollo ágil de software, el tiempo reducido requerido para el testing a causa del uso de esta herramienta ayuda a acortar los plazos entre diseño y testing, y en consecuencia, incrementando la eficiencia del equipo en cada iteración.

There are many methods for modelling the reliability of systems based on component failure data. This task becomes more complex as systems increase in size, or undertake missions that comprise multiple discrete modes of operation, or phases. Existing techniques require certain levels of expertise in the model generation and calculation processes, meaning that risk and reliability assessments of systems can often be expensive and time-consuming. This is exacerbated as system complexity increases. This thesis presents a novel method which generates reliability models for phasedmission systems, based on Petri nets, from simple input files. The process has been automated with a piece of software designed for engineers with little or no experience in the field of risk and reliability. The software can generate models for both repairable and non-repairable systems, allowing redundant components and maintenance cycles to be included in the model. Further, the software includes a simulator for the generated models. This allows a user with simple input files to perform automatic model generation and simulation with a single piece of software, yielding detailed failure data on components, phases, missions and the overall system. A system can also be simulated across multiple consecutive missions. To assess performance, the software is compared with an analytical approach and found to match within 5% in both the repairable and non-repairable cases. The software documented in this thesis could serve as an aid to engineers designing new systems to validate the reliability of the system. This would not require specialist consultants or additional software, ensuring that the analysis provides results in a timely and cost-effective manner.

With the advent of autonomous systems and deep learning systems, safety pertaining to these systems has become a major concern. The existing failure analysis techniques are not enough to thoroughly analyze the safety in these systems. Moreover, because these systems are created to operate in various conditions, they are susceptible to unknown safety issues. Hence, we need mechanisms which can take into account the complexity of operational design domains, identify safety issues other than failures, and expose unknown safety issues. Moreover, existing safety analysis approaches require a lot of effort and time for analysis and do not consider machine learning (ML) safety. To address these limitations, in this dissertation, we discuss an artificial-intelligence driven model-based methodology that aids in identifying unknown safety issues and analyzing ML safety. Our methodology consists of 4 major tasks: 1) automated model generation, 2) automated analysis of component state transition model specification, 3) undesired states analysis, and 4) causal factor analysis. In our methodology we identify unknown safety issues by finding undesired combinations of components' states and environmental entities' states as well as causes resulting in these undesired combinations. In our methodology, we refer to the behaviors that occur because of undesired combinations as off-nominal behaviors (ONBs). To identify undesired combinations and ONBs that aid in exposing unknown safety issues with less effort and time we proposed various approaches for each of the task and performed corresponding empirical studies. We also discussed machine learning safety analysis from the perspective of machine learning engineers as well as system and software safety engineers. The results of studies conducted as part of our research shows that our proposed methodology helps in identifying unknown safety issues effectively. Our results also show that combinatorial methods are effective in reducing effort and time for analysis of off-nominal behaviors without overlooking any dependencies among components and environmental entities of a system. We also found that safety analysis of machine learning components is different from analysis of conventional software components and detail the aspects we need to consider for ML safety.