This dissertation concerns testability of event-triggered real-time systems. Real-time systems are known to be hard to test because they are required to function correct both with respect to what the system does and when it does it. An event-triggered real-time system is directly controlled by the events that occur in the environment, as opposed to a time-triggered system, which behavior with respect to when the system does something is constrained, and therefore more predictable. The focus in this dissertation is the behavior in the time domain and it is shown how testability is affected by some factors when the system is tested for timeliness. This dissertation presents a survey of research that focuses on software testability and testability of real-time systems. The survey motivates both the view of testability taken in this dissertation and the metric that is chosen to measure testability in an experiment. We define a method to generate sets of traces from a model by using a meta algorithm on top of a model checker. Defining such a method is a necessary step to perform the experiment. However, the trace sets generated by this method can also be used by test strategies that are based on orderings, for example execution orders. An experimental study is presented in detail. The experiment investigates how testability of an event-triggered real-time system is affected by some constraining properties of the execution environment. The experiment investigates the effect on testability from three different constraints regarding preemptions, observations and process instances. All of these constraints were claimed in previous work to be significant factors for the level of testability. Our results support the claim for the first two of the constraints while the third constraint shows no impact on the level of testability. Finally, this dissertation discusses the effect on the event-triggered semantics when the constraints are applied on the execution environment. The result from this discussion is that the first two constraints do not change the semantics while the third one does. This result indicates that a constraint on the number of process instances might be less useful for some event-triggered real-time systems.

Model Based Testing (MBT) is a new-age test automation technique traditionally used for Functional Black-Box Testing. Its capability of generating test cases by using model developed from the analysis of the abstract behavior of the System under Test is gaining popularity. Many commercial and open source MBT tools are available currently in market. But each one has its own specific way of modeling and test case generation mechanism that is suitable for varied types of systems. Ericsson, a telecommunication equipment provider company, is currently adapting Model Based Testing in some of its divisions for functional testing. Those divisions haven’t yet attempted adapting Model Based Testing for non-functional testing in a full-pledged manner. A comparative study between various MBT tools will help one of the Ericsson’s testing divisions to select the best tool for adapting to its existing test environment. This also helps in improving the quality of testing while reducing cost, time and effort. This thesis work helps Ericsson testing division to select such an effective MBT tool. Based on aspects such as functionality, flexibility, adaptability, performance etc., a comparative study is carried out on various available MBT tools and a few were selected among them: Qtronic, ModelJUnit and Elvior Motes.This thesis also helps to understand the usability of the selected tools for modeling of non-functional requirements using a new method. A brief idea of modeling the non-functional requirements is suggested in this thesis. A System under Test was identified and its functional behavior was modeled along with the non functional requirements in Qtronic and ModelJUnit. An experimental analysis, backed by observations of using the new proposed method indicates that the method is efficient enough to carry out modeling non-functional requirements along with modeling of functional requirements by identifying the appropriate approach.Model Based Testing (MBT) is a new-age test automation technique traditionally used for Functional Black-Box Testing. Its capability of generating test cases by using model developed from the analysis of the abstract behavior of the System under Test is gaining popularity. Many commercial and open source MBT tools are available currently in market. But each one has its own specific way of modeling and test case generation mechanism that is suitable for varied types of systems. Ericsson, a telecommunication equipment provider company, is currently adapting Model Based Testing in some of its divisions for functional testing. Those divisions haven’t yet attempted adapting Model Based Testing for non-functional testing in a full-pledged manner. A comparative study between various MBT tools will help one of the Ericsson’s testing divisions to select the best tool for adapting to its existing test environment. This also helps in improving the quality of testing while reducing cost, time and effort. This thesis work helps Ericsson testing division to select such an effective MBT tool. Based on aspects such as functionality, flexibility, adaptability, performance etc., a comparative study is carried out on various available MBT tools and a few were selected among them: Qtronic, ModelJUnit and Elvior Motes. This thesis also helps to understand the usability of the selected tools for modeling of non-functional requirements using a new method. A brief idea of modeling the non-functional requirements is suggested in this thesis. A System under Test was identified and its functional behavior was modeled along with the non functional requirements in Qtronic and ModelJUnit. An experimental analysis, backed by observations of using the new proposed method indicates that the method is efficient enough to carry out modeling non-functional requirements along with modeling of functional requirements by identifying the appropriate approach.

Web pages serve a huge number of internet users in nearly every area. An adequate testing is needed to address the problems of web domains for more efficient and accurate services. We present an automated tool to test web applications against execution errors and the errors occured when many users connect the same server concurrently. Our tool, called NaMoX, attains the clickables of the web pages, creates a model exerting depth first search algorithm. NaMoX simulates a number of users, parses the developed model, and tests the model by branch coverage analysis. We have performed experiments on five web sites. We have reported the response times when a click operation is eventuated. We have found 188 errors in total. Quality metrics are extracted and this is applied to the case studies.

The main goal of testing is to find errors in the System Under Test (SUT). Prior research indicated that Model-Based Testing is indeed good at finding SUT errors, can lead to less time and effort spent on testing if the time needed to write and maintain the model plus the time spent on directing the test generation is less than the cost of manually designing and maintaining a test suite. This study proposed a methodology for Model-Based Testing. In this approach, Sequence Diagrams and Class Diagram are used to determine the testing path and test case and Object Constraint Language is used to specify the business logic constraint. Three real-world cases and a CASE tool are used to test the usability (including the concepts, application, and advantages) of the proposed methodology. With this approach, SUT errors can be found at the systems analysis and design stage and thereby reduce the cost of software testing and enhance the efficiency of system development.

The main goal of model testing is to find system errors at the systems analysis and design stage and thereby reduce the cost of software testing and enhance the efficiency of system development. Prior researches proposed an Integrated Method which utilizes the artifacts from the Platform Independent Model (PIM) to construct the test path and constrained class tuples (CCT). These two can then be integrated to construct an Integrated Testing Model (ITM) for determining the test data and test cases for Model-Based Testing. This study develops a methodology which extracting the information from the Sequence Diagram and Class Diagram from the PIM to construct an ITM. The research methodology is articulated using the design science research methodology. A prototype embedded methodology has been developed. A usability evaluation is performed on the prototype to demonstrate its usability in terms of the factors, efficiency and effectiveness. With this methodology, the test paths, CCT and ITM can be generated automatically; thereby reducing the costs and increasing the efficiency of Model-Based Testing.

Submitted by Dilene Paulo (dilene.fatima@ufcg.edu.br) on 2018-01-25T12:53:52Z No. of bitstreams: 1 ADRIANA CARLA DAMASCENO – TESE PPGCC 2015.pdf: 3272500 bytes, checksum: 779024871de692299535f2de0eeabfb9 (MD5) / Made available in DSpace on 2018-01-25T12:53:52Z (GMT). No. of bitstreams: 1 ADRIANA CARLA DAMASCENO – TESE PPGCC 2015.pdf: 3272500 bytes, checksum: 779024871de692299535f2de0eeabfb9 (MD5) Previous issue date: 2015-03-06 / We can find many examples of Real-time Systems (RTS) in critical applications such as patient monitoring, air traffic control and others. A failure in this kind of system can be catastrophic. For example, it can harm human lives or increase project budgets. Hence, the testing of real-time systems must be accurate. Models are used to perform this task, since they contain information about how the system behaves and when actions may happen. Due to the complexity of the available systems, most RTS are composed of subsystems that interact as part of a bigger system. These subsystems are combined through operators to model their specification behavior. However, works on the testing of compositional models for RTS are practically nonexistent. Among the available approaches to perform testing for non-compositional RTS models, the tioco conformance testing theory focuses on generating test cases based on implementation and specification models. Moreover, a conformance relation defines whether success in testing means conformance between an implementation and a specification. To express specifications and to represent implementations under test, we use Timed Input Output Symbolic Transitions Systems (TIOSTS). These models store symbolic data and clock variables, avoiding the state space and region explosion problems. Regarding the testing of compositional models, some questions may arise: If two subsystem implementations are tioco conformant to their specifications, is it correct to assume that the composition of the implementations is also tioco conformant to the composition of their specifications? In this case, how can operators be defined to work with TIOSTS and tioco? To answer these questions, this thesis proposes the sequential, interruption and parallel operators for the TIOSTS model. For each operator, we study how the tioco conformance relation behaves with respect to subsystems and the composed system. We present results towards properties of compositional operators when the subsystems are composed, as well as implementing them. Besides, we show three examples where each operator can be used and illustrate the applicability of our approach in two exploratory studies. The first models components of a aircraft specification and the second presents application level interruptions in an Android system. / We can nd many examples of Real-time Systems (RTS) in critical applications such as patient monitoring, air tra c control and others. A failure in this kind of system can be catastrophic. For example, it can harm human lives or increase project budgets. Hence, the testing of real-time systems must be accurate. Models are used to perform this task, since they contain information about how the system behaves and when actions may happen. Due to the complexity of the available systems, most RTS are composed of subsystems that interact as part of a bigger system. These subsystems are combined through operators to model their speci cation behavior. However, works on the testing of compositional models for RTS are practically nonexistent. Among the available approaches to perform testing for non-compositional RTS models, the tioco conformance testing theory focuses on generating test cases based on implementation and speci cation models. Moreover, a conformance relation de nes whether success in testing means conformance between an implementation and a speci cation. To express speci cations and to represent implementations under test, we use Timed Input Output Symbolic Transitions Systems (TIOSTS). These models store symbolic data and clock variables, avoiding the state space and region explosion problems. Regarding the testing of compositional models, some questions may arise: If two subsystem implementations are tioco conformant to their speci cations, is it correct to assume that the composition of the implementations is also tioco conformant to the composition of their speci cations? In this case, how can operators be de ned to work with TIOSTS and tioco? To answer these questions, this thesis proposes the sequential, interruption and parallel operators for the TIOSTS model. For each operator, we study how the tioco conformance relation behaves with respect to subsystems and the composed system. We present results towards properties of compositional operators when the subsystems are composed, as well as implementing them. Besides, we show three examples where each operator can be used and illustrate the applicability of our approach in two exploratory studies. The rst models components of a aircraft speci cation and the second presents application level interruptions in an Android system.

Testing is a critical activity in the software development process in order to obtain systems of high quality. Tieto typically develops complex systems, which are currently tested through a large number of manually designed test cases. Recent development within software testing has resulted in methods and tools that can automate the test case design, the generation of test code and the test result evaluation based on a model of the system under test. This testing approach is called model-based testing (MBT). This thesis is a feasibility study of the model-based testing concept and has been performed at the Tieto office in Karlstad. The feasibility study included the use and evaluation of the model-based testing tool Qtronic, developed by Conformiq, which automatically designs test cases given a model of the system under test as input. The experiments for the feasibility study were based on the incremental development of a test object, which was the client protocol module of a simplified model for an ATM (Automated Teller Machine) client-server system. The experiments were evaluated both individually and by comparison with the previous experiment since they were based on incremental development. For each experiment the different tasks in the process of testing using Qtronic were analyzed to document the experience gained as well as to identify strengths and weaknesses. The project has shown the promise inherent in using a model-based testing approach. The application of model-based testing and the project results indicate that the approach should be further evaluated since experience will be crucial if the approach is to be adopted within Tieto’s organization.

Software testing is an important process for ensuring the quality of the software. As the complexity of the software increases, traditional means of manual testing becomes increasingly more complex and time consuming. In most embedded systems, designing software with as few errors as possible is often critical. Resource usage is also of concern for proper behavior because of the very nature of embedded systems.  To design reliable and energy-efficient systems, methods are needed to detect hot points of consumption and correct them prior to deployment. To reduce testing effort, Model-based testing can be used which is one testing method that allows for automatic testing of model based systems. Model-based testing has not been investigated extensively for revealing resource usage anomalies in embedded systems. UPPAAL SMC is a statistical model checking tool which can be used to model the system’s resource usage. Currently UPPAAL SMC lacks the support for performing automatic test generation and test selection. In this thesis we provide this support with a framework for automatic test generation and test selection using mutation analysis, a method for minimizing the generated test suite while maximizing the fault coverage and a tool implementing the framework on top of the UPPAAL SMC tool. The thesis also evaluates the framework on a Brake by Wire industrial system. Our results show that we could for a Brake-by-wire system, simulated on a consumer processor with five mutants, in best case find a test case that achieved 100% mutation score within one minute and confidently identify at least one test case that achieved full mutation score within five minutes. The evaluation shows that this framework is applicable and relatively efficient on an industrial system for reducing continues resource usage target testing effort.

Graphical user interfaces represent significant part of computer systems. This causes an increase of demands for their proper behavior, which can be achieved by a proper testing during development. This thesis describes the principles of model based testing for automated generation of test cases. In this way, the productivity of test teams can increase. The thesis defines model for modeling the most common GUI elements, algorithm for obtaining such a model from the GUI, and a tool for definition of coverage criteria.

Traceability in the software industry is a topic which has been studied for a long time, but there still remain questions to be answered. This thesis looks into the state of software traceability in the software industry is today. It is also a case study in implementing traceability in a model-based testing process at a large automotive company. The thesis finds that while progress has been made in identifying the important aspects of traceability, there does not yet exist a clear consensus on general traceability models which can be used to implement traceability in an arbitrary company or project. Through a case study in implementing traceability in a model-based testing process, the thesis gives one example of how this could potentially be done.