<p>Today, automated software testing has been implemented successfully in many systems, however there does still exist relatively unexplored areas as how automated testing can be implemented in a real-time embedded system. This problem has been the foundation for the work in this master thesis, to investigate the possibility to implement an automated software testing process for the testing of an embedded real-time system at IVU Traffic Technologies AG in Aachen, Germany.</p><p>The system that has been the test object is the on board system i.box.</p><p>This report contains the result of a literature study in order to present the foundation behind the solution to the problem of the thesis. Questions answered in the study are: when to automate, how to automate and which traps should one avoid when implementing an automated software testing process in an embedded system.</p><p>The process of automating the manual process has contained steps as constructing test cases for automated testing, analysing whether an existing tool should be used or a unique test system needs to be developed. The analysis, based on the requirements on the test system, the literature study and an investigation of available test tools, lead to the development of a new test tool. Due to limited devlopement time and characterstics of the i.box, the new tool was built based on post execution evaluation. The tool was therefore divided into two parts, a part that executed the test and a part that evaluated the result. By implementing an automated test tool it has been proved that it is possible to automate the test process at system test level in the i.box.</p>

Today, automated software testing has been implemented successfully in many systems, however there does still exist relatively unexplored areas as how automated testing can be implemented in a real-time embedded system. This problem has been the foundation for the work in this master thesis, to investigate the possibility to implement an automated software testing process for the testing of an embedded real-time system at IVU Traffic Technologies AG in Aachen, Germany. The system that has been the test object is the on board system i.box. This report contains the result of a literature study in order to present the foundation behind the solution to the problem of the thesis. Questions answered in the study are: when to automate, how to automate and which traps should one avoid when implementing an automated software testing process in an embedded system. The process of automating the manual process has contained steps as constructing test cases for automated testing, analysing whether an existing tool should be used or a unique test system needs to be developed. The analysis, based on the requirements on the test system, the literature study and an investigation of available test tools, lead to the development of a new test tool. Due to limited devlopement time and characterstics of the i.box, the new tool was built based on post execution evaluation. The tool was therefore divided into two parts, a part that executed the test and a part that evaluated the result. By implementing an automated test tool it has been proved that it is possible to automate the test process at system test level in the i.box.

Context: Software testing is a crucial area in software development. It has been found that there is little evidence on how testing is performed in software automotive domain. Challenges in this domain have not yet been fully evaluated. Furthermore, interactions between test procedures, and other activities of software development are left unexposed. Assessment of test process is an important part of improvement initiatives. Objectives: In this study we perform an in-depth investigation to identify challenges which lead to wastes in test process in the context of automotive software domain. To this end, we also identified strengths that add value in test process. Methods: We performed a Case study at a Sweden based large automotive organization. To this end, we conducted semi-structured interviews and used Grounded Theory method for interview data analysis. We used Value Stream Mapping (VSM), a Lean methodology tool for test process assessment. Furthermore, we conducted Systematic Literature Reviews to look for evidence of similar problems and solutions offered in peer-reviewed literature. Results: We grouped the identified challenges and strengths into 10 and 4 categories respectively. We also identified 7 kinds of waste and 4 kinds of values based using VSM. Based on systematic literature reviews, we proposed 7 solution proposals, an agile process model with practices. Conclusions: We conclude that VSM is an efficient tool in eliciting improvement potentials in software testing context using qualitative data. In regard to solution proposals, we propose further empirical work to evaluate the solution proposals themselves and tailor to company needs. However, the results obtained through this study can be compared to test processes at other companies in automotive domain. / kasoju.abhinaya@gmail.com

It is important to manage iterative projects in a way to maximize quality and minimize cost. To achieve high quality, accurate project estimates are of high importance. It is challenging to predict the effort that is required to perform test activities in an iterative development. If testers put extra effort in testing then schedule might be delayed, however, if testers spend less effort then quality could be affected. Currently there is no model for test effort prediction in iterative development to overcome such challenges. This paper introduces and validates a dynamic Bayesian network to predict test effort in iterative software development. In this research work, the proposed framework is evaluated in a number of ways: First, the framework behavior is observed by considering different parameters and performing initial validation. Then secondly, the framework is validated by incorporating data from two industrial projects. The accuracy of the results has been verified through different prediction accuracy measurements and statistical tests. The results from the verification confirmed that the framework has the ability to predict test effort in iterative projects accurately.

The main part in the testing of the software is in the generation of test cases suitable for software system testing. The quality of the test cases plays a major role in reducing the time of software system testing and subsequently reduces the cost. The test cases, in model de- sign stages, are used to detect the faults before implementing it. This early detection offers more flexibility to correct the faults in early stages rather than latter ones. The best of these tests, that covers both static and dynamic software system model specifications, is one of the chal- lenges in the software testing. The static and dynamic specifications could be represented efficiently by Unified Modelling Language (UML) class diagram and sequence diagram. The work in this thesis shows that High Level Petri Nets (HLPN) can represent both of them in one model. Using a proper model in the representation of the software specifications is essential to generate proper test cases. The research presented in this thesis introduces novel and automated test cases generation techniques that can be used within a software sys- tem design testing. Furthermore, this research introduces e cient au- tomated technique to generate a formal software system model (HLPN) from semi-formal models (UML diagrams). The work in this thesis con- sists of four stages: (1) generating test cases from class diagram and Object Constraint Language (OCL) that can be used for testing the software system static specifications (the structure) (2) combining class diagram, sequence diagram and OCL to generate test cases able to cover both static and dynamic specifications (3) generating HLPN automat- ically from single or multi sequence diagrams (4) generating test cases from HLPN. The test cases that are generated in this work covered the structural and behavioural of the software system model. In first two phases of this work, the class diagram and sequence diagram are decomposed to nodes (edges) which are linked by Classes Hierarchy Table (CHu) and Edges Relationships Table (ERT) as well. The linking process based on the classes and edges relationships. The relationships of the software system components have been controlled by consistency checking technique, and the detection of these relationships has been automated. The test cases were generated based on these interrelationships. These test cases have been reduced to a minimum number and the best test case has been selected in every stage. The degree of similarity between test cases is used to ignore the similar test cases in order to avoid the redundancy. The transformation from UML sequence diagram (s) to HLPN facilitates the simpli cation of software system model and introduces formal model rather than semi-formal one. After decomposing the sequence diagram to Combined Fragments, the proposed technique converts each Combined Fragment to the corresponding block in HLPN. These blocks are con- nected together in Combined Fragments Net (CFN) to construct the the HLPN model. The experimentations with the proposed techniques show the effectiveness of these techniques in covering most of the software system specifications.

Mobile applications are developed to run on fast-evolving platforms, such as Android or iOS. Respective mobile devices are heterogeneous concerning hardware (e.g., sensors, displays, communication interfaces) and software, especially operating system functions. Software vendors cope with platform evolution and various hardware configurations by abstracting from these variable assets. However, they cannot be sure about their assumptions on the inner conformance of all device parts and that the application runs reliably on each of them—in consequence, comprehensive testing is required. Thereby, in testing, variability becomes tedious due to the large number of test cases required to validate behavior on all possible device configurations. In this paper, we provide remedy to this problem by combining model-based testing with variability concepts from Software Product Line engineering. For this purpose, we use feature-based test modeling to generate test cases from variable operational models for individual application configurations and versions. Furthermore, we illustrate our concepts using the commercial mobile application “runtastic” as example application.

Context. Human oracle cost, the cost associated in estimating the correctness of the output for the given test inputs is manually evaluated by humans and this cost is significant and is a concern in the software test data generation field. This study has been designed in the context to assess metrics that might predict human oracle cost. Objectives. The major objective of this study is to address the human oracle cost, for this the study identifies the metrics that are good predictors of human oracle cost and can further help to solve the oracle problem. In this process, the identified suitable metrics from the literature are applied on the test input, to see if they can help in predicting the correctness of the output for the given test input. Methods. Initially a literature review was conducted to find some of the metrics that are relevant to the test data. Besides finding the aforementioned metrics, our literature review also tries to find out some possible code metrics that can be ap- plied on test data. Before conducting the actual experiment two pilot experiments were conducted. To accomplish our research objectives an experiment is conducted in the BTH university with master students as sample population. Further group interviews were conducted to check if the participants perceive any new metrics that might impact the correctness of the output. The data obtained from the experiment and the interviews is analyzed using linear regression model in SPSS suite. Further to analyze the accuracy vs metric data, linear discriminant model using SPSS pro- gram suite was used. Results.Our literature review resulted in 4 metrics that are suitable to our study. As our test input is HTML we took HTML depth, size, compression size, number of tags as our metrics. Also, from the group interviews another 4 metrics are drawn namely number of lines of code and number of &lt;div&gt;, anchor &lt;a&gt; and paragraph &lt;p&gt; tags as each individual metric. The linear regression model which analyses time vs metric data, shows significant results, but with multicollinearity effecting the result, there was no variance among the considered metrics. So, the results of our study are proposed by adjusting the multicollinearity. Besides, the above analysis, linear discriminant model which analyses accuracy vs metric data was conducted to predict the metrics that influences accuracy. The results of our study show that metrics positively correlate with time and accuracy. Conclusions. From the time vs metric data, when multicollinearity is adjusted by applying step-wise regression reduction technique, the program size, compression size and &lt;div&gt; tag are influencing the time taken by sample population. From accuracy vs metrics data number of &lt;div&gt; tags and number of lines of code are influencing the accuracy of the sample population.

The article identifies the challenges during the system and specifically the software development process for safety critical electro-hydraulic control systems by using the example of the hydrostatic driveline with a four speed transmission of a feeder mixer. An optimized development approach for mobile machinery has to fulfill all the requirements according to the Machinery Directive 2006/42/EC, considering functional safety, documentation and testing requirements from the beginning and throughout the entire machine life cycle. The functionality of the drive line control could be verified in advance of the availability of a prototype by using a “software-in-the-loop” development approach, based on a MATLAB/SIMULINK model of the drive line in connection with the embedded software.

The main part in the testing of the software is in the generation of test cases suitable for software system testing. The quality of the test cases plays a major role in reducing the time of software system testing and subsequently reduces the cost. The test cases, in model de- sign stages, are used to detect the faults before implementing it. This early detection offers more flexibility to correct the faults in early stages rather than latter ones. The best of these tests, that covers both static and dynamic software system model specifications, is one of the chal- lenges in the software testing. The static and dynamic specifications could be represented efficiently by Unified Modelling Language (UML) class diagram and sequence diagram. The work in this thesis shows that High Level Petri Nets (HLPN) can represent both of them in one model. Using a proper model in the representation of the software specifications is essential to generate proper test cases. The research presented in this thesis introduces novel and automated test cases generation techniques that can be used within a software sys- tem design testing. Furthermore, this research introduces e cient au- tomated technique to generate a formal software system model (HLPN) from semi-formal models (UML diagrams). The work in this thesis con- sists of four stages: (1) generating test cases from class diagram and Object Constraint Language (OCL) that can be used for testing the software system static specifications (the structure) (2) combining class diagram, sequence diagram and OCL to generate test cases able to cover both static and dynamic specifications (3) generating HLPN automat- ically from single or multi sequence diagrams (4) generating test cases from HLPN. The test cases that are generated in this work covered the structural and behavioural of the software system model. In first two phases of this work, the class diagram and sequence diagram are decomposed to nodes (edges) which are linked by Classes Hierarchy Table (CHu) and Edges Relationships Table (ERT) as well. The linking process based on the classes and edges relationships. The relationships of the software system components have been controlled by consistency checking technique, and the detection of these relationships has been automated. The test cases were generated based on these interrelationships. These test cases have been reduced to a minimum number and the best test case has been selected in every stage. The degree of similarity between test cases is used to ignore the similar test cases in order to avoid the redundancy. The transformation from UML sequence diagram (s) to HLPN facilitates the simpli cation of software system model and introduces formal model rather than semi-formal one. After decomposing the sequence diagram to Combined Fragments, the proposed technique converts each Combined Fragment to the corresponding block in HLPN. These blocks are con- nected together in Combined Fragments Net (CFN) to construct the the HLPN model. The experimentations with the proposed techniques show the effectiveness of these techniques in covering most of the software system specifications.

Die Dissertationsschrift diskutiert die Machbarkeit einer modularen, portierbaren Middleware für die automatisierte Ausführung und Dokumentation von Software-Tests mit einer durchgehenden Nachverfolgbarkeit von der Anforderungsspezifikation bis hin zur Dokumentation der Testergebnisse. Es werden die Eigenschaften und Probleme bestehender Testautomatisierungslösungen analysiert und dargelegt. Unter Berücksichtigung dieser Probleme werden neuartige Lösungsansätze entwickelt. Die Neuheiten dieser Arbeit sind der modulare Aufbau der Middleware mit einer unproblematischen Portierbarkeit auf neue Testsysteme in Verbindung mit dem neu erarbeiteten Speicherformat für die Testergebnisse. Es wird die Möglichkeit aufgezeigt, Testfälle sowohl mit graphischer als auch textueller Eingabe zu bearbeiten. Neben den typischen Einsatzbereichen, wie zum Beispiel Hardware In The Loop-Tests (HIL), werden auch weitere Felder, vom Modul-Test bis zum Bandende-Test, abgedeckt. Das Speicherformat der Testergebnisse ermöglicht die Ablage aller wichtigen Informationen zu den Tests, ist flexibel erweiterbar und erlaubt die Generierung von Testreports in unterschiedlichen Zielformaten. Ein weiterer zentraler Punkt ist der automatisierte Austausch von Informationen und Testergebnissen mit verschiedenen Requirementsmanagement-Systemen sowie eine nahtlose Integration in vorhandene Versionsmanagement-Systeme. Basierend auf den theoretischen Ausarbeitungen wurde eine modulare, portierbare Middleware in Form des modularen aufgebauten Testautomatisierungs-Frameworks (modTF) umgesetzt. Anhand der dabei gesammelten Erfahrungen und der Ergebnisse der praktischen Erprobung werden die Vorteile des Frameworks gezeigt. / These PhD discusses the feasibility of a modular, portable middleware for automated execution and documentation of software tests with a continuous traceability from the requirements to the test results. The properties and the problems of existing test automation solutions are analyzed and presented. Based on the problems novel solutions are developed. The novelties of this PhD are the modular structure of the middleware with the easy portability to new test systems in cooperation with the novel storage format for the test results. The possibility for a graphical and textual description of the test cases is shown. Beside the typical applications like Hardware In The Loop tests (HIL) also applications from the module test to the line end test are include. The storage format for the results allows the storage of all needed information according to the test cases in one file. The format is flexible and extendable. The generation of test reports in different target formats is possible. Another imported point is the automated exchange of information and test results with different requirements management systems and the seamless integration in existing version management systems. By means of the collected experiences and the results of the practical proving the advantages of the framework are shown.