Functional Test of any given system is used to verify that the top level system is performing according to the specifications and all of the sub systems, i.e., hardware, software, inputs, outputs and sensors are operating properly. The term System is used here in context of any device or equipment consisting of hardware, software, sensors, virtual sensors and inputs / outputs. One of the examples of such a system is a semiconductor fabrication equipment. There have only been a few approaches that are used to perform the functional test of a system. Engineers typically develop custom test equipment to interface with the system under test and determine proper functioning of all the sub systems and behavior of overall system. In another method, domain experts, utilize the software of the system under test, and manually actuate / stimulate sub systems and then observe and record outcomes to determine whether the system exhibited correct behavior. A novel solution of a reusable functional test automation framework is developed. The framework interfaces with the system under test via the exposed API, and allow domain experts with minimal or no programming background to create test suites to methodically test overall functionality of the system. / text

When exploring a verification approach for Komatsu Forest's control system regarding their forest machines (Valmet), the context of soft real-time systems is illuminated. Because of the nature of such context, the verification process is based on empirical corroboration of requirements fulfillment rather than being a formal proving process. After analysis of the literature with respect to the software testing field, two paradigms have been defined in order to highlight important concepts for soft real-time systems. The paradigms are based on an abstract stimuli/response model, which conceptualize a system with inputs and output. Since the system is perceived as a black box, its internal details are hidden and thus focus is placed on a more abstract level. The first paradigm, the “input data paradigm”, is concerned about what data to input to the system. The second paradigm, the “input data mechanism paradigm” is concerned about how the data is sent, i.e. the actual input mechanism is focused. By specifying different dimensions associated with each paradigm, it is possible to define their unique characteristics. The advantage of this kind of theoretical construction is that each paradigm creates an unique sub-field with its own problems and techniques. The problems defined for this thesis is primarily focused on the input data mechanism paradigm, where devised dimensions are applied. New verification techniques are deduced and analyzed based on general software testing principles. Based on the constructed theory, a test system architecture for the control system is developed. Finally, an implementation is constructed based on the architecture and a practical scenario. Its automation capability is then assessed. The practical context for the thesis is a new simulator under development. It is based upon LabVIEW and PXI technology and handles over 200 I/O. Real machine components are connected to the environment, together with artificial components that simulate the engine, hydraulic systems and a forest. Additionally, physical control sticks and buttons are connected to the simulator to enable user testing of the machine being simulated. The results associated with the thesis is first of all that usable verification techniques were deduced. Generally speaking, some of these techniques are scalable and are possible to apply for an entire system, while other techniques may be appropriate for selected subsets that needs extra attention. Secondly, an architecture for an automated test system based on a selection of techniques has been constructed for the control system. Last but not least, as a result of this, an implementation of a general test system has been possible and successful. The implemented test system is based on both C# and LabVIEW. What remains regarding the implementation is primarily to extend the system to include the full scope of features described in the architecture and to enable result analysis. / Då verifikationstekniker för Komatu Forests styrsystem utreds angående Valmet skogsmaskiner, hamnar det mjuka realtidssystemkontextet i fokus. Ett sådant kontext antyder en process där empirisk styrkning av kravuppfyllande står i centrum framför formella bevisföringsprocesser. Efter en genomgång och analys av litteratur för mjukvarutestområdet har två paradigmer definierats med avsikten att belysa viktiga concept för mjuka realtidssystem. Paradigmerna är baserade på en abstrakt stimuli/responsmodell, som beskriver ett system med in- och utdata. Eftersom detta system betraktas som en svart låda är inre detaljer gömda, vilket medför att fokus hamnar på ett mer abstrakt plan. Det första paradigmet benämns som “indata-paradigmet” och inriktar sig på vilket data som skickas in i systemet. Det andra paradigmet går under namnet “indatamekanism-paradigmet” och behandlar hur datat skickas in i systemet, dvs fokus placeras på själva inskickarmekanismen. Genom att definiera olika dimensioner för de två paradigmen, är det möjligt att beskriva deras utmärkande drag. Fördelen med att använda denna teoretiska konstruktion är att ett paradigm skapar ett eget teoriområde med sina egna frågeställningar och tekniker. De problem som definierats för detta arbete är främst fokuserade på indatamekanism-paradigmet, där framtagna dimensioner tillämpas. Nya verifikationstekniker deduceras och analyseras baserat på generella mjukvarutestprinciper. Utifrån den skapade teorin skapas en testsystemarkitektur för kontrollsystemet. Sedan utvecklas ett testsystem baserat på arkitekturen samt ett praktiskt scenario med syftet att utreda systemets automationsgrad. Den praktiska miljön för detta arbete kretsar kring en ny simulator under utveckling. Den är baserad på LabVIEW och PXI-teknik och hanterar över 200 I/O. Verkliga maskinkomponenter ansluts till denna miljö tillsammans med konstgjorda komponenter som simulerar motorn, hydralik samt en skog. Utöver detta, ansluts styrspakar och knappar för att möjliggöra användarstyrning av maskinen som simuleras. Resultatet förknippat med detta arbete är för det första användbara verifikationstekniker. Man kan generellt säga att några av dessa tekniker är skalbara och därmed möjliga att tillämpa för ett helt system. Andra tekniker är ej skalbara, men lämpliga att applicera på en systemdelmängd som behöver testas mer utförligt. För det andra, en arkitektur har konstruerats för kontrollsystemet baserat på ett urval av tekniker. Sist men inte minst, som en följd av ovanstående har en lyckad implementation av ett generellt testsystem utförts. Detta system implementerades med hjälp av C# och LabVIEW. Det som återstår beträffande implementationen är att utöka systemet så att alla funktioner som arkitekturen beskriver är inkluderade samt att införa resultatanalys.

In recent years, automotive engineering becomes more challenging and complex than previous years because of the innovation that transformed the cars powertrain system. By the quick developing of the automotive systems and increasing the consumer demand for Hybrid Vehicles (HV), Electric Vehicles (EV) and Autonomous cars, authenticity and security of engine systems are an essential part. The Hybrid Electric Vehicle structure contained several system sections like Electric Motor, Inverter, DC-DC Converter, etc. For testing this device, OEM’s are providing a special automotive standard norm (these norms are individually defining by automotive companies as LV124/123/148) which used for simulates variety of the electrical disturbances and monitoring the system behavior. This standard profile applies, simulates and records by EM Autowave system. Electrical tests have been done for a long time manually. Manual testing takes a huge amount of time, money and human resources. Based on the facts, the thesis purpose is to provide the automated system test for supporting test case execution on the HEV test bench, and offline classification independent of the test bench. This scope of the project will be developed by adapting automation library which is going to equip new features for automation system tool. On the other hand, in order to conform the test simulator and their communication system, a device file is implemented for loading and testing the electrical profiles on the test bench system. In addition, automated classification and evaluation used Matlab functions and the test result will be provided as plots which makes differential discovering very easier.

Automated Test Generation (ATG) has been successfully applied in many domains. For the modeling and simulation language Simulink, there has been research on developing tools for ATG with promising results. However, most tools developed as part of academic research and are not publicly available, or severely limited in their ability to be integrated into an industrial workflow. There are commercial ATG tools for Simulink, with Simulink Design Verifier (SLDV) as the de-facto standard tool. For this thesis, we perform an empirical comparison of manual tests to those generated by SLDV. For the comparison, we used 180 components from the propulsion control software developed by our industry partner. All except two components are compatible for test generation to some extent. The majority of components are partially compatible, requiring block replacement or stubbing. Approximation of floating-point numbers is the primary reason for block replacement, which can be performed automatically by SLDV. Two components were incompatible, and 14 required full stubbing of blocks. Using a pre-processing step, the generated tests achieve similar coverage as the manual tests. We performed a Mann–Whitney U test with the hypothesis that the generated tests achieve higher coverage than the manual tests. There are no statistically significant differences for either decision coverage (0.0719), or condition coverage (0.8357). However, for Modified Condition/Decision Coverage, the generated tests achieve higher coverage, and the difference is significant (0.0027). The limitations of ATG were explored by looking at the cases where the generated tests achieved lower coverage than the manual test. We found that the use of floating-point arithmetic and temporal logic increases the time required for test generation, and causes the analysis to hit the time limit. The test generation does not support all custom S-functions and perform stubbing of these blocks. This made the tool unable to reason about persistent storage. Configuration constants have limited support, which was the reason for the coverage difference in three cases. We have concluded that while much effort is required for custom tooling and initial setup, ATG can prove useful for early fault detection in an industrial workflow. ATG would prove especially useful in an automated continuous integration workflow for integration-level conformance testing.

Accuracy and efficiency are extremely critical factors for large real-time control applications. A small oversight can cause catastrophic failure of a real-time system. Thus, these applications have to be tested meticulously to prevent any catastrophe that might occur. But, testing these applications exhaustively is not tractable, mainly due to the inherent complexity of the applications and also the huge amount of inputs and outputs that these applications involve. In order to save valuable amounts of time and resources, automated testing is imperative. Also, quantitative metrics have to be provided that assess the existing quality of the system and help increase the confidence in the user towards the software. However, to improve the overall quality of the software, additional focused testing needs to be done. The work in this thesis involves providing specific test suggestions that help the user conduct thorough and precise domain tests based on the knowledge of the various parameters used in previous test runs. The information about the defective portions of the input domain is provided by dividing the input range into percentiles, which is referred to here as bucketing. The goal is to expose the exact inputs causing the defects and the range of inputs that have been lightly tested or left untested during previous tests. A Reliability Analysis Test Tool (RATT) was developed to implement these test suggestions.

Da Prüfungen und Tests häufig dazu dienen, den Zugang zu Bildungsprogrammen zu steuern und die Grundlage zur Abschlussvergabe am Ende von Bildungsprogrammen bilden, ist ihre Fairness und Validität von größter Bedeutung. Ein kontrovers diskutierter Aspekt standardisierter Tests ist die Verwendung von Zeitlimits. Unabhängig davon ob eine Testadministration Zeitdruck hervorrufen soll oder nicht, sollten Testentwickler:innen in die Lage versetzt werden, den Zeitdruck einer Testadministrationen explizit gestalten zu können. Zu diesem Zweck schlägt van der Linden (2011a, 2011b) einen Ansatz zur Kontrolle des Zeitdrucks von Tests in der automatisierten Testhefterstellung (ATA) unter Verwendung von Mixed Integer Linear Programming und eines lognormalen Antwortzeitmodells vor. Dabei hat der Ansatz von van der Linden jedoch eine zentrale Limitation: Er ist auf das zwei-parametrische lognormale Antwortzeitmodell beschränkt, das gleiche Geschwindigkeits-Sensitivitäten (d.h. Faktorladungen) für alle Items annimmt. Diese Arbeit zeigt, dass ansonsten parallele Testhefte mit unterschiedlichen Geschwindigkeits-Sensitivitäten für bestimmte Testteilnehmende unfair sind. Darüber hinaus wird eine Erweiterung des van der Linden-Ansatzes vorgestellt, die unterschiedliche Geschwindigkeits-Sensitivitäten von Items in ATA berücksichtigt. Weiter wird diskutiert, wie Testhefte mit identischen, aber unterschiedlich angeordneten Items zu Fairness-Problemen aufgrund von Item-Positionseffekten führen können und wie dies verhindert werden kann. Die vorliegende Arbeit enthält zusätzlich Anleitungen zur Verwendung des R-Pakets eatATA für ATA und zur Verwendung von Stan und rstan für Bayesianische hierarchische Antwortzeitmodellierung. Abschließend werden Alternativen, praktische Implikationen und Grenzen der vorgeschlagenen Ansätze diskutiert und Vorschläge für zukünftige Forschungsthemen gemacht. / As examinations and assessments are often used to control access to educational programs and to assess successful participation in an educational program, their fairness and validity is of great importance. A controversially discussed aspect of standardized tests is setting time limits on tests and how this practice can result in test speededness. Regardless of whether a test should be speeded or not, being able to deliberately control the speededness of tests is desirable. For this purpose, van der Linden (2011a, 2011b) proposed an approach to control the speededness of tests in automated test assembly (ATA) using mixed integer linear programming and a lognormal response time model. However, the approach by van der Linden (2011a, 2011b) has an important limitation, in that it is restricted to the two-parameter lognormal response time model which assumes equal speed sensitivities (i.e., factor loadings) across items. This thesis demonstrates that otherwise parallel test forms with differential speed sensitivities are indeed unfair for specific test-takers. Furthermore, an extension of the van der Linden approach is introduced, which incorporates speed sensitivities in ATA. Additionally, test speededness can undermine the fairness of a test if identical but differently ordered test forms are used. To prevent that the score of test-takers depends on whether easy or difficult items are located at the end of a test form, it is proposed that the same, most time intensive items should be placed at the end of all test forms. The thesis also provides introductions and tutorials on using the R package eatATA for ATA and using Stan and rstan for Bayesian hierarchical response time modeling. Finally, the thesis discusses alternatives, practical implications, and limitations of the proposed approaches and provides an outlook on future related research topics.

A modern truck contains a large number of functionalities implemented in its electronics system. Thus, testing all of these functions employs a considerable effort. The execution of tests against the system has been automated for a long time. Unfortunately, most of the test is still designed manually these days. This manual test design is sometimes not comprehensive enough to cover all possible scenarios within a complex system. At the moment, there is also a growing trend in the development process based on the model. Furthermore, model-based software can handle events and signal behaviour more robustly[1]. This thesis investigates the technique in model-based testing. The study evaluates the requirement modelling and automated abstract test generation of model-based testing over the existing testing method. A cause-effect graph is utilized for the modelling in Matlab Simulink tool with DesignVerifier feature. The case study is the truck dosing system in Scania. The results are the following. The temporal and static requirements modelling are capable of being modelled using the cause-effect graph in Matlab Simulink. Compared to the traditional method, the MBT method can achieve higher requirement coverage and more rigorous test with optimized test case generation. The MBT method also has a rapid test case generation time suitable for quick design iteration. However, the total test development time (including test case generation time) of using MBT is 12.5% higher than the manual method. Using a model-based platform like Matlab Simulink is recommended to assist the manual testing, not to replace the test flow entirely with the current research state. / En modern truck innehåller ett stort antal funktioner implementerade i dess elektroniksystem. Att testa alla dessa funktioner kräver därför en avsevärd ansträngning. Utförandet av tester mot systemet har varit automatiserat under lång tid. Tyvärr är det mesta av testet fortfarande utformat manuellt nu för tiden. Denna manuella testdesign är ibland inte tillräckligt omfattande för att täcka alla möjliga scenarier inom ett komplext system. För tillfället finns det också en växande trend i utvecklingsprocessen utifrån modellen. Dessutom kan modellbaserad programvara hantera händelser och signalbeteende mer robust[1]. Detta examensarbete undersöker tekniken i modellbaserad testning. Studien utvärderar kravmodellering och automatiserad abstrakt testgenerering av modellbaserad testning över den befintliga testmetoden. En cause-effect graph används för modelleringen i Matlab Simulink-verktyget med Design Verifier-funktionen. Fallstudien är lastbilens doseringssystem i Scania. Resultaten är följande. Den tidsmässiga och statiska kravmodelleringen kan modelleras med hjälp av cause-effect graph i Matlab Simulink. Jämfört med den traditionella metoden kan MBT-metoden uppnå högre kravtäckning och mer rigorösa test med optimerad testfallsgenerering. MBT-metoden har också en snabb genereringstid för testfall som är lämplig för snabb designiteration. Den totala testutvecklingstiden (inklusive genereringstid för testfall) för att använda MBT är 12,5% högre än den manuella metoden. Att använda en modellbaserad plattform som Matlab Simulink rekommenderas för att underlätta den manuella testningen, inte för att ersätta testflödet helt med det aktuella forskningsläget.

The traditional radiation testing of space electronics has been used for more than fifty years to support the radiation hardness assurance. Its typical goal is to ensure reliable operation of the spacecraft in the harsh environment of space. This PhD research looks into the radiation testing from a different perspective; the goal is to develop radiation testing methods that are focused not only on the reliability of the components but also on a continuous radiation-induced degradation of their performance. Such data are crucial for the understanding of the impact of radiation on the measurement uncertainty of data acquisition systems onboard research space missions.

The software industry is expanding at a rapid rate. To keep up with the fast-growing and ever-changing technologies, it has become necessary to produce high-quality software in a short time and at an affordable cost. This research aims to demonstrate to Java developers the use of Automated Test Case Generation (ATCG) tools by presenting the tools that exist today, their usefulness, and their effectiveness. The main focus is on the automated testing tools for the Java industry, which can help developers achieve their goals faster and make better software. Moreover, the discussion covers the availability, features, prerequisites, effectiveness, and limitations of the automated testing tools. Among these tools, the most widely used are Evosuite, JUnit, TestNG, and Selenium. Each tool has its advantages and purpose. Furthermore, these ATCG-tools were compared to provide a clear picture to Java developers, answer the research questions, and show strengths and limitations of each selected tool. Results show that there is no single ultimate tool that can do all kinds of testing independently. It all depends on what the developer aims to achieve. If one tool is good at generating unit test cases for Java classes, another tool is good at testing the code security through penetration testing. Therefore, the Java developers may choose a tool/s based on their requirements. This study has revealed captivating findings regarding the ATCG-tools, which ought to be explored in the future.