Testing is an important activity in software development used to ensure the quality of a product. In industrial practice when developing control software, such as Programmable Logic Controllers (PLCs), software testing is, in some cases, neglected during the development process. There is little support for automated testing on PLC software and the creation of test cases is hence mainly a manual activity that can be both costly and time-consuming. In this thesis, we propose a solution for an integrated tool chain for automating the testing process for IEC 61131-3 PLC software in the integrated development environment (IDE) of CODESYS (Controller Development System), using a combinatorial testing tool (i.e., SEAFOX) for test case generation. Further, we measure the applicability and usefulness of the integrated tool chain in terms of the decision coverage (i.e, Observable Decision Coverage) achieved by the generated test cases. For this purpose, several available tools for testing in CODESYS IDE have been examined as well as solutions on how to integrate the chosen tool (i.e., CfUnit) with SEAFOX. After a solution for the tool chain was implemented and integrated, an experiment was conducted to measure decision coverage for the generated test cases on nine industrial programs using Pairwise, Base choice and Random combinatorial testing techniques. The result of our thesis is a fully integrated tool chain consisting of CODESYS, CfUnit, SEAFOX, and a python script used to combine these, and where the SEAFOX tool was further extended to support additional standard data types. This tool chain can be used to create test cases, generate new input values for these using SEAFOX, automate the additional test case creation and integration with CfUnit in CODESYS IDE, as well as for test execution. The results of the experiment show that test cases generated by this tool chain achieved on average 90% decision coverage or higher regardless of the combinatorial technique used, with the overall average level being 94%. Interestingly enough, Random testing scored higher than both Pairwise and Base choice testing. We present our integrated tool chain as a contribution to the automation of test creation and execution for industrial control software in CODESYS IDE. However, we identified several limitations with testing CODESYS timer blocks and when executing larger test suites that can make it problematic to fully test PLC programs in a real-time simulation environment. These challenges should be investigated further by both researchers and practitioners.

This document's objective is to propose a set of methods for improvement of the organization and methodology of software testing and subsequent application of those in a real environment of the Siemens CZ company. The first part discusses the development models in terms of testing segment and the following section focuses specifically on the methodology of agile testing and development as a whole, including applicable metrics for evaluating various aspects of software exiting the testing phase. The third major section lets the reader get acquainted with the Siemens organization and methods of their software testing and by analyzing real-world project it assesses the strengths and weaknesses of the particular testing process and its management. The practical part of this thesis lies in the design and implementation of a plug-in for the JIRA environment. This tool is able to identify and quantify the long term status of the project phases of agile development and testing in a clear and interactive way, while the outputs are represented by clearly defined problems that can be more easily treated, as one can argue further progress.