Testing is one of the most important parts of software development. It is used to ensure that the software is of a certain quality. In many situations it is a time consuming task that is manually performed and error prone. In the last couple of years a wide range of techniques for automated test generation have been explored with the goal of performing more efficient testing both in terms of cost and time. Many of these techniques are using combinatorial methods to explore different combinations of test inputs. Base Choice (BC) is a combinatorial method that has been shown to be efficient and effective at detecting faults. However, it is not very well studied how BC compares to manual testing performed by industrial engineers with experience in software testing. This thesis presents the results of a case study comparing BC testing with manual testing. We investigate the quality of manual tests and tests created using BC strategy in terms of decision coverage, fault detection capability and cost efficiency (in terms of number of test cases). We used recently developed industrial programs written in the IEC 61131-3 FBD language, a popular programming language for embedded software using programmable logic controllers. For generating tests using BC we used the Combinatorial Testing Tool (CTT) developed at M¨alardalen University. The results of this thesis show that manual tests performed significantly better than BC generated tests in terms of achieved decision coverage and fault detection. In average manually written tests achieved 97.38% decision coverage while BC tests suites only achieved 83.10% decision coverage. In fault detection capabilities, manual test suites found in average 88.90% of injected faults compared to 69.53% fault detection by BC generated test suites. We also found that manual tests are slightly shorter in terms of number of tests compared to BC testing. We found that the use of BC is heavily affected by the choice of the base values chosen by the tester. By using more precise base choice values in BC testing may have yielded different results in terms of decision coverage and fault detection.
| Identifer | oai:union.ndltd.org:UPSALLA1/oai:DiVA.org:mdh-40522 |
| Date | January 2018 |
| Creators | Eklund, Simon |
| Publisher | Mälardalens högskola, Akademin för innovation, design och teknik |
| Source Sets | DiVA Archive at Upsalla University |
| Language | English |
| Detected Language | English |
| Type | Student thesis, info:eu-repo/semantics/bachelorThesis, text |
| Format | application/pdf |
| Rights | info:eu-repo/semantics/openAccess |
Page generated in 0.0016 seconds