Optimizing compilers are large and complex systems, potentially consisting of millions of lines of code. As a result, the implementation of an industry standard compiler can contain several serious bugs. This is alarming given the critical part that software plays in modern society. Therefore, we investigate type-directed program generation and shrinking for the testing of compilers for imperative programming languages. We implement a type-directed generator and shrinker for a subset of C in Haskell. The generator is capable of generating type-correct programs that contain variable mutations and challenging program constructs, such as while-statements and if statements. We evaluate the quality of the generated programs by measuring statement coverage of a reference interpreter and by comparing the results to reference test suite. The results show that the generated programs can surpass the test suite in terms of quality by one percentage point of coverage. To test the bug finding capabilities of the generator and the shrinker, we use them to test seven interpreters and manage to identify a total of seven bugs. Furthermore, we test the same interpreters with the reference test suite and find only three bugs. The implications of the results lead us to question the reliability of code coverage as a measure of quality for compiler test suites. Finally, we conclude that type-directed generation and shrinking is a powerful method that can be used for the testing of compilers for imperative programming languages.
| Identifer | oai:union.ndltd.org:UPSALLA1/oai:DiVA.org:mdh-39673 |
| Date | January 2018 |
| Creators | Samuel, Bodin, Joel, Söderman |
| Publisher | Mälardalens högskola, Akademin för innovation, design och teknik, Mälardalens högskola, Akademin för innovation, design och teknik |
| Source Sets | DiVA Archive at Upsalla University |
| Language | English |
| Detected Language | English |
| Type | Student thesis, info:eu-repo/semantics/bachelorThesis, text |
| Format | application/pdf |
| Rights | info:eu-repo/semantics/openAccess |
Page generated in 0.0015 seconds