A case study for the tool LBTest illustrating benets and limitations of the tool along the terms of usability, results and costs. The study shows the use of learning based testing on a safety critical application in the avionics industry. While requiring the user to have the oretical knowledge of the tools inner workings, the process of using the tool has benefits in terms of requirement analysis and the possibility of finding design and implementation errors in both the early and late stages of development

In this thesis we investigate the feasibility of learning-based testing (LBT) as a viable testing methodology for reactive systems. In LBT, a large number of test cases are automatically generated from black-box requirements for the system under test (SUT) by combining an incremental learning algorithm with a model checking algorithm. The integration of the SUT with these algorithms in a feedback loop optimizes test generation using the results from previous outcomes. The verdict for each test case is also created automatically in LBT. To realize LBT practically, existing algorithms in the literature both for complete and incremental learning of finite automata were studied. However, limitations in these algorithms led us to design, verify and implement new incremental learning algorithms for DFA and Kripke structures. On the basis of these algorithms we implemented an LBT architecture in a practical tool called LBTest which was evaluated on pedagogical and industrial case studies. The results obtained from both types of case studies show that LBT is an effective methodology which discovers errors in reactive SUTs quickly and can be scaled to test industrial applications. We believe that this technology is easily transferrable to industrial users because of its high degree of automation. / <p>QC 20130312</p>

Learning-based testing (LBT) is a relatively new testing paradigm which automatically generates test cases for black-box testing of a system under test (SUT). LBT uses machine learning to model a SUT, and combines this with model-based testing. This thesis uses LBTest, a research tool created at CSC, in order to apply LBT on a new architectural style of distributed systems called microservices. Two new approaches to using LBT have been implemented to test a commercial product for counter-party credit risk. One approach is to monitor the internal processes to extract the states of the software. The second is based on fault injection on the software level. Errors have been found during the fault injection approach. Lastly, some general recommendations are given on how to implement LBT. / Inlärningsbaserad testning (LBT) är en relativt ny testningsparadigm som automatiskt genererar testfall för black-box-testning av ett system under test (SUT). LBT använder sig av maskininlärning för att modellera ett SUT, och kombinerar det med modellbaserad testning. I det här examensarbetet används LBTest, ett forskningverktyg skapat på CSC, för att applicera LBT på microservices. Två nya tillvägagångssätt att använda LBT på har implementerats för att testa en kommersiell produkt för uträkning av kreditrisk hos motparter. Ett tillvägagångssätt är att avlyssna interna processer för att extrahera tillstånden hos mjukvaran. Det andra tillvägagångssättet är baserat på felinjicering på mjukvarunivå. Fel har hittats med hjälp av felinjiceringstillvägagångssättet. Som avslutning ges rekommendationer till hur LBT implementeras.

Learning-based testing is a testing paradigm that combines model-based testing with machine learning algorithms to automate the modeling of the SUT, test case generation, test case execution and verdict construction. A tool that implements LBT been developed at the CSC school at KTH called LBTest. LBTest utilizes machine learning algorithms with off-the-shelf equivalence- and model-checkers, and the modeling of user requirements by propositional linear temporal logic. In this study, it is be investigated whether LBT may be suitable for testing a micro bus architecture within an embedded telecommunication device. Furthermore ideas to further automate the testing process by designing a data model to automate user requirement generation are explored. / Inlärningsbaserad testning är en testningsparadigm som kombinerar model-baserad testning med maskininlärningsalgoritmer för att automatisera systemmodellering, testfallsgenering, exekvering av tester och utfallsbedömning. Ett verktyg som är byggt på LBT är LBTest, utvecklat på CSC skolan på KTH. LBTest nyttjar maskininlärningsalgoritmer med färdiga ekvivalent- och model-checkers, och modellerar användarkrav med linjär temporal logik. I denna studie undersöks det om det är lämpat att använda LBT för att testa en mikrobus arkitektur inom inbyggda telekommunikationsenheter. Utöver det undersöks även hur testprocessen skulle kunna ytterligare automatiseras med hjälp av en data modell för att automatisera generering av användarkrav.