Many models have been proposed for software reliability prediction, but none of these models could capture a necessary amount of software characteristic. We have proposed a mixed approach using both analytical and data driven models for finding the accuracy in reliability prediction involving case study. This report includes qualitative research strategy. Data is collected from the case study conducted on three different companies. Based on the case study an analysis will be made on the approaches used by the companies and also by using some other data related to the organizations Software Quality Assurance (SQA) team. Out of the three organizations, the first two organizations used for the case study are working on reliability prediction and the third company is a growing company developing a product with less focus on quality. Data collection was by the means of interviewing an employee of the organization who leads a team and is in the managing position for at least last 2 years. / svra06@student.bth.se

To improve quality of software systems, change-based fault prediction and scope-bounded checking have been used to predict or detect faults during software development. In fault prediction, changes to program source code, such as added lines or deleted lines, are used to predict potential faults. In fault detection, scope-bounded checking of programs is an effective technique for finding subtle faults. The central idea is to check all program executions up to a given bound. The technique takes two basic forms: scope-bounded static checking, where all bounded executions of a program are transformed into a formula that represents the violation of a correctness property and any solution to the formula represents a counterexample; or scope-bounded testing where a program is tested against all (small) inputs up to a given bound on the input size. Although the accuracies of change-based fault prediction and scope-bounded checking have been evaluated with experiments, both of them have effectiveness and efficiency limitations. Previous change-based fault predictions only consider the code modified by a change while ignoring the code impacted by a change. Scope-bounded testing only concerns the correctness specifications, and the internal structure of a program is ignored. Although scope-bounded static checking considers the internal structure of programs, formulae translated from structurally complex programs might choke the backend analyzer and fail to give a result within a reasonable time. To improve effectiveness and efficiency of these approaches, we introduce local semantic analysis into change-based fault prediction and scope-bounded checking. We use data-flow analysis to disclose internal dependencies within a program. Based on these dependencies, we identify code segments impacted by a change and apply fault prediction metrics on impacted code. Empirical studies with real data showed that semantic analysis is effective and efficient in predicting faults in large-size changes or short-interval changes. While generating inputs for scope-bounded testing, we use control-flow to guide test generation so that code coverage can be achieved with minimal tests. To increase the scalability of scope-bounded checking, we split a bounded program into smaller sub-programs according to data-flow and control-flow analysis. Thus the problem of scope-bounded checking for the given program reduces to several sub-problems, where each sub-problem requires the constraint solver to check a less complex formula, thereby likely reducing the solver’s overall workload. Experimental results show that our approach provides significant speed-ups over the traditional approach. / text

This master thesis addresses the issue of software testing. It discusses different types and forms of testing such as white-box testing, black-box testing, unit testing, integration testing and regression testing. Further it deals with the specific problems of testing network applications and automation of the testing process. To solve these issues, the thesis presents a language based on XML for describing testing cases. The following chapter describes the design of this language interpreter. The thesis also deals with problems occurring during implementation phase of similar tools and their possible solutions. Functionality demonstration of designed tool helped to discover an error in product of AVG Technologies CZ, s.r.o..

Model-based testing (MBT) is a black-box software testing technique that focuses on specification of the system-under-test (SUT) and/or its environment. It uses models to automatically generate a large number of tests. To the best of our knowledge, no study has investigated the correlation of model coverage with SUT coverage using more advanced coverage criteria (such as edge-pair coverage) and the correlation of coverage (at model level and SUT level) with test suite effectiveness using non-adequate test suites in the context of MBT despite the prominence of non-adequate test suites in industry. To carry out the investigation, we extend an existing open-source MBT tool called Modbat to measure edge-pair coverage at model level, implement a new tool called PaCovForJbc to measure edge-pair coverage, edge coverage and node coverage at SUT level. Finally, we perform an experiment using these tools applied on three projects: “ArrayList”, and “LinkedList” of Java standard library, and “Apache ZooKeeper”. Overall, the results suggest the following: Edge and edge-pair coverage at model level often have a moderate to high correlation with the same type of coverage at SUT level, while that link between model and SUT for node coverage is weaker. Moreover, coverage criteria at SUT level often have a moderate to high correlation with test suite effectiveness, and a coverage criterion at SUT level has a slightly higher correlation with test suite effectiveness than the same type of coverage at model level. Regarding coverage at model level, edge and edge-pair coverage at model level have a slightly higher correlation with test suite effectiveness than node coverage at model level. Note that the mentioned suggestions need to be taken with discretion, because results vary depending on the project and/or coverage criterion under investigation. / Modellbaserad testning (MBT) är en black-box-testteknik som fokuserar på specifikation av system-under-test (SUT) och/eller dess miljö. MBT använder modeller för att generera ett stort antal tester automatiskt. Såvitt vi vet, finns ingen studie som undersökt korrelationen mellan modelltäckning och täckning av SUT med hjälp av mer avancerade täckningskriterier såsom kantparstäckning. Dessutom finns ingen studie som undersökt korrelationen mellan täckning (på modellnivå och SUT-nivå) och effektivitet av icke- adekvata testsviter som genereras med hjälp av MBT trots betydelsen av icke-adekvata testsviter i industrin. För att utföra undersökningen, utökar vi ett ”open-source” MBT-verktyg som kallas för Modbat för att mäta kantparstäckning på modellnivå. Dessutom implementerar vi ett nytt verktyg som kallas för PaCovForJbc för att mäta kantpars-, kant- och nodtäckning på SUT-nivå. Till slut utför vi experiment genom att applicera Modbat och PaCovForJbc på tre projekt: ”ArrayList” och ”LinkedList” av Javas standardbibliotek samt ”Apache ZooKeeper”. Sammantaget indikerar resultaten följande: Kant- och kantparstäckning på modellnivå har ofta en måttlig till hög korrelation med samma typ av täckning på SUT- nivå, medan länken mellan modell och SUT för nodtäckning är svagare. Dessutom har täckningskriterier på SUT-nivå ofta en måttlig till hög korrelation med testsvitseffektivitet, och ett täckningskriterium på SUT-nivå har en aning högre korrelation med testsvitseffektivitet än samma typ av täckning på modellnivå. Angående täckning på modellnivå har kant- och kantparstäckning på modellnivå en aning högre korrelation med testsvitseffektivitet än nodtäckning på modellnivå. Observera att de nämnda förslagen måste tas med diskretion, eftersom resultaten varierar beroende på projektet och/eller täckningskriteriet som undersöks.