The emerging industrial trend towards agile software development processes brings forth new concerns, challenges as well as opportunities. One of the main concerns is with respect to the achievable quality levels of the final product, for which testing is the well-known assurance mechanism. However, it is not well defined for the community on how to perform testing using existing expertise in an agile environment. This uncertainty may create confusion and contra productivity that can lead to testing teams and their practices considered as an obstacle for full implementation of agile processes within an organisation.   This thesis outlines our current research activities towards identifying and addressing important organisational and technical challenges in the agile environment.  In this context, we propose a new role for traditional testers which will enable them to integrate into the agile team as well as to fully exploit their knowledge in the new context. We have conducted an elaborate industrial survey on the preferences and practices with respect to the contemporary aspects of software testing and identified test-driven development as an important technical area for improvement.  A systematic review on empirical evidences related to test-driven development was performed subsequently, which revealed a list of factors limiting its widespread industrial acceptance. Knowledge of testing was identified as one of those factors and was further investigated in a controlled experiment performed with undergraduate students.   Our future works aim to confirm these research findings in wider as well as industrial settings and investigate other limiting factors in detail, with the aim of providing guidelines for achieving better utilisation of testers and testing practices.

Ipendo Systems in Linköping, Sweden, is a small IT consulting firm developing among other things custom portal applications based on Microsoft Sharepoint 2010. The purpose of this thesis is to investigate whether the test tools TestComplete and LoadComplete provide sufficient compatibility for functional and non-functional testing of Ipendo Systems’ Sharepoint based applications, as well as design testing related activities to be incorporated into their existing software development framework. A test process maturity framework was chosen and applied for the design of the testing activities, while the test tool investigation resulted in guidelines on how to apply the tools effectively and circumvent any issues discovered.

When systematically adopting a CASE (Computer-Aided Software Engineering) tool, an organisation evaluates candidate tools against a framework of requirements, and selects the most suitable tool for usage. A method, called 2G, has been proposed that aims at developing such frameworks based on the needs of a specific organisation. This method includes a pilot evaluation phase, where state-of-the-art CASE-tools are explored with the aim of gaining more understanding of the requirements that the organisation adopting CASE-tools puts on candidate tools. This exploration results in certain output data, parts of which are used in interviews to discuss the findings of the tool exploration with the organisation. This project has focused on identifying the characteristics of these data, and subsequently to hypothesise a representation of the data, with the aim of providing guidelines for future tool support for the 2G method. The approach to reaching this aim was to conduct a case study of a new application of the pilot evaluation phase, which resulted in data that could subsequently be analysed with the aim of identifying characteristics. This resulted in a hypothesised data representation, which was found to fit the data from the conducted application well, although certain situations were identified that the representation might not be able to handle.

Testing is a critical activity in the software development process in order to obtain systems of high quality. Tieto typically develops complex systems, which are currently tested through a large number of manually designed test cases. Recent development within software testing has resulted in methods and tools that can automate the test case design, the generation of test code and the test result evaluation based on a model of the system under test. This testing approach is called model-based testing (MBT). This thesis is a feasibility study of the model-based testing concept and has been performed at the Tieto office in Karlstad. The feasibility study included the use and evaluation of the model-based testing tool Qtronic, developed by Conformiq, which automatically designs test cases given a model of the system under test as input. The experiments for the feasibility study were based on the incremental development of a test object, which was the client protocol module of a simplified model for an ATM (Automated Teller Machine) client-server system. The experiments were evaluated both individually and by comparison with the previous experiment since they were based on incremental development. For each experiment the different tasks in the process of testing using Qtronic were analyzed to document the experience gained as well as to identify strengths and weaknesses. The project has shown the promise inherent in using a model-based testing approach. The application of model-based testing and the project results indicate that the approach should be further evaluated since experience will be crucial if the approach is to be adopted within Tieto’s organization.

Context: ISO 26262 requires compilation of traceable work products across the application lifecycle as product based safety evidence. The compilation of such safety evidence is a time consuming and arduous task. Open Services Lifecycle Collaboration (OSLC) is an initiative that supports traceability through tool interoperability. The meta modelling of the ISO 26262 work products in the structure of Resource Description Framework (RDF) can be used for achieving interoperability. Thus, OSLC services used on the RDF exchanged between interoperating tools aids in an effective way of compiling the product based safety evidence for ISO 26262 safety case. Objectives: Representing the compilation of traceable work product types for the software testing and verification in ISO 26262, in form of a RDF-based conceptual meta-model. Testing and extending the concepts by instantiating the meta-model with work products to be represented in RDF for a case of a truck Electronic Control Unit (ECU) system. Lastly, validating the effectiveness of the conceptual meta-model for its compliance to ISO 26262. Methods: To realise the objectives, a case study was conducted at Scania CV AB, Södertälje, Sweden, a manufacturer of safety critical ECU systems used in heavy automobiles. The case study was conducted in three consecutive cycles. The first cycle of qualitative inductive content analysis of the ISO 26262 standard and its related document at the company for defining the conceptual meta model. The second cycle of qualitative deductive content analysis for testing, extending and refining the conceptual meta model. The last cycle of validating the effectiveness of the tested and extended conceptual meta model for compliance to ISO 26262. Results: The main result was the tested, extended and refined RDF based ISO 26262 conceptual meta model depicting traceable work product types for software testing and verification of a safety critical ECU system. The testing and extending of the conceptual meta model was performed with respect to the Main1 (M1) ECU system at Scania. The RDF was defined for the work products of M1 ECU system. Finally, the conceptual meta model was validated for its effectiveness in realising the criteria of abstraction, confirmability and traceability based on ISO 26262.  Conclusions: Thus, the RDF-based conceptual meta-model depicting product based safety evidence provides a structure for realising the traceability required for compiling the software testing and verification part of ISO 26262 safety case. The meta model was tested by defining the RDF for the work products of a truck ECU system that would be exchanged for achieving interoperability. Finally, the conceptual meta-model was validated for representing the knowledge required for showing traceable product based safety evidence for ISO 26262 safety case. / ESPRESSO, Scania CV AB, Södertälje / Gen&ReUsableSafety

Mobile devices have evolved rapidly in recent years and increased usage and performance are pushing contemporary battery technology to its limits. The constrained battery resources mean that the importance of energy-efficient application design is growing and in this regard wireless network accesses are a major contributor to a mobile device's overall energy consumption. Additionally, the energy consumption characteristics of modern cellular technologies make small volumes of poorly scheduled traffic account for a substantial share of a device's total energy consumption. However, quantifying the communication energy footprint is cumbersome, making it difficult for developers to profile applications from an energy consumption perspective and optimize traffic patterns. This thesis examines the traffic patterns of the Android client of the popular multimedia streaming service Spotify with the intention to reduce its energy footprint, in terms of 3G energy consumption. The application's automated test environment is extended to capture network traffic, which is used to estimate energy consumption. Automated system tests are designed and executed on a physical Android device connected to a 3G network, shedding light on the traffic patterns of different application features. All traffic between the Spotify client application and the backend servers is encrypted. To extract information about the traffic, the application code is instrumented to output supplementary information to the Android system log. The system log is then used as a source of information to attribute data traffic to different application modules and specific lines of code. Two simple traffic shaping techniques, traffic aggregation and piggybacking, are implemented in the application to provide more energy-efficient traffic patterns. As a result, 3G energy consumption during normal music playback is reduced by 22-54%, and a more contrived scenario achieves a 60% reduction. The reductions are attained by rescheduling a small class of messages, most notably data tracking application usage. These messages were found to account for a small fraction of total traffic volume, but a large portion of the application's overall 3G energy consumption.

Aspect-oriented programming is a comparatively new programming paradigm which intends to overcome some limitations that approaches such as procedural programming and object-oriented programming have. Traditional approaches are unable to properly capture some design decisions. Aspect-oriented programming introduces some new properties that we don’t find in the structural programming or object-oriented programming. New design patterns of aspect-oriented software introduce new fault types and new challenges for testing. Testing is an important part in the software development to produce quality software. Research on testing aspect-oriented software has been going on for several years but it still remains to invent testing approaches that cover all features of aspect-oriented software. This dissertation surveys test methods for aspect-oriented software and presents a comparison among the testing methods which reveals the strengths and weaknesses of current methods for testing of aspect-oriented software. This comparative overview of proposed test methods can be helpful for testers who intend to test aspect-oriented software. The conclusion presents the research contribution of this dissertation and proposes future work.

Software testing plays an important role in the process of verifying software functionality and preventing bugs in production code. By writing automated tests using code instead of conducting manual tests, the amount of tedious work during the development process can be reduced and the software quality can be improved. This thesis presents the results of a conducted case study on how automated testing can be used when implementing new functionality in a Ruby on Rails web application. Different frameworks for automated software testing are used as well as test-driven development methodology, with the purpose of getting a broad perspective on the subject. This thesis studies common issues with testing web applications, and discuss drawbacks and advantages of different testing approaches. It also looks into quality factors that are applicable for tests, and analyze how these can be measured.

Master of Science / Department of Computing and Information Sciences / David A. Gustafson / The calculation of reliability based on running test cases refers to the probability of the software not generating faulty output consequent to the testing process. The metric used to measure this reliability is referred in terms of E(Θ) value. The concept of E(Θ) gives precise formulae to calculate the probability of failure of software after testing, debug or operational. This report aims at extending the functionalities of E(Θ) into the realm of multiple faults spread across multiple sub-domains. This generalization involves introduction of a new set of formulae for E(Θ) calculation which can account for faults spread over both single as well as multiple sub-domains in a code. The validity of the formulae is verified by matching the obtained theoretical results against the empirical data generated from running a test case simulator. The report further examines the possibility of an upper bound calculation on the derived formulae and its possible ramifications.

In this thesis, the overall conclusion is that combination strategies, (i.e., test case selection methods that manage the combinatorial explosion of possible things to test), can improve the software testing in most organizations. The research underlying this thesis emphasizes relevance by working in close relationship with industry. Input parameter models of test objects play a crucial role for combination strategies. These models consist of parameters with corresponding parameter values and represent the input space and possibly other properties, such as state, of the test object. Test case selection is then defined as the selection of combinations of parameter values from these models. This research describes a complete test process, adapted to combination strategies. Guidelines and step-by-step descriptions of the activities in process are included in the presentation. In particular, selection of suitable combination strategies, input parameter modeling and handling of conflicts in the input parameter models are addressed. It is also shown that several of the steps in the test process can be automated. The test process is validated through a set of experiments and case studies involving industrial testers as well as actual test problems as they occur in industry. In conjunction with the validation of the test process, aspects of applicability of the combination strategy test process (e.g., usability, scalability and performance) are studied. Identification and discussion of barriers for the introduction of the combination strategy test process in industrial projects are also included. This research also presents a comprehensive survey of existing combination strategies, complete with classifications and descriptions of their different properties. Further, this thesis contains a survey of the testing maturity of twelve software-producing organizations. The data indicate low test maturity in most of the investigated organizations. Test managers are often aware of this but have trouble improving. Combination strategies are suitable improvement enablers, due to their low introduction costs.