In the 1990s, software projects became larger in size and more complicated in structure. The traditional development processes were not able to answer the needs of these growing projects. Comprehensive documentation in traditional methodologies made processes slow and discouraged the developers. Testing, after all code is written, was time consuming, too costly and made error correction and debugging much harder. Fixing the code at the end of the project also affects the internal quality of the software. Agile software development processes evolved to bring quick solutions to these existing problems of the projects. Test Driven Development (TDD) is a technique, used in many agile methodologies, that suggests minimizing documentation, writing automated tests before implementing the code and frequently run tests to get immediate feedback. The aim is to increase software productivity by shortening error correction duration and increase software quality by providing rapid feedback to the developer. In this thesis work, a software project is developed with TDD and compared with a control project developed using traditional development techniques in terms of software productivity and software quality. In addition, TDD project is compared with an early work in terms of product quality. The benefits and the challenges of TDD are also investigated during the whole process.

One of the most emphasised software testing activities in an Agile environment is the usage of the Test Driven Development (TDD) approach. TDD is a development activity where test cases are created by developers before writing the code, and all for the purpose of guiding the actual development process. In other words, test cases created when following TDD could be considered as a by-product of software development. However, TDD is not fully adopted by the industry, as indicated by respondents from our industrial survey who pointed out that TDD is the most preferred but least practised activity. Our further research identified seven potentially limiting factors for industrial adoption of TDD, out of which one of the prominent factor was lack of developers’ testing skills. We subsequently defined and categorised appropriate quality attributes which describe the quality of test case design when following TDD. Through a number of empirical studies, we have clearly established the effect of “positive test bias”, where the participants focused mainly on the functionality while generating test cases. In other words, there existed less number of “negative test cases” exercising the system beyond the specified functionality, which is an important requirement for high reliability systems. On an average, in our studies, around 70% of test cases created by the participants were positive while only 30% were negative. However, when measuring defect detecting ability of those sets of test cases, an opposite ratio was observed. Defect detecting ability of negative test cases were above 70% while positive test cases contributed only by 30%. We propose a TDDHQ concept as an approach for achieving higher quality testing in TDD by using combinations of quality improvement aspects and test design techniques to facilitate consideration of unspecified requirements during the development to a higher extent and thus minimise the impact of potentially inherent positive test bias in TDD. This way developers do not necessarily focus only on verifying functionality, but they can as well increase security, robustness, performance and many other quality improvement aspects for the given software product. An additional empirical study, evaluating this method, showed a noticeable improvement in the quality of test cases created by developers utilising TDDHQ concept. Our research findings are expected to pave way for further enhancements to the way of performing TDD, eventually resulting in better adoption of it by the industry.

The main objective of this dissertation is to assist in reducing the chaotic state of the software engineering discipline by providing insights into both the effectiveness of software defect reduction methods and ways these methods can be improved. The dissertation is divided into two main parts. The first is a quasi-experiment comparing the software defect rates and initial development costs of two methods of software defect reduction: software inspection and test-driven development (TDD). Participants, consisting of computer science students at the University of Arizona, were divided into four treatment groups and were asked to complete the same programming assignment using either TDD, software inspection, both, or neither. Resulting defect counts and initial development costs were compared across groups. The study found that software inspection is more effective than TDD at reducing defects, but that it also has a higher initial cost of development. The study establishes the existence of a defect-reduction gap between software inspection and TDD and highlights the need to improve TDD because of its other benefits.The second part of the dissertation explores a method of applying mutation analysis to TDD to reduce the defect reduction gap between the two methods and to make TDD more reliable and predictable. A new change impact analysis algorithm (CHA-AS) based on CHA is presented and evaluated for applications of software change impact analysis where a predetermined set of program entry points is not available or is not known. An estimated average case complexity analysis indicates that the algorithm's time and space complexity is linear in the size of the program under analysis, and a simulation experiment indicates that the algorithm can capitalize on the iterative nature of TDD to produce a cost savings in mutation analysis applied to TDD projects. The algorithm should also be useful for other change impact analysis situations with undefined program entry points such as code library and framework development.An enhanced TDD method is proposed that incorporates mutation analysis, and a set of future research directions are proposed for developing tools to support mutation analysis enhanced TDD and to continue to improve the TDD method.

This thesis explores the possibility of the functional programming paradigm in the domain of hard embedded real-time systems. The implementation consists of re-implementing an already developed system that is written with the imperative and object oriented paradigms. The functional implementation of the system in question is compared with the original implementation and a study of code complexity, timing properties, CPU utilization and memory usage is performed. The implementation of this thesis consists of re-developing three of the periodic tasks of the original system and the whole development process is facilitated with the TDD development cycle. The programming language used in this thesis is C but with a functional approach to the problem. We conclusions of this thesis is that the functional implementation will give a more stable, reliable and readable system but some code volume, memory usage and CPU utilization overhead is present. The main benefit of using the functional paradigm in this type of system is the ability of using the TDD development cycle. The main con of this type of implementation is that it relies heavily on garbage collection due to the enforcement of data immutability. We find in conclusion that one can only use the functional paradigm if one has an over dimensioned system when it comes to hardware, mainly when it comes to memory size and CPU power. When developing small systems with scarce resources one should choose another paradigm.

In this thesis, the Test Driven Development method (TDD) is studied for use in developing embedded software. The required framework is written for the development environment Rhapsody. Integration of TDD into a classical development cycle, without necessitating a transition to agile methodologies of software development and required unit test framework to apply TDD to an object oriented embedded software development project with a specific development environment and specific project conditions are done in this thesis. A software tool for unit testing is developed specifically for this purpose, both to support the proposed approach and to illustrate its application. The results show that RhapUnit supplies the required testing functionality for developing embedded software in Rhapsody with TDD. Also, development of RhapUnit is a successful example of the application of TDD.

Context: Developer’s within software companies work chosen choice of software development process. Choice of a particular Software Development Process impacts the work environment, end-product and might also have financial risks due to delay in delivering in the final software product. Objectives: we are investigating if the TDD could impact/applicable in large-scale industries. To achieve that we first identify the strengths and challenges while using TDD in large-scale industries. Identify what modifications can enhance the impact of TDD in large-Scale industries. Methods: Systematic Literature Review (SLR) has been used to investigate the application of TDD in large-scale industries. Using the info from SLR we conducted an online survey for validating the results obtained from SLR. As a last step we have conducted semi structured interviews to gather information from developers across industries practicing and practiced TDD. The data from the qualitative and quantitative methods is triangulated by identifying the strengths, challenges and modifications in applying TDD to large-scale industries. Results: The findings from our SLR, results validated from Survey and responses from interview participants show that TDD have both strengths, Challenges and modifications. Conclusions: Some challenges encountered when using TDD in large-scale industries are Negative test cases, linking test case modules of several developers, Communication, Overall idea about project, lack of TDD knowledge for developers practicing TDD, also lack of automation tools supporting the development of unit test-cases.

Context: In recent times, many organizations have started using agile software development methodologies instead of using traditional methodologies. The main reason for this shift is the ability of agile approaches to cope with changes in the requirements, customer satisfaction and assurance of on-time delivery of quality products [19]. Test-Driven Development (TDD) is a software development methodology that is considered to be one of the most prominent practices of eXtreme Programming (XP) (an agile methodology) [1][9][10]. Test-list in TDD is considered as a temporary repository in which test items are stored and later by using those items test cases are developed. Requirements Traceability is also a major problem in agile development mainly because of lack of formal requirements specification and frequent requirements change. Objectives: This study explores the utilization of test-list and possibility of using test-list for requirements traceability in TDD. This study describes concept of test-list, its formation and exploring its utilization in TDD. Methods for implementing requirements traceability in and identification of possibility of utilizing test-list for requirements traceability in TDD is also explored. Methods: Methods used in this study are systematic literature review, surveys and interviews. Systematic literature review was done using seven electronic databases, including Inspec, IEEE Xplore, ACM Digital Library, Springer, Science Direct and Scopus. Studies were selected on the bases of preliminary, basic and advanced developed criteria. Survey was conducted using online questionnaire from TDD practitioners. Findings from literature review and surveys were used to develop interview questionnaires. Interviews were conducted from the same practitioners that were involved in surveys. Results: Based on the findings of literature review, questionnaire and interviews, we obtained TDD practices for test-list development and requirements traceability. Analysis was performed on results of SLR and questionnaire and possibility of using test-list for requirements traceability was identified. Based on the analysis of literature review and surveys, interview questionnaire were developed to further investigate the area of interest. We have found that in literature there is no defined method to develop test-list. and survey participants also confirms it. Majority of survey participants create test-list temporarily and informal. On question of whether test-list can be use for requirements traceability around 70% of participants are agree for its use. Interview respondents also confirm the findings of survey. Conclusions: Literature has not provided any test-list development method and practitioners also have no clear guideline to develop test-list prior to Test development. Systematic literature review and practitioner’s survey and interviews confirm it. Literature is also silent for any specific requirements change management or requirements traceability method in TDD. We identified requirements traceability practices in agile and management through literature and survey. After analysis of gathered data we found TDD lacks in test-list formalization, none of the study focuses on requirements traceability in TDD. In this study our contribution is exploration of test-list creation and utilization through literature and state of the practice; after practitioners feedback we also explored that test-list can be used for requirements traceability. / hasmkh@gmail.com, ibrararshad@gmail.com

Context: Test-Driven development (TDD) is a software development approach where test cases are written before actual development of the code in iterative cycles. TDD has gained attention of many software practitioners during the last decade since it has suggested several benefits in the software development process. However, empirical evidence of its dominance in terms of internal code quality, external code quality and productivity is fairly limited. Objectives: The aim behind conducting this study is to explore what has been achieved so far in the field of Test-driven development. The study reports the benefits and limitation of TDD compared to TLD and the outcome variables in all the reported studies along with their measurement criteria. Additionally, an experiment is conducted to see the impact of Test-driven development (TDD) on internal code quality, external code quality and productivity compared to Test-Last development (TLD). Methods: In this study two research methodologies are used specifically systematic literature review according to Kitchenham guidelines and controlled pilot experiment. In systematic literature review number of article sources are considered and used, including Inspec, Compendex, ACM, IEEE Xplore, Science direct (Elsevier) and ISI web of science. A review protocol is created first to ensure the objectivity and repeatability of the whole process. Second, a controlled experiment is conducted with professional software developers to explore the assumed benefits of Test-Driven development (TDD) compared to Test-Last development (TLD). Results: 9 distinct categories related to Test-driven development (TDD) are found that are investigated and reported in the literature. All the reported experiments revealing very little or no difference in internal code quality, external code quality and productivity in Test-Driven development (TDD) over Test-Last development (TLD). However, results were found contradictory when research methods are taken into account because case studies tend to find more positive results in the favor Test-Driven development (TDD) compared to experiments possibly due to the fact that experiment are mostly conducted in artificially created software development environment and mostly with students as a test subjects. On the other hand, experimental results and statistical analysis show no statistically significant result in the favor TDD compared to TLD. All the values found related to number of acceptance test cases passed (Mann-Whitney U test Exact Sig. 0.185), McCabe’s Cyclomatic complexity (Mann-Whitney U test Exact Sig. 0.063), Branch coverage (Mann-Whitney U test Exact Sig. 0.212), Productivity in terms of number of lines of code per person hours (Independent sample Ttest Sig. 0.686), productivity in terms number of user stories implemented per person hours (Independent sample T-test Sig. 0.835) in experiment are statistically insignificant. However, static code analysis (Independent sample T-test Sig. 0.03) result was found statistically significant but due to the low statistical power of test it was not possible to reject the null hypothesis. The results of the survey revealed that the majority of developers in the experiment prefer TLD over TDD, given the lesser required level of learning curve as well as the minimum effort needed to understand and employ TLD compared to TDD Conclusion: Systematic literature review confirms that the reported benefits of TDD development compared to Test-Last development are very small. However, case studies tend to find more positive results in the favor of Test-Driven development (TDD) compared to Test-Last development (TLD). Similarly, experimental findings are also confirming the fact that TDD has small benefits over TLD. However, given the small effect size there is an indication that (Test-Driven development) TDD endorses less complex code compared to Test-Last development (TLD). / Systematic literature review confirms that the reported benefits of TDD development compared to Test-Last development are very small. However, case studies tend to find more positive results in the favor of Test-Driven development (TDD) compared to Test-Last development (TLD). Similarly, experimental findings are also confirming the fact that TDD has small benefits over TLD. However, given the small effect size there is an indication that (Test-Driven development) TDD endorses less complex code compared to Test-Last development (TLD). / hassanmunirr@hotmail.com, mm1844@gmail.com

The emerging industrial trend towards agile software development processes brings forth new concerns, challenges as well as opportunities. One of the main concerns is with respect to the achievable quality levels of the final product, for which testing is the well-known assurance mechanism. However, it is not well defined for the community on how to perform testing using existing expertise in an agile environment. This uncertainty may create confusion and contra productivity that can lead to testing teams and their practices considered as an obstacle for full implementation of agile processes within an organisation.   This thesis outlines our current research activities towards identifying and addressing important organisational and technical challenges in the agile environment.  In this context, we propose a new role for traditional testers which will enable them to integrate into the agile team as well as to fully exploit their knowledge in the new context. We have conducted an elaborate industrial survey on the preferences and practices with respect to the contemporary aspects of software testing and identified test-driven development as an important technical area for improvement.  A systematic review on empirical evidences related to test-driven development was performed subsequently, which revealed a list of factors limiting its widespread industrial acceptance. Knowledge of testing was identified as one of those factors and was further investigated in a controlled experiment performed with undergraduate students.   Our future works aim to confirm these research findings in wider as well as industrial settings and investigate other limiting factors in detail, with the aim of providing guidelines for achieving better utilisation of testers and testing practices.

Agile methods and Test-Driven Development are well established methodologies within the software development industry. As a large part of today’s software development is done in Object-Oriented languages like Java it’s only natural that agile best practices have evolved to fit into the Object-Oriented paradigm. Clojure is a relatively young programming language that greatly differs from Object-Oriented languages and it’s thus not certain that these best practices can be directly applicable in Clojure development. This thesis attempts to identify key differences between Clojure and Java that may affect unit testing and the Test-Driven Development process. It finds that although the two languages are fundamentally the difference between them in regards to unit test creation and execution is small. The most striking consequence of Clojure’s lack of Object-Orientation is that dependency injection must be done between functions rather than between classes and objects. It’s also argued that the relative immaturity of the available Clojure development tools can have negative effects on the Test-Driven Development process. / Agila metoder och Testdriven Utveckling är väletablerade metoder inom mjukvaruindustrin. Då en stor del av dagens mjukvaruutveckling sker inom Objektorienterade språk som Java är det bara naturligt att agila best-practices har utvecklats för att passa den Objektorienterade paradigmen. Clojure är ett relativt ungt programmeringsspråk som skiljer sig kraftigt från Objektorienterade språk och det är därför inte självklart att dessa best-practices kan vara direkt applicerbara på utveckling i Clojure. Denna uppsats försöker identifiera de huvudskillnader mellan Clojure och Java som kan ha en direkt påverkan på enhetstestning och den Testdrivna Utvecklingsprocessen. Man upptäcker att det - de två språkens fundamentala skillnader till trots - endast finns små skillnader som påverkar skapandet och körandet av enhetstester. Den mest iögonfallande konsekvensen av bristen på Objektorientering i Clojure är att Dependency Injection måste ske på funktionsnivå, istället för klass- och objektsnivå. Man argumenterar även för att den relativa omogenheten hos verktyg som används vid mjukvaruutveckling i Clojure kan ha en negativ effekt på den Testdrivna Utvecklingsprocessen.