Context: Test-Driven development (TDD) is a software development approach where test cases are written before actual development of the code in iterative cycles. TDD has gained attention of many software practitioners during the last decade since it has suggested several benefits in the software development process. However, empirical evidence of its dominance in terms of internal code quality, external code quality and productivity is fairly limited. Objectives: The aim behind conducting this study is to explore what has been achieved so far in the field of Test-driven development. The study reports the benefits and limitation of TDD compared to TLD and the outcome variables in all the reported studies along with their measurement criteria. Additionally, an experiment is conducted to see the impact of Test-driven development (TDD) on internal code quality, external code quality and productivity compared to Test-Last development (TLD). Methods: In this study two research methodologies are used specifically systematic literature review according to Kitchenham guidelines and controlled pilot experiment. In systematic literature review number of article sources are considered and used, including Inspec, Compendex, ACM, IEEE Xplore, Science direct (Elsevier) and ISI web of science. A review protocol is created first to ensure the objectivity and repeatability of the whole process. Second, a controlled experiment is conducted with professional software developers to explore the assumed benefits of Test-Driven development (TDD) compared to Test-Last development (TLD). Results: 9 distinct categories related to Test-driven development (TDD) are found that are investigated and reported in the literature. All the reported experiments revealing very little or no difference in internal code quality, external code quality and productivity in Test-Driven development (TDD) over Test-Last development (TLD). However, results were found contradictory when research methods are taken into account because case studies tend to find more positive results in the favor Test-Driven development (TDD) compared to experiments possibly due to the fact that experiment are mostly conducted in artificially created software development environment and mostly with students as a test subjects. On the other hand, experimental results and statistical analysis show no statistically significant result in the favor TDD compared to TLD. All the values found related to number of acceptance test cases passed (Mann-Whitney U test Exact Sig. 0.185), McCabe’s Cyclomatic complexity (Mann-Whitney U test Exact Sig. 0.063), Branch coverage (Mann-Whitney U test Exact Sig. 0.212), Productivity in terms of number of lines of code per person hours (Independent sample Ttest Sig. 0.686), productivity in terms number of user stories implemented per person hours (Independent sample T-test Sig. 0.835) in experiment are statistically insignificant. However, static code analysis (Independent sample T-test Sig. 0.03) result was found statistically significant but due to the low statistical power of test it was not possible to reject the null hypothesis. The results of the survey revealed that the majority of developers in the experiment prefer TLD over TDD, given the lesser required level of learning curve as well as the minimum effort needed to understand and employ TLD compared to TDD Conclusion: Systematic literature review confirms that the reported benefits of TDD development compared to Test-Last development are very small. However, case studies tend to find more positive results in the favor of Test-Driven development (TDD) compared to Test-Last development (TLD). Similarly, experimental findings are also confirming the fact that TDD has small benefits over TLD. However, given the small effect size there is an indication that (Test-Driven development) TDD endorses less complex code compared to Test-Last development (TLD). / Systematic literature review confirms that the reported benefits of TDD development compared to Test-Last development are very small. However, case studies tend to find more positive results in the favor of Test-Driven development (TDD) compared to Test-Last development (TLD). Similarly, experimental findings are also confirming the fact that TDD has small benefits over TLD. However, given the small effect size there is an indication that (Test-Driven development) TDD endorses less complex code compared to Test-Last development (TLD). / hassanmunirr@hotmail.com, mm1844@gmail.com

Desenvolvimento Guiado por Testes (TDD) e uma das praticas sugeridas na Programacao Extrema. A mecanica da pratica e simples: o programador escreve o teste antes de escrever o codigo. E, portanto, possivel inferir que a pratica de TDD e uma pratica de testes de software. Entretanto, muitos autores de livros conhecidos pela industria e academia afirmam que os efeitos da pratica vao alem. Segundo eles, TDD ajuda o desenvolvedor durante o processo de criacao do projeto classes, fazendo-os criar classes menos acopladas e mais coesas. Entretanto, grande parte dos trabalhos da literatura sao voltados a descobrir se a pratica faz diferenca na qualidade do codigo gerado, mas poucos sao os autores que discutem como a pratica realmente auxilia. Mesmo os proprios praticantes nao entendem ou conseguem expressar bem como a pratica os guia. Este trabalho tem por objetivo compreender melhor os efeitos de TDD e como sua pratica influencia o desenvolvedor durante o processo de projeto de sistemas orientados a objetos. Para entende-las, neste trabalho optamos por um estudo exploratorio essencialmente qualitativo, no qual participantes foram convidados a resolver exercicios pre-preparados utilizando TDD e, a partir dos dados colhidos nessa primeira parte, nos levantamos detalhes sobre como a pratica influenciou as decisoes de projeto de classes dos participantes por meio de entrevistas. Ao final, observamos que a pratica de TDD pode guiar o desenvolvedor durante o processo de criacao do projeto de classes por meio de constantes feedbacks sobre a qualidade do projeto. Esses feedbacks alertam desenvolvedores sobre possiveis problemas, como alto acoplamento ou baixa coesao. Os desenvolvedores, por sua vez, devem interpretar e melhorar o projeto de classes. Este trabalho catalogou e nomeou os padroes de feedback percebidos pelos participantes. / Test-Driven Development (TDD) is one of the suggested practices in Extreme Programming (XP). The mechanical is simple: the developer writes a test before writing the implementation. Thus, TDD is often seen as a software testing technique. However, many famous book authors suggest that TDD can help developers during the class design creation process, enabling developers to create less coupled highly cohesive classes. Most of the academic studies are interested on finding the difference between a TDDd and a non-TDDd code. Only a few of them discuss how the practice really supports class design. Even practitioners do not understand how the practice guides them. This work aims to understand better the effects of TDD and how the practice influences the practitioner during the class design process in object-oriented systems. To better understand them, we did a essencially qualitative explorative study, in which participants were invited to solve a set of pre-prepared exercises using TDD and, based on the gathered data, we retrieved details of how the practice influenced the developers class design decisions through interviews. At the end, we observed that the practice of TDD can guide developers during the class design creation process through constant feedback about its quality. These feedbacks alert developers about possible problems, such as high coupling or low cohesion. Developers then should interpret and improve the class design accordingly. This study also catalogues the TDD feedback patterns perceived by the participants.

Desenvolvimento Guiado por Testes (TDD) e uma das praticas sugeridas na Programacao Extrema. A mecanica da pratica e simples: o programador escreve o teste antes de escrever o codigo. E, portanto, possivel inferir que a pratica de TDD e uma pratica de testes de software. Entretanto, muitos autores de livros conhecidos pela industria e academia afirmam que os efeitos da pratica vao alem. Segundo eles, TDD ajuda o desenvolvedor durante o processo de criacao do projeto classes, fazendo-os criar classes menos acopladas e mais coesas. Entretanto, grande parte dos trabalhos da literatura sao voltados a descobrir se a pratica faz diferenca na qualidade do codigo gerado, mas poucos sao os autores que discutem como a pratica realmente auxilia. Mesmo os proprios praticantes nao entendem ou conseguem expressar bem como a pratica os guia. Este trabalho tem por objetivo compreender melhor os efeitos de TDD e como sua pratica influencia o desenvolvedor durante o processo de projeto de sistemas orientados a objetos. Para entende-las, neste trabalho optamos por um estudo exploratorio essencialmente qualitativo, no qual participantes foram convidados a resolver exercicios pre-preparados utilizando TDD e, a partir dos dados colhidos nessa primeira parte, nos levantamos detalhes sobre como a pratica influenciou as decisoes de projeto de classes dos participantes por meio de entrevistas. Ao final, observamos que a pratica de TDD pode guiar o desenvolvedor durante o processo de criacao do projeto de classes por meio de constantes feedbacks sobre a qualidade do projeto. Esses feedbacks alertam desenvolvedores sobre possiveis problemas, como alto acoplamento ou baixa coesao. Os desenvolvedores, por sua vez, devem interpretar e melhorar o projeto de classes. Este trabalho catalogou e nomeou os padroes de feedback percebidos pelos participantes. / Test-Driven Development (TDD) is one of the suggested practices in Extreme Programming (XP). The mechanical is simple: the developer writes a test before writing the implementation. Thus, TDD is often seen as a software testing technique. However, many famous book authors suggest that TDD can help developers during the class design creation process, enabling developers to create less coupled highly cohesive classes. Most of the academic studies are interested on finding the difference between a TDDd and a non-TDDd code. Only a few of them discuss how the practice really supports class design. Even practitioners do not understand how the practice guides them. This work aims to understand better the effects of TDD and how the practice influences the practitioner during the class design process in object-oriented systems. To better understand them, we did a essencially qualitative explorative study, in which participants were invited to solve a set of pre-prepared exercises using TDD and, based on the gathered data, we retrieved details of how the practice influenced the developers class design decisions through interviews. At the end, we observed that the practice of TDD can guide developers during the class design creation process through constant feedback about its quality. These feedbacks alert developers about possible problems, such as high coupling or low cohesion. Developers then should interpret and improve the class design accordingly. This study also catalogues the TDD feedback patterns perceived by the participants.