ITC/USA 2015 Conference Proceedings / The Fifty-First Annual International Telemetering Conference and Technical Exhibition / October 26-29, 2015 / Bally's Hotel & Convention Center, Las Vegas, NV / Due to the quick change of business processes in organizations, software needs to adapt quickly to meet new requirements by implementing new business rules. In Research and Development (R&D) scenario, the research is highly non-linear and changes are inevitable. In this context, it is known that traditional methodologies (e.g. waterfall) may lead to the detection of failures late, increase the time and cost of development and maintenance of software. On the other hand, agile methodologies are based on Test- Driven Development (TDD), maintain the technical debt under control, maximize the Return on Investment and reduce the risks for customers and companies. In this paper, we show the use of Scrum and TDD in the development of an experimental tool that aims to make the calibration in real time of the rudder of a fighter aircraft. The preliminary results allowed to increase the coverage testing of the software and hence the quality of the tool.

The traditional focus for Information Systems (IS) quality assurance relies on the evaluation of its implementation. However, the quality of an IS can be largely determined in the first stages of its development. Several studies reveal that more than half the errors that occur during systems development are requirements errors. A requirements error is defined as a mismatch between requirements specification and stakeholders¿ needs and expectations. Conceptual modeling is an essential activity in requirements engineering aimed at developing the conceptual schema of an IS. The conceptual schema is the general knowledge that an IS needs to know in order to perform its functions. A conceptual schema specification has semantic quality when it is valid and complete. Validity means that the schema is correct (the knowledge it defines is true for the domain) and relevant (the knowledge it defines is necessary for the system). Completeness means that the conceptual schema includes all relevant knowledge. The validation of a conceptual schema pursues the detection of requirements errors in order to improve its semantic quality. Conceptual schema validation is still a critical challenge in requirements engineering. In this work we contribute to this challenge, taking into account that, since conceptual schemas of IS can be specified in executable artifacts, they can be tested. In this context, the main contributions of this Thesis are (1) an approach to test conceptual schemas of information systems, and (2) a novel method for the incremental development of conceptual schemas supported by continuous test-driven validation. As far as we know, this is the first work that proposes and implements an environment for automated testing of UML/OCL conceptual schemas, and the first work that explores the use of test-driven approaches in conceptual modeling. The testing of conceptual schemas may be an important and practical means for their validation. It allows checking correctness and completeness according to stakeholders¿ needs and expectations. Moreover, in conjunction with the automatic check of basic test adequacy criteria, we can also analyze the relevance of the elements defined in the schema. The testing environment we propose requires a specialized language for writing tests of conceptual schemas. We defined the Conceptual Schema Testing Language (CSTL), which may be used to specify automated tests of executable schemas specified in UML/OCL. We also describe a prototype implementation of a test processor that makes feasible the approach in practice. The conceptual schema testing approach supports test-last validation of conceptual schemas, but it also makes sense to test incomplete conceptual schemas while they are developed. This fact lays the groundwork of Test-Driven Conceptual Modeling (TDCM), which is our second main contribution. TDCM is a novel conceptual modeling method based on the main principles of Test-Driven Development (TDD), an extreme programming method in which a software system is developed in short iterations driven by tests. We have applied the method in several case studies, in the context of Design Research, which is the general research framework we adopted. Finally, we also describe an integration approach of TDCM into a broad set of software development methodologies, including the Unified Process development methodology, MDD-based approaches, storytest-driven agile methods and goal and scenario-oriented requirements engineering methods. / Els enfocaments per assegurar la qualitat deis sistemes d'informació s'han basal tradicional m en! en l'avaluació de la seva implementació. No obstan! aix6, la qualitat d'un sis tema d'informació pot ser ampliament determinada en les primeres fases del seu desenvolupament. Diversos estudis indiquen que més de la meitat deis errors de software són errors de requisits . Un error de requisit es defineix com una desalineació entre l'especificació deis requisits i les necessitats i expectatives de les parts im plicades (stakeholders ). La modelització conceptual és una activitat essencial en l'enginyeria de requisits , l'objectiu de la qual és desenvolupar !'esquema conceptual d'un sistema d'informació. L'esquema conceptual és el coneixement general que un sistema d'informació requereix per tal de desenvolupar les seves funcions . Un esquema conceptual té qualitat semantica quan és va lid i complet. La valides a implica que !'esquema sigui correcte (el coneixement definit és cert peral domini) i rellevant (el coneixement definit és necessari peral sistema). La completes a significa que !'esquema conceptual inclou tot el coneixement rellevant. La validació de !'esquema conceptual té coma objectiu la detecció d'errors de requisits per tal de millorar la qualitat semantica. La validació d'esquemes conceptuals és un repte crític en l'enginyeria de requisits . Aquesta te si contribueix a aquest repte i es basa en el fet que els es quemes conceptuals de sistemes d'informació poden ser especificats en artefactes executables i, per tant, poden ser provats. Les principals contribucions de la te si són (1) un enfocament pera les pro ves d'esquemes conceptuals de sistemes d'informació, i (2) una metodología innovadora pel desenvolupament incremental d'esquemes conceptuals assistit per una validació continuada basada en proves . Les pro ves d'esquemes conceptuals poden ser una im portant i practica técnica pera la se va validació, jaque permeten provar la correctesa i la completesa d'acord ambles necessitats i expectatives de les parts interessades. En conjunció amb la comprovació d'un conjunt basic de criteris d'adequació de les proves, també podem analitzar la rellevancia deis elements definits a !'esquema. L'entorn de test proposat inclou un llenguatge especialitzat per escriure proves automatitzades d'esquemes conceptuals, anomenat Conceptual Schema Testing Language (CSTL). També hem descrit i implementa! a un prototip de processador de tes tos que fa possible l'aplicació de l'enfocament proposat a la practica. D'acord amb l'estat de l'art en validació d'esquemes conceptuals , aquest és el primer treball que proposa i implementa un entorn pel testing automatitzat d'esquemes conceptuals definits en UML!OCL. L'enfocament de proves d'esquemes conceptuals permet dura terme la validació d'esquemes existents , pero també té sentit provar es quemes conceptuals incomplets m entre estant sent desenvolupats. Aquest fet és la base de la metodología Test-Driven Conceptual Modeling (TDCM), que és la segona contribució principal. El TDCM és una metodología de modelització conceptual basada en principis basics del Test-Driven Development (TDD), un métode de programació en el qual un sistema software és desenvolupat en petites iteracions guiades per proves. També hem aplicat el métode en diversos casos d'estudi en el context de la metodología de recerca Design Science Research. Finalment, hem proposat enfocaments d'integració del TDCM en diverses metodologies de desenvolupament de software.

In the 1990s, software projects became larger in size and more complicated in structure. The traditional development processes were not able to answer the needs of these growing projects. Comprehensive documentation in traditional methodologies made processes slow and discouraged the developers. Testing, after all code is written, was time consuming, too costly and made error correction and debugging much harder. Fixing the code at the end of the project also affects the internal quality of the software. Agile software development processes evolved to bring quick solutions to these existing problems of the projects. Test Driven Development (TDD) is a technique, used in many agile methodologies, that suggests minimizing documentation, writing automated tests before implementing the code and frequently run tests to get immediate feedback. The aim is to increase software productivity by shortening error correction duration and increase software quality by providing rapid feedback to the developer. In this thesis work, a software project is developed with TDD and compared with a control project developed using traditional development techniques in terms of software productivity and software quality. In addition, TDD project is compared with an early work in terms of product quality. The benefits and the challenges of TDD are also investigated during the whole process.

One of the most emphasised software testing activities in an Agile environment is the usage of the Test Driven Development (TDD) approach. TDD is a development activity where test cases are created by developers before writing the code, and all for the purpose of guiding the actual development process. In other words, test cases created when following TDD could be considered as a by-product of software development. However, TDD is not fully adopted by the industry, as indicated by respondents from our industrial survey who pointed out that TDD is the most preferred but least practised activity. Our further research identified seven potentially limiting factors for industrial adoption of TDD, out of which one of the prominent factor was lack of developers’ testing skills. We subsequently defined and categorised appropriate quality attributes which describe the quality of test case design when following TDD. Through a number of empirical studies, we have clearly established the effect of “positive test bias”, where the participants focused mainly on the functionality while generating test cases. In other words, there existed less number of “negative test cases” exercising the system beyond the specified functionality, which is an important requirement for high reliability systems. On an average, in our studies, around 70% of test cases created by the participants were positive while only 30% were negative. However, when measuring defect detecting ability of those sets of test cases, an opposite ratio was observed. Defect detecting ability of negative test cases were above 70% while positive test cases contributed only by 30%. We propose a TDDHQ concept as an approach for achieving higher quality testing in TDD by using combinations of quality improvement aspects and test design techniques to facilitate consideration of unspecified requirements during the development to a higher extent and thus minimise the impact of potentially inherent positive test bias in TDD. This way developers do not necessarily focus only on verifying functionality, but they can as well increase security, robustness, performance and many other quality improvement aspects for the given software product. An additional empirical study, evaluating this method, showed a noticeable improvement in the quality of test cases created by developers utilising TDDHQ concept. Our research findings are expected to pave way for further enhancements to the way of performing TDD, eventually resulting in better adoption of it by the industry.

The main objective of this dissertation is to assist in reducing the chaotic state of the software engineering discipline by providing insights into both the effectiveness of software defect reduction methods and ways these methods can be improved. The dissertation is divided into two main parts. The first is a quasi-experiment comparing the software defect rates and initial development costs of two methods of software defect reduction: software inspection and test-driven development (TDD). Participants, consisting of computer science students at the University of Arizona, were divided into four treatment groups and were asked to complete the same programming assignment using either TDD, software inspection, both, or neither. Resulting defect counts and initial development costs were compared across groups. The study found that software inspection is more effective than TDD at reducing defects, but that it also has a higher initial cost of development. The study establishes the existence of a defect-reduction gap between software inspection and TDD and highlights the need to improve TDD because of its other benefits.The second part of the dissertation explores a method of applying mutation analysis to TDD to reduce the defect reduction gap between the two methods and to make TDD more reliable and predictable. A new change impact analysis algorithm (CHA-AS) based on CHA is presented and evaluated for applications of software change impact analysis where a predetermined set of program entry points is not available or is not known. An estimated average case complexity analysis indicates that the algorithm's time and space complexity is linear in the size of the program under analysis, and a simulation experiment indicates that the algorithm can capitalize on the iterative nature of TDD to produce a cost savings in mutation analysis applied to TDD projects. The algorithm should also be useful for other change impact analysis situations with undefined program entry points such as code library and framework development.An enhanced TDD method is proposed that incorporates mutation analysis, and a set of future research directions are proposed for developing tools to support mutation analysis enhanced TDD and to continue to improve the TDD method.

This thesis explores the possibility of the functional programming paradigm in the domain of hard embedded real-time systems. The implementation consists of re-implementing an already developed system that is written with the imperative and object oriented paradigms. The functional implementation of the system in question is compared with the original implementation and a study of code complexity, timing properties, CPU utilization and memory usage is performed. The implementation of this thesis consists of re-developing three of the periodic tasks of the original system and the whole development process is facilitated with the TDD development cycle. The programming language used in this thesis is C but with a functional approach to the problem. We conclusions of this thesis is that the functional implementation will give a more stable, reliable and readable system but some code volume, memory usage and CPU utilization overhead is present. The main benefit of using the functional paradigm in this type of system is the ability of using the TDD development cycle. The main con of this type of implementation is that it relies heavily on garbage collection due to the enforcement of data immutability. We find in conclusion that one can only use the functional paradigm if one has an over dimensioned system when it comes to hardware, mainly when it comes to memory size and CPU power. When developing small systems with scarce resources one should choose another paradigm.

In this thesis, the Test Driven Development method (TDD) is studied for use in developing embedded software. The required framework is written for the development environment Rhapsody. Integration of TDD into a classical development cycle, without necessitating a transition to agile methodologies of software development and required unit test framework to apply TDD to an object oriented embedded software development project with a specific development environment and specific project conditions are done in this thesis. A software tool for unit testing is developed specifically for this purpose, both to support the proposed approach and to illustrate its application. The results show that RhapUnit supplies the required testing functionality for developing embedded software in Rhapsody with TDD. Also, development of RhapUnit is a successful example of the application of TDD.

Context: In recent times, many organizations have started using agile software development methodologies instead of using traditional methodologies. The main reason for this shift is the ability of agile approaches to cope with changes in the requirements, customer satisfaction and assurance of on-time delivery of quality products [19]. Test-Driven Development (TDD) is a software development methodology that is considered to be one of the most prominent practices of eXtreme Programming (XP) (an agile methodology) [1][9][10]. Test-list in TDD is considered as a temporary repository in which test items are stored and later by using those items test cases are developed. Requirements Traceability is also a major problem in agile development mainly because of lack of formal requirements specification and frequent requirements change. Objectives: This study explores the utilization of test-list and possibility of using test-list for requirements traceability in TDD. This study describes concept of test-list, its formation and exploring its utilization in TDD. Methods for implementing requirements traceability in and identification of possibility of utilizing test-list for requirements traceability in TDD is also explored. Methods: Methods used in this study are systematic literature review, surveys and interviews. Systematic literature review was done using seven electronic databases, including Inspec, IEEE Xplore, ACM Digital Library, Springer, Science Direct and Scopus. Studies were selected on the bases of preliminary, basic and advanced developed criteria. Survey was conducted using online questionnaire from TDD practitioners. Findings from literature review and surveys were used to develop interview questionnaires. Interviews were conducted from the same practitioners that were involved in surveys. Results: Based on the findings of literature review, questionnaire and interviews, we obtained TDD practices for test-list development and requirements traceability. Analysis was performed on results of SLR and questionnaire and possibility of using test-list for requirements traceability was identified. Based on the analysis of literature review and surveys, interview questionnaire were developed to further investigate the area of interest. We have found that in literature there is no defined method to develop test-list. and survey participants also confirms it. Majority of survey participants create test-list temporarily and informal. On question of whether test-list can be use for requirements traceability around 70% of participants are agree for its use. Interview respondents also confirm the findings of survey. Conclusions: Literature has not provided any test-list development method and practitioners also have no clear guideline to develop test-list prior to Test development. Systematic literature review and practitioner’s survey and interviews confirm it. Literature is also silent for any specific requirements change management or requirements traceability method in TDD. We identified requirements traceability practices in agile and management through literature and survey. After analysis of gathered data we found TDD lacks in test-list formalization, none of the study focuses on requirements traceability in TDD. In this study our contribution is exploration of test-list creation and utilization through literature and state of the practice; after practitioners feedback we also explored that test-list can be used for requirements traceability. / hasmkh@gmail.com, ibrararshad@gmail.com

The emerging industrial trend towards agile software development processes brings forth new concerns, challenges as well as opportunities. One of the main concerns is with respect to the achievable quality levels of the final product, for which testing is the well-known assurance mechanism. However, it is not well defined for the community on how to perform testing using existing expertise in an agile environment. This uncertainty may create confusion and contra productivity that can lead to testing teams and their practices considered as an obstacle for full implementation of agile processes within an organisation.   This thesis outlines our current research activities towards identifying and addressing important organisational and technical challenges in the agile environment.  In this context, we propose a new role for traditional testers which will enable them to integrate into the agile team as well as to fully exploit their knowledge in the new context. We have conducted an elaborate industrial survey on the preferences and practices with respect to the contemporary aspects of software testing and identified test-driven development as an important technical area for improvement.  A systematic review on empirical evidences related to test-driven development was performed subsequently, which revealed a list of factors limiting its widespread industrial acceptance. Knowledge of testing was identified as one of those factors and was further investigated in a controlled experiment performed with undergraduate students.   Our future works aim to confirm these research findings in wider as well as industrial settings and investigate other limiting factors in detail, with the aim of providing guidelines for achieving better utilisation of testers and testing practices.

Agile methods and Test-Driven Development are well established methodologies within the software development industry. As a large part of today’s software development is done in Object-Oriented languages like Java it’s only natural that agile best practices have evolved to fit into the Object-Oriented paradigm. Clojure is a relatively young programming language that greatly differs from Object-Oriented languages and it’s thus not certain that these best practices can be directly applicable in Clojure development. This thesis attempts to identify key differences between Clojure and Java that may affect unit testing and the Test-Driven Development process. It finds that although the two languages are fundamentally the difference between them in regards to unit test creation and execution is small. The most striking consequence of Clojure’s lack of Object-Orientation is that dependency injection must be done between functions rather than between classes and objects. It’s also argued that the relative immaturity of the available Clojure development tools can have negative effects on the Test-Driven Development process. / Agila metoder och Testdriven Utveckling är väletablerade metoder inom mjukvaruindustrin. Då en stor del av dagens mjukvaruutveckling sker inom Objektorienterade språk som Java är det bara naturligt att agila best-practices har utvecklats för att passa den Objektorienterade paradigmen. Clojure är ett relativt ungt programmeringsspråk som skiljer sig kraftigt från Objektorienterade språk och det är därför inte självklart att dessa best-practices kan vara direkt applicerbara på utveckling i Clojure. Denna uppsats försöker identifiera de huvudskillnader mellan Clojure och Java som kan ha en direkt påverkan på enhetstestning och den Testdrivna Utvecklingsprocessen. Man upptäcker att det - de två språkens fundamentala skillnader till trots - endast finns små skillnader som påverkar skapandet och körandet av enhetstester. Den mest iögonfallande konsekvensen av bristen på Objektorientering i Clojure är att Dependency Injection måste ske på funktionsnivå, istället för klass- och objektsnivå. Man argumenterar även för att den relativa omogenheten hos verktyg som används vid mjukvaruutveckling i Clojure kan ha en negativ effekt på den Testdrivna Utvecklingsprocessen.