Software permeates every aspect of our modern lives. In many applications, such in the software for airplane flight controls, or nuclear power control systems software failures can have catastrophic consequences. As we place so much trust in software, how can we know if it is trustworthy? Through software assurance, we can attempt to quantify just that. Building complex, high assurance software is no simple task. The difficult information landscape of a software engineering project can make verification and validation, the process by which the assurance of a software is assessed, very difficult. In order to manage the inevitable information overload of complex software projects, we need software traceability, "the ability to describe and follow the life of a requirement, in both forwards and backwards direction." The Center of Excellence for Software Traceability (CoEST) has created a compelling research agenda with the goal of ubiquitous traceability by 2035. As part of this goal, they have developed TraceLab, a visual experimental workbench built to support design, implementation, and execution of traceability experiments. Through our collaboration with CoEST, we have made several contributions to TraceLab and its community. This work contributes to the goals of the traceability research community. The three key contributions are (a) a machine learning component package for TraceLab featuring six (6) classifier algorithms, five (5) clustering algorithms, and a total of over 40 components for creating TraceLab experiments, built upon the WEKA machine learning package, as well as implementing methods outside of WEKA; (b) the design for an automated tracing system that uses clustering to decompose the task of tracing into many smaller tracing subproblems; and (c) an implementation of several key components of this tracing system using TraceLab and its experimental evaluation.

Requirements tracing is crucial for software engineering practices including change analysis, regression testing, and reverse engineering. The requirements tracing process produces a requirements traceability matrix(TM) which links high- and low-level document elements. Manually generating a TM is laborious, time consuming, and error-prone. Due to these challenges TMs are often neglected. Automated information retrieval(IR) techniques are used with some efficiency. However, in mission- or safety-critical systems a human analyst is required to vet the candidate TM. This introduces semi-automated requirements tracing, where IR methods present a candidate TM and a human analyst validates it, producing a final TM. In semi-automated tracing the focus becomes the quality of the final TM. This thesis expands upon the research of Cuddeback et al. by examining how human analysts interact with candidate TMs. We conduct two experiments, one using an automated tracing tool and the other using manual validation. We conduct formal statistical analysis to determine the key factors impacting the analyst’s tracing performance. Additionally, we conduct a pilot study investigating how analysts interact with TMs generated by automated IR methods. Our research statistically confirms the finding of Cuddeback et al. that the strongest impact on analyst performance is the initial TM quality. Finally we show evidence that applying local filters to IR results produce the best candidate TMs.

It is important to track how a requirement changes throughout the software lifecycle. Each requirement should be validated during and at the end of each phase of the software lifecycle. It is common to build traceability matrices to demonstrate that requirements are satisfied by the design. Traceability matrices are needed in various tasks in the software development process. Unfortunately, developers and designers do not always build traceability matrices or maintain traceability matrices to the proper level of detail. Therefore, traceability matrices are often built after-the-fact. The generation of traceability matrices is a time consuming, error prone, and mundane process. Most of the times, the traceability matrices are built manually. Consider the case where an analyst is tasked to trace a high level requirement document to a lower level requirement specification. The analyst may have to look through M x N elements, where M and N are the number of high and low level requirements, respectively. There are not many tools available to assist the analysts in tracing unstructured textual artifacts and the very few tools that are available require enormous pre-processing. The prime objective of this work was to dynamically generate traceability links for unstructured textual artifacts using information retrieval (IR) methods. Given a user query and a document collection, IR methods identify all the documents that match the query. A closer observation of the requirements tracing process reveals the fact that it can be stated as a recursive IR problem. The main goals of this work were to solve the requirements traceability problem using IR methods and to improve the accuracy of the traceability links generated while best utilizing the analysts time. This work looked into adopting different IR methods and using user feedback to improve the traceability links generated. It also applied wrinkles such as filtering to the original IR methods. It also analyzed using a voting mechanism to select the traceability links identified by different IR methods. Finally, the IR methods were evaluated using six datasets. The results showed that automating requirements tracing process using IR methods helped save analysts time and generate good quality traceability matrices.