This thesis study presents a software measurement program. The literature on software measurement is reviewed. Conditions for an effective implementation are investigated. A specific measurement system is designed and implemented in ASELSAN, Inc. This has involved organizational as well as technical work. A software tool has been developed to assist in aggregating measurements obtained from various CASE tools in use. Results of the implementation have started to be achieved. Lots of useful feedbacks have been returned to the organization as a result of analyzing of the measurement data.

Background and purpose. This thesis documents a program of five studies concerned with the assessment of software measurement. The goal of this program is to assist the software industry to improve the information support for managers, analysts and software engineers by providing evidence of where opportunities for improving measurement and analysis exist. Methods. The first study examined the assessment of software measurement frameworks using models of best practice based on performance/success factors. The software measurement frameworks of thirteen organisations were surveyed. The association between a factor and the outcome experienced with the organisations' frameworks was then evaluated. The subsequent studies were more info-centric and investigated using models of information quality to assess the support provided for software processes. For these studies, information quality models targeting specific software processes were developed using practitioner focus groups. The models were instantiated in survey instruments and the responses were analysed to identify opportunities to improve the information support provided. The final study compared the use of two different information quality models for the assessing and improving information support. Assessments of the same quantum of information were made using a targeted model and a generic model. The assessments were then evaluated by an expert panel in order to identify which information quality model was more effective for improvement purposes. Results. The study of performance factors for software measurement frameworks confirmed the association of some factors with success and quantified that association. In particular, it demonstrated the importance of evaluating contextual factors. The conclusion is that factor-based models may be appropriately used for risk analysis and for identifying constraints on measurement performance. Note, however, that a follow-up study showed that some initially successful frameworks subsequently failed. This implied an instability in the dependent variable, success, that could reduce the value of factor-based models for predicting success. The studies of targeted information quality models demonstrated the effectiveness of targeted assessments for identifying improvement opportunities and suggest that they are likely to be more effective for improvement purposes than using generic information quality models. The studies also showed the effectiveness of importance-performance analysis for prioritizing improvement opportunities.

One of the important aims of the continuous software development process is to localize and remove all existing program bugs as fast as possible. Such goal is highly related to software engineering and defectiveness estimation. Many big companies started to store source code in software repositories as the later grew in popularity. These repositories usually include static source code as well as detailed data for defects in software units. This allows analyzing all the data without interrupting programing process. The main problem of large, complex software is impossibility to control everything manually while the price of the error can be very high. This might result in developers missing defects on testing stage and increase of maintenance cost. The general research goal is to find a way of predicting future software defectiveness with high precision. Reducing maintenance and development costs will contribute to reduce the time-to-market and increase software quality. To address the problem of estimating residual defects an approach was found to predict residual defectiveness of a software by the means of machine learning. For a prime machine learning algorithm, a regression decision tree was chosen as a simple and reliable solution. Data for this tree is extracted from static source code repository and divided into two parts: software metrics and defect data. Software metrics are formed from static code and defect data is extracted from reported issues in the repository. In addition to already reported bugs, they are augmented with unreported bugs found on “discussions” section in repository and parsed by a natural language processor. Metrics were filtered to remove ones, that were not related to defect data by applying correlation algorithm. Remaining metrics were weighted to use the most correlated combination as a training set for the decision tree. As a result, built decision tree model allows to forecast defectiveness with 89% chance for the particular product. This experiment was conducted using GitHub repository on a Java project and predicted number of possible bugs in a single file (Java class). The experiment resulted in designed method for predicting possible defectiveness from a static code of a single big (more than 1000 files) software version.