Context: Early prediction of software cost and quality is important for better software planning and controlling. In early development phases, design complexity metrics are considered as useful indicators of software testing effort and some quality attributes. Although many studies investigate the relationship between design complexity and cost and quality, it is unclear what we have learned from these studies, because no systematic synthesis exists to date. Aim: The research presented in this thesis is intended to contribute for the body of knowledge about cost and quality prediction. A major part of this thesis presents the systematic review that provides detail discussion about state of the art of research on relationship between software design metric and cost and software quality. Method: This thesis starts with a literature review to identify the important complexity dimensions and potential predictors for predicting external software quality attributes are identified. Second, we aggregated Spearman correlation coefficients and estimated odds ratios from univariate logistic regression models from 59 different data sets from 57 primary studies by a tailored meta-analysis approach. At last, it is an attempt to evaluate and explain for disagreement among selected studies. Result: There are not enough studies for quantitatively summarizing relationship between design complexity and development cost. Fault proneness and maintainability is the main focused characteristics that consume 75% total number of studies. Within fault proneness and maintainability studies, coupling and scale are two complexity dimensions that are most frequently used. Vote counting shows evidence about positive impact of some design metrics on these two quality attributes. Meta analysis shows the aggregated effect size of Line of code (LOC) is stronger than those of WMC, RFC and CBO. The aggregated effect sizes of LCOM, DIT and NOC are at trivial to small level. In subgroup analysis, defect collections phase explains more than 50% of observed variation in five out of seven investigated metrics. Conclusions: Coupling and scale metrics are stronger correlated to fault proneness than cohesion and inheritance metrics. No design metrics are stronger single predictors than LOC. We found that there is a strong disagreement between the individual studies, and that defect collection phase is able to partially explain the differences between studies.

Software organizations have been initiating measurement programs as part of their software process improvement practices which in turn increase the quality of the software products. However, high numbers of organizations reported their measurement programs as failures. One of the major difficulties they faced was stated to be the difficulty in deciding which set of measures to collect for different software entities; products, processes, and resources. In addition to what should be collected, organizations would like to collect as minimum number of measures as possible in order to be cost effective. Different goal based measures selection models have been introduced to aid in these, namely Goal Question Metric (GQM) is one of the most well known and adopted ones by software organizations. It facilitates measurement programs to define goals and then collect measures according to goals. However, the current approaches require being more structured and integrated to enable an effective software measurement process. ‘Structured’ means the goals, questions and measures as well as their vertical and horizontal relations are well-defined. A structured measurement process can be ‘integrated’ to the organization via establishing links between business and organizational goals and the measurement process. The whole process becomes more ‘effective’ as these goals and measures become traceable, and hence the mechanisms to prioritize and select the optimum set of measures could be established. In this research work our main focus is to evaluate goal based models and to develop a framework for a more effective measurement process, called Structured Optimized Measurement Selection (SOMS). As part of SOMS, alternative ways for goals prioritization and measures optimization are proposed. In order to make the whole process more efficient, a tool is developed as well. Both SOMS with different alternative prioritization and optimization techniques as well as the tool were evaluated by conducting case studies in a CMMI level 3 software development organization and the results are compared to a previous case study conducted on the same case by using other techniques which are extensions of GQM, i.e. namely Structured Prioritized Goals Questions Metrics (SPGQM) and Optimum Measures Set Decision (OMSD).

Complex modern software is often built using existing application frameworks and middleware frameworks. These frameworks provide useful common services, while simultaneously imposing architectural rules and constraints. Existing software architecture evaluation methods do not explicitly consider the implications of these frameworks for software architecture. This research extends scenario-based architecture evaluation methods by incorporating framework-related information into different evaluation activities. I propose four techniques which target four different activities within a scenario-based architecture evaluation method. 1) Scenario development: A new technique was designed aiming to extract general scenarios and tactics from framework-related architectural patterns. The technique is intended to complement the current scenario development process. The feasibility of the technique was validated through a case study. Significant improvements of scenario quality were observed in a controlled experiment conducted by another colleague. 2) Architecture representation: A new metrics-driven technique was created to reconstruct software architecture in a just-in-time fashion. This technique was validated in a case study. This approach has significantly improved the efficiency of architecture representation in a complex environment. 3) Attribute specific analysis (performance only): A model-driven approach to performance measurement was applied by decoupling framework-specific information from performance testing requirements. This technique was validated on two platforms (J2EE and Web Services) through a number of case studies. This technique leads to the benchmark producing more representative measures of the eventual application. It reduces the complexity behind the load testing suite and framework-specific performance data collecting utilities. 4) Trade-off and sensitivity analysis: A new technique was designed seeking to improve the Analytical Hierarchical Process (AHP) for trade-off and sensitivity analysis during a framework selection process. This approach was validated in a case study using data from a commercial project. The approach can identify 1) trade-offs implied by an architecture alternative, along with the magnitude of these trade-offs. 2) the most critical decisions in the overall decision process 3) the sensitivity of the final decision and its capability for handling quality attribute priority changes.

Statische Codemetriken sind wichtige Indikatoren für die Qualität eines Softwaresystems. Sie beleuchten dabei unterschiedliche Aspekte eines Softwaresystems. Deshalb ist es notwendig, mehrere Codemetriken zu nutzen, um die Qualität eines Softwaresystems in seiner Gesamtheit bewerten zu können. Wünschenswert wäre zudem eine Darstellung, die die Struktur des Gesamtsystems und die Bewertung einzelner Elemente eines Softwaresystems in einer Darstellung kombiniert. Die Arbeit untersucht deshalb, welche Metaphern geeignet sind, um eine solche Darstellung zu ermöglichen. Ein zweites Ziel der Arbeit war es, eine solche Visualisierung automatisch erzeugen zu können. Dafür wurde ein Generator entwickelt, der diese Anforderung erfüllt. Zur Konzeption dieses Generators kamen Techniken aus der generativen Softwareentwicklung zum Einsatz. Bei der Umsetzung des Generators wurde auf Techniken aus der modellgetriebenen Softwareentwicklung zurückgegriffen, vor allem auf Techniken aus dem openArchitectureWare-Framework. Der Generator kann in Eclipse eingebunden werden und ist in der Lage, aus einem Java-Projekt die Struktur und die Metrikwerte automatisch zu extrahieren. Diese Werte werden daraufhin in ein dreidimensionales Modell überführt, das auf dem offenen Extensible 3D Standard basiert. Der Generator ermöglichte zudem die Evaluierung zweier unterschiedlicher Metaphern, die im Rahmen der Arbeit durchgeführt wurde.

Software quality ensures that applications that are developed are failure free. Some modern systems are intricate, due to the complexity of their information processes. Software fault prediction is an important quality assurance activity, since it is a mechanism that correctly predicts the defect proneness of modules and classifies modules that saves resources, time and developers’ efforts. In this study, a model that selects relevant features that can be used in defect prediction was proposed. The literature was reviewed and it revealed that process metrics are better predictors of defects in version systems and are based on historic source code over time. These metrics are extracted from the source-code module and include, for example, the number of additions and deletions from the source code, the number of distinct committers and the number of modified lines. In this research, defect prediction was conducted using open source software (OSS) of software product line(s) (SPL), hence process metrics were chosen. Data sets that are used in defect prediction may contain non-significant and redundant attributes that may affect the accuracy of machine-learning algorithms. In order to improve the prediction accuracy of classification models, features that are significant in the defect prediction process are utilised. In machine learning, feature selection techniques are applied in the identification of the relevant data. Feature selection is a pre-processing step that helps to reduce the dimensionality of data in machine learning. Feature selection techniques include information theoretic methods that are based on the entropy concept. This study experimented the efficiency of the feature selection techniques. It was realised that software defect prediction using significant attributes improves the prediction accuracy. A novel MICFastCR model, which is based on the Maximal Information Coefficient (MIC) was developed to select significant attributes and Fast Correlation Based Filter (FCBF) to eliminate redundant attributes. Machine learning algorithms were then run to predict software defects. The MICFastCR achieved the highest prediction accuracy as reported by various performance measures. / School of Computing / Ph. D. (Computer Science)

Made available in DSpace on 2015-03-03T15:47:44Z (GMT). No. of bitstreams: 1 TassiaAVF.pdf: 1514001 bytes, checksum: a6dbf5832432db823e99d729ccf7fa2b (MD5) Previous issue date: 2009-02-20 / Nowadays, there are many aspect-oriented middleware implementations that take advantage of the modularity provided by the aspect oriented paradigm. Although the works always present an assessment of the middleware according to some quality attribute, there is not a specific set of metrics to assess them in a comprehensive way, following various quality attributes. This work aims to propose a suite of metrics for the assessment of aspect-oriented middleware systems at different development stages: design, refactoring, implementation and runtime. The work presents the metrics and how they are applied at each development stage. The suite is composed of metrics associated to static properties (modularity, maintainability, reusability, exibility, complexity, stability, and size) and dynamic properties (performance and memory consumption). Such metrics are based on existing assessment approaches of object-oriented and aspect-oriented systems. The proposed metrics are used in the context of OiL (Orb in Lua), a middleware based on CORBA and implemented in Lua, and AO-OiL, the refactoring of OIL that follows a reference architecture for aspect-oriented middleware systems. The case study performed in OiL and AO-OiL is a system for monitoring of oil wells. This work also presents the CoMeTA-Lua tool to automate the collection of coupling and size metrics in Lua source code / Atualmente, ha diversas implementa??es de sistemas de middleware orientado a aspectos que aproveitam o suporte a modulariza??o do paradigma de orienta??o a aspectos. Apesar desses trabalhos sempre apresentarem uma avalia??o do middleware de acordo com algum atributo de qualidade, n?o ha ainda um conjunto de metricas especificamente de nidas para avali?-los de forma abrangente, seguindo v?rios atributos de qualidade. Este trabalho tem como objetivo propor um conjunto de metricas para avalia??o de sistemas de middleware orientado a aspectos em diferentes fases de desenvolvimento: design, refatora??o, implementa??o e execu??o. O trabalho apresenta as metricas e como elas s?o aplicadas em cada uma das fases de desenvolvimento. O conjunto e composto por metricas associadas a propriedades estaticas (modularidade, manutenibilidade, reusabilidade, exibilidade, complexidade, estabilidade e tamanho) e din?micas (desempenho e consumo de mem oria). Tais metricas s?o baseadas em abordagens existentes de avalia??o de sistemas orientados a aspectos e a objetos. As metricas propostas s?o utilizadas no contexto do OiL (Orb in Lua), um middleware baseado em CORBA e implementado em Lua, e AO-OiL, uma refatora??o do OIL que segue uma arquitetura de refer?ncia para sistemas de middleware orientados a aspectos. O estudo de caso executado no OiL e no AO-OiL e um sistema de monitoramento de po?os de petroleo. Esse trabalho apresenta ainda a ferramenta CoMeTA-Lua para automatizar a coleta das metricas de tamanho e acoplamento em codigo-fonte Lua

Le contrôle de version est la pierre angulaire des processus de développement de logiciels modernes. Tout en construisant des logiciels de plus en plus complexes, les développeurs doivent comprendre des sous-systèmes de code source qui leur sont peu familier. Alors que la compréhension de la logique d'un code étranger est relativement simple, la compréhension de sa conception et de sa genèse est plus compliquée. Elle n'est souvent possible que par les descriptions des révisions et de la documentation du projet qui sont dispersées et peu fiables -- quand elles existent. Ainsi, les développeurs ont besoin d'une base de référence fiable et pertinente pour comprendre l'historique des projets logiciels. Dans cette thèse, nous faisons les premiers pas vers la compréhension des motifs de changement dans les historiques de révision. Nous étudions les changements prenant place dans les métriques logicielles durant l'évolution d'un projet. Au travers de multiples études exploratoires, nous réalisons des expériences quantitatives et qualitatives sur plusieurs jeux de données extraits à partir d'un ensemble de 13 projets. Nous extrayons les changements dans les métriques logicielles de chaque commit et construisons un jeu de donnée annoté manuellement comme vérité de base. Nous avons identifié plusieurs catégories en analysant ces changements. Un motif en particulier nommé "compromis", dans lequel certaines métriques peuvent s'améliorer au détriment d'autres, s'est avéré être un indicateur prometteur de changements liés à la conception -- dans certains cas, il laisse également entrevoir une intention de conception consciente de la part des auteurs des changements. Pour démontrer les observations de nos études exploratoires, nous construisons un modèle général pour identifier l'application d'un ensemble bien connu de principes de conception dans de nouveaux projets. Nos résultats suggèrent que les fluctuations de métriques ont le potentiel d'être des indicateurs pertinents pour gagner des aperçus macroscopiques sur l'évolution de la conception dans l'historique de développement d'un projet. / Version control is the backbone of the modern software development workflow. While building more and more complex systems, developers have to understand unfamiliar subsystems of source code. Understanding the logic of unfamiliar code is relatively straightforward. However, understanding its design and its genesis is often only possible through scattered and unreliable commit messages and project documentation -- when they exist. Thus, developers need a reliable and relevant baseline to understand the history of software projects. In this thesis, we take the first steps towards understanding change patterns in commit histories. We study the changes in software metrics through the evolution of projects. Through multiple exploratory studies, we conduct quantitative and qualitative experiments on several datasets extracted from a pool of 13 projects. We mine the changes in software metrics for each commit of the respective projects and manually build oracles to represent ground truth. We identified several categories by analyzing these changes. One pattern, in particular, dubbed "tradeoffs", where some metrics may improve at the expense of others, proved to be a promising indicator of design-related changes -- in some cases, also hinting at a conscious design intent from the authors of the changes. Demonstrating the findings of our exploratory studies, we build a general model to identify the application of a well-known set of design principles in new projects. Our overall results suggest that metric fluctuations have the potential to be relevant indicators for valuable macroscopic insights about the design evolution in a project's development history.

Statische Codemetriken sind wichtige Indikatoren für die Qualität eines Softwaresystems. Sie beleuchten dabei unterschiedliche Aspekte eines Softwaresystems. Deshalb ist es notwendig, mehrere Codemetriken zu nutzen, um die Qualität eines Softwaresystems in seiner Gesamtheit bewerten zu können. Wünschenswert wäre zudem eine Darstellung, die die Struktur des Gesamtsystems und die Bewertung einzelner Elemente eines Softwaresystems in einer Darstellung kombiniert. Die Arbeit untersucht deshalb, welche Metaphern geeignet sind, um eine solche Darstellung zu ermöglichen. Ein zweites Ziel der Arbeit war es, eine solche Visualisierung automatisch erzeugen zu können. Dafür wurde ein Generator entwickelt, der diese Anforderung erfüllt. Zur Konzeption dieses Generators kamen Techniken aus der generativen Softwareentwicklung zum Einsatz. Bei der Umsetzung des Generators wurde auf Techniken aus der modellgetriebenen Softwareentwicklung zurückgegriffen, vor allem auf Techniken aus dem openArchitectureWare-Framework. Der Generator kann in Eclipse eingebunden werden und ist in der Lage, aus einem Java-Projekt die Struktur und die Metrikwerte automatisch zu extrahieren. Diese Werte werden daraufhin in ein dreidimensionales Modell überführt, das auf dem offenen Extensible 3D Standard basiert. Der Generator ermöglichte zudem die Evaluierung zweier unterschiedlicher Metaphern, die im Rahmen der Arbeit durchgeführt wurde.

The dissertation aims to obtain an integrated and comprehensive perspective on measurement issues that play a strategic role in organisations that aim at continuous quality improvement through TQM. The multidimensional definition of quality is proposed to view quality holistically. The definition is dynamic, thus dimensions are subject to evolution. Measurement of the quality dimensions is investigated. The relationship between quality and cost, productivity and profitability respectively is examined. The product quality dimensions are redefined for processes. Measurement is a strategic component ofTQM. Integration of financial measures with supplier-; customer-; performance- and internal process measurement is essential for synergism. Measurement of quality management is an additional strategic quality dimension. Applicable research was integrated. Quantitative structures used successfully in industry to achieve quality improvement is important, thus the quality management maturity grid, cleanroom software engineering, software factories, quality function deployment, benchmarking and the ISO 9000 standards are briefly described. Software Metrics Programs are considered to be an application of a holistic measurement approach to quality. Two practical approaches are identified. A framework for initiating implementation is proposed. Two strategic software measurement issues are reliability and cost estimation. Software reliability measurement and modelling are introduced. A strategic approach to software cost estimation is suggested. The critical role of data collection is emphasized. Different approaches to implement software cost estimation in organisations are proposed. A total installed cost template as the ultimate goal is envisaged. An overview of selected software cost estimation models is provided. Potential research areas are identified. The linearity/nonlinearity nature of the software production function is analysed. The synergy between software cost estimation models and project management techniques is investigated. The quantification aspects of uncertainty in activity durations, pertaining to project scheduling, are discussed. Statistical distributions for activity durations are reviewed and compared. A structural view of criteria determining activity duration distribution selection is provided. Estimation issues are reviewed. The integration of knowledge from dispersed fields leads to new dimensions of interaction. Research and practical experience regarding software metrics and software metrics programs can be successfully applied to address the measurement of strategic indicators in other industries. / Business Management / D. Phil. (Operations Research)

Made available in DSpace on 2016-12-23T14:33:37Z (GMT). No. of bitstreams: 1 Ciro Xavier Maretto -- parte1.pdf: 1560777 bytes, checksum: b87dbad912d2ed99abdf096e4b07bdbc (MD5) Previous issue date: 2013-08-19 / Durante a execução dos projetos de software, muitos dados precisam ser coletados, armazenados e analisados para apoiar decisões tanto no contexto dos projetos quanto da organização. A medição de software é prática fundamental para o gerenciamento de projetos e para a melhoria de processos e está presente nos principais modelos e padrões de melhoria de processos de software, como a ISO/IEC 12207 (ISO/IEC, 2008), o CMMI (Capability Maturity Model Integration) (SEI, 2010) e o MR-MPS-SW (Modelo de Referência para Melhoria de Processo de Software Brasileiro) (SOFTEX, 2012). Em modelos que tratam a melhoria de processos em níveis, tais como o CMMI e o MR-MPS-SW, a medição é introduzida nos níveis iniciais (nível 2 do CMMI e nível F do MR-MPS-SW) e evolui à medida que a maturidade dos processos aumenta. Na alta maturidade, caracterizada nos níveis 4 e 5 do CMMI e B e A do MR-MPS-SW, a medição evolui, sendo necessário realizar o controle estatístico dos processos (CEP). Para realizar a medição de software de forma efetiva, é necessária uma infraestrutura computacional para apoiar a coleta, o armazenamento e a análise de medidas. Nesse sentido, este trabalho propõe uma arquitetura de referência para medição de software, que considera tanto a medição tradicional quanto em alta maturidade. A arquitetura proposta é independente de plataforma e foi definida com base na Ontologia de Referência para Medição de Software (BARCELLOS, 2009). Ela pode ser utilizada como base para a definição de arquiteturas específicas para soluções computacionais que apoiem o processo de medição de software. Como prova de conceito, a arquitetura de referência proposta foi utilizada como base para a definição de uma arquitetura específica e uma ferramenta. Além disso, como uma avaliação preliminar do trabalho, um estudo experimental foi realizado / During the execution of software projects, it is necessary to collect, store and analyze data in order to support project and organizational making decisions. Software measurement is a fundamental practice for project management and process improvement. It is present in the main models and standards that address software process improvement, such as ISO/IEC 12207 (ISO/IEC, 2008), CMMI (Capability Maturity Model Integration) (SEI, 2010) and MR-MPS-SW (Reference Model for Process Improvement of Brazilian Software) (SOFTEX, 2012). In models that address software process improvement in levels, such as CMMI and MR-MPS-SW, measurement starts in the initial levels (CMMI level 2 and MR-MPS-SW level F) and evolves as the maturity level increases. In the high maturity, characterized in CMMI levels 4 and 5 and MR-MPS-SW level B and A, measurement includes carrying out statistic process control (SPC). In order to perform the measurement process in an effective way, a computational infrastructure able to support measurement collection, storage and analysis is necessary. In this sense, this work proposes a reference architecture for software measurement that considers both traditional and high maturity measurement. The proposed architecture is platform independent and has been defined based on the Reference Ontology for Software Measurement (BARCELLOS, 2009). It can be used as a basis for defining specific architectures for computational solutions that support the software measurement process. As a proof of concept, the reference architecture was used as a basis for a specific architecture and a tool. In addition, aiming at evaluating the reference architecture proposed, an experimental study was carried out and some preliminary results were obtained