Submitted by Automa??o e Estat?stica (sst@bczm.ufrn.br) on 2016-08-09T22:42:29Z No. of bitstreams: 1 FelipeAlvesPereiraPinto_TESE.pdf: 3382982 bytes, checksum: 85ac4517804b6893a9d66c8dc3ea1c78 (MD5) / Approved for entry into archive by Arlan Eloi Leite Silva (eloihistoriador@yahoo.com.br) on 2016-08-10T21:37:28Z (GMT) No. of bitstreams: 1 FelipeAlvesPereiraPinto_TESE.pdf: 3382982 bytes, checksum: 85ac4517804b6893a9d66c8dc3ea1c78 (MD5) / Made available in DSpace on 2016-08-10T21:37:28Z (GMT). No. of bitstreams: 1 FelipeAlvesPereiraPinto_TESE.pdf: 3382982 bytes, checksum: 85ac4517804b6893a9d66c8dc3ea1c78 (MD5) Previous issue date: 2015-11-18 / The maintenance and evolution of software systems have become a critical task over the last years due to the diversity and high demand of features, devices and users. The ability to understand and analyze how newly introduced changes impact the quality attributes of the architecture of those software systems is an essential prerequisite for avoiding the deterioration of the engineering quality of them during their evolution. This thesis proposes an automated approach for the deviation analysis of the quality attribute of performance in terms of execution time (response time). It is implemented by a framework that adopts dynamic analysis and mining software repository techniques to provide an automated way to revel potential sources - commits and issues - of performance deviation in scenarios of an evolving software system. The approach defines four phases: (i) preparation - choosing the scenarios and preparing the target releases; (ii) dynamic analysis - determining the performance of scenarios and methods by calculating their execution time; (iii) deviation analysis - processing and comparing the results of the dynamic analysis for different releases; and (iv) repository mining - identifying development issues and commits associated with performance deviation. Several empirical studies have been developed to assess the approach from different perspectives. An initial study shows the feasibility of the approach to support traceability of quality attributes with static analysis. An exploratory study analyzed the usefulness and domain independence of the proposal in automatically identifying source code assets with performance deviation and the changes that have affected them during an evolution. This study was performed using three systems: (i) SIGAA - a web academic management system; (ii) ArgoUML - an UML modeling tool; and (iii) Netty - a network application framework. A third study has performed an evolutionary analysis of applying the approach to multiple releases of Netty, and the web frameworks Wicket and Jetty. It has analyzed twenty-one releases (seven releases of each system) and addressed a total of 57 scenarios. Overall, we have found 14 scenarios with significant performance deviation for Netty, 13 for Wicket, and 9 for Jetty. In addition, the feedback obtained from an online survey with eight developers of Netty, Wicket and Jetty is also discussed. Finally, in our last study, we built a performance regression model in order to indicate the properties of code changes that are more likely to cause performance degradation. We mined a total of 997 commits, of which 103 were retrieved from degraded code assets, 19 from optimized, while 875 had no impact on execution time. Number of days before release and day of week were the most relevant variables of commits that cause performance degradation in our model. The receiver operating characteristic (ROC) area of our regression model is 60%, which means that deciding if a commit will cause performance degradation or not by using the model is 10% better than a random guess. / A manuten??o e evolu??o de sistemas de software tornou-se uma tarefa bastante cr?tica ao longo dos ?ltimos anos devido ? diversidade e alta demanda de funcionalidades, dispositivos e usu?rios. Entender e analisar como novas mudan?as impactam os atributos de qualidade da arquitetura de tais sistemas ? um pr?-requisito essencial para evitar a deteriora??o de sua qualidade durante sua evolu??o. Esta tese prop?e uma abordagem automatizada para a an?lise de varia??o do atributo de qualidade de desempenho em termos de tempo de execu??o (tempo de resposta). Ela ? implementada por um framework que adota t?cnicas de an?lise din?mica e minera??o de reposit?rio de software para fornecer uma forma automatizada de revelar fontes potenciais ? commits e issues ? de varia??o de desempenho em cen?rios durante a evolu??o de sistemas de software. A abordagem define quatro fases: (i) prepara??o ? escolher os cen?rios e preparar os releases alvos? (ii) an?lise din?mica ? determinar o desempenho de cen?rios e m?todos calculando seus tempos de execu??o? (iii) an?lise de varia??o ? processar e comparar os resultados da an?lise din?mica para releases diferentes? e (iv) minera??o de reposit?rio ? identificar issues e commits associados com a varia??o de desempenho detectada. Estudos emp?ricos foram realizados para avaliar a abordagem de diferentes perspectivas. Um estudo explorat?rio analisou a viabilidade de se aplicar a abordagem em sistemas de diferentes dom?nios para identificar automaticamente elementos de c?digo fonte com varia??o de desempenho e as mudan?as que afetaram tais elementos durante uma evolu??o. Esse estudo analisou tr?s sistemas: (i) SIGAA ? um sistema web para ger?ncia acad?mica? (ii) ArgoUML ? uma ferramenta de modelagem UML? e (iii) Netty ? um framework para aplica??es de rede. Outro estudo realizou uma an?lise evolucion?ria ao aplicar a abordagem em m?ltiplos releases do Netty, e dos frameworks web Wicket e Jetty. Nesse estudo foram analisados 21 releases (sete de cada sistema), totalizando 57 cen?rios. Em resumo, foram encontrados 14 cen?rios com varia??o significante de desempenho para Netty, 13 para Wicket e 9 para Jetty. Adicionalmente, foi obtido feedback de oito desenvolvedores desses sistemas atrav?s de um formul?rio online. Finalmente, no ?ltimo estudo, um modelo de regress?o para desempenho foi desenvolvido visando indicar propriedades de commits que s?o mais prov?veis a causar degrada??o de desempenho. No geral, 997 commits foram minerados, sendo 103 recuperados de elementos de c?digo fonte degradados e 19 de otimizados, enquanto 875 n?o tiveram impacto no tempo de execu??o. O n?mero de dias antes de disponibilizar o release e o dia da semana se mostraram como as vari?veis mais relevantes dos commits que degradam desempenho no nosso modelo. A ?rea de caracter?stica de opera??o do receptor (ROC ? Receiver Operating Characteristic) do modelo de regress?o ? 60%, o que significa que usar o modelo para decidir se um commit causar? degrada??o ou n?o ? 10% melhor do que uma decis?o aleat?ria.