As a software project ages, its source code is improved by refining existing features, adding new ones, and fixing bugs. Software developers can attest that such changes often require accompanying changes to the infrastructure that converts source code into executable software packages, i.e., the build system. Intuition suggests that these build system changes slow down development progress by diverting developer focus away from making improvements to the source code. While source code evolution and maintenance is studied extensively, there is little work that focuses on the build system. In this thesis, we empirically study the static and dynamic evolution of build system complexity in proprietary and open source projects. To help counter potential bias of the study, 13 projects with different sizes, domains, build technologies, and release strategies were selected for examination, including Eclipse, Linux, Mozilla, and JBoss. We find that: (1) similar to Lehman's first law of software evolution, Java build system specifications tend to grow unless explicit effort is invested into restructuring them, (2) the build system accounts for up to 31\% of the code files in a project, and (3) up to 27\% of source code related development tasks require build maintenance. Project managers should include build maintenance effort of this magnitude in their project planning and budgeting estimations. / Thesis (Master, Computing) -- Queen's University, 2011-01-31 12:46:06.082

No description available.

Continues Integration has been a vital part of software development process in order to make the development process fast and reliable. There are number of actors which play an important role with support of third party tools that helps the development process to be effective and productive in nature. The CI- toolchain is capable of doing much more than the compilation of the software project which covers the daily life tasks of the developers like testing, documentation etc. The important part of automated toolchain is the conversion of source code artifacts into executables with the help of the build system. The selection of proper build system is a matter of subjective in nature and it depends upon the number of factors that should be analyzed before proceeding forward towards the selection mechanism. This thesis focuses on software rebuilding and proves practically with experiments that could help developers and managers to decide between two important software build systems SCons and CMake. It has been experimentally proved that what are the conditions and situations where SCons performs better and what are the moments where it is wise to select CMake as a build tool. At first, individual build tools are evaluated in terms of scalability, conveniency, consistency, correctness, performance (in terms of speed and targets) and later, the build systems are experimented by automating the workflow by increasing the source code artifacts to evaluate the performance when there is limited user interaction. The behavior of the build systems are also tried with other third party tools like Tessy for testing purposes, Jenkins as CI server, and Polarion as requirement engineering tool to show how much effort is required to integrate third party tools with the build system in order to increase the functionality. The evaluation of the build systems is important because that will highlights the areas where potential candidates are better and where there is lack of functional specifications. Generally speaking, SCons has an advantage of being Pythonic in nature and provides the developer ease of use to specify the build configurations using programmatic skills. CMake on other hand are on top of shelves where there is no need to understanding and caring about the underlying platform and where developers want to generate the native build tool solutions which are readily available for exporting them into IDEs. Though both of the build systems has different goals, for example SCons is ready to sacrifices the performance while providing user correctness of the build while CMake focuses on generating native build tools by understanding the underlying platform. All of these types of situations are discussed with experiments in this thesis and serves as the practical guides for high level managers to decide the build tools among others. After evaluation, this thesis firstly suggests the general techniques where the bottlenecks could be covered and then build tool specific optimizations and recommendations are discussed to speed-up the development process.

Continuous software engineering is a rapidly growing discipline in software engineering. Among its many reported benefits is increased development velocity, faster feedback for developers, and better software quality. It also comes with its own share of challenges, most of which are centered on making automated builds more efficient or detecting problems with build configuration. However, the majority of literature in this area does not take into account software developers, which are arguably the cornerstone of software development. Software development is still a human-driven endeavour. It is a developer who writes the code, tests it, makes the final decision while factoring in the build results, and so on. Furthermore, software development does not happen in a vacuum. Development takes place within the context of practices dictating how it should be done, and perceived benefits that drive practice adoption and implementation. Software development, and by extension continuous software development, is a socio-technical endeavour that features interactions between human aspects (developers, testers, etc.), technical aspects (automation), and environmental aspects (process, project-specific characteristics, infrastructure, etc.). While the software engineering field has its share of theories, frameworks, and models, or borrows them from other fields, we still do not have a human-centric framework for software engineering that takes into account other socio-technical aspects (technical and environmental). My dissertation addresses this need for a socio-technical framework by illustrating a series of studies that ultimately resulted in the creation of a socio-technical theory of continuous software engineering that focuses on phenomena involving both humans and automation. In particular, I focus on the role of continuous software engineering tools (automation) in the software development process and how they displace existing tools, disrupt existing workflows, and feature in software developer decision making. This theory will enable further research in this area as well as allow researchers to make more grounded recommendations for industrial applications. / Graduate

Software Supply Chain attacks are becoming more frequent. It is not enough to trust the source code of a project; the build process can insert malicious contents into build artefacts. This calls for the need of valid verification methods regarding the build process, and a good way of doing so is ensuring that the build process is deterministic. This means, that given two binaries built from the same source code and in the same environment, the resulting build artefacts should be bit-wise identical. There are existing tools that check this, but they are not integrated into build systems. This thesis resulted in an extension of GNU make which is called rmake, where diffoscope - a tool for detecting differences between a large number of file types - was integrated into the workflow of make. rmake was later used to answer the posed research questions for this thesis. We found that different build paths and offsets are a big problem as three out of three tested Free and Open Source Software projects all contained these variations. The results also showed that gcc’s optimisation levels did not affect reproducibility, but link-time optimisation embeds a lot of unreproducible information in build artefacts. Lastly, the results showed that build paths, build ID’s and randomness are the three most common groups of variations encountered in the wild and potential solutions for some variations were proposed.

Upgrading or migrating a build system can be a daunting task. Complete build system migration requires significant effort. To make the process more effective, we automated the first steps of migration, and attempted to analyze the new build results to find anomalies. Our findings show promise for automation as a first step of migration, and we see that automated evaluation could have some potential.

Continues Integration has been a vital part of software development process in order to make the development process fast and reliable. There are number of actors which play an important role with support of third party tools that helps the development process to be effective and productive in nature. The CI- toolchain is capable of doing much more than the compilation of the software project which covers the daily life tasks of the developers like testing, documentation etc. The important part of automated toolchain is the conversion of source code artifacts into executables with the help of the build system. The selection of proper build system is a matter of subjective in nature and it depends upon the number of factors that should be analyzed before proceeding forward towards the selection mechanism. This thesis focuses on software rebuilding and proves practically with experiments that could help developers and managers to decide between two important software build systems SCons and CMake. It has been experimentally proved that what are the conditions and situations where SCons performs better and what are the moments where it is wise to select CMake as a build tool. At first, individual build tools are evaluated in terms of scalability, conveniency, consistency, correctness, performance (in terms of speed and targets) and later, the build systems are experimented by automating the workflow by increasing the source code artifacts to evaluate the performance when there is limited user interaction. The behavior of the build systems are also tried with other third party tools like Tessy for testing purposes, Jenkins as CI server, and Polarion as requirement engineering tool to show how much effort is required to integrate third party tools with the build system in order to increase the functionality. The evaluation of the build systems is important because that will highlights the areas where potential candidates are better and where there is lack of functional specifications. Generally speaking, SCons has an advantage of being Pythonic in nature and provides the developer ease of use to specify the build configurations using programmatic skills. CMake on other hand are on top of shelves where there is no need to understanding and caring about the underlying platform and where developers want to generate the native build tool solutions which are readily available for exporting them into IDEs. Though both of the build systems has different goals, for example SCons is ready to sacrifices the performance while providing user correctness of the build while CMake focuses on generating native build tools by understanding the underlying platform. All of these types of situations are discussed with experiments in this thesis and serves as the practical guides for high level managers to decide the build tools among others. After evaluation, this thesis firstly suggests the general techniques where the bottlenecks could be covered and then build tool specific optimizations and recommendations are discussed to speed-up the development process.

Software applications are built by teams of developers that constantly iterate over the codebase. Software projects rely on a build system, which handles the management of dependencies, compilation, testing, and deployment of the software. The execution of the tests during each build allow developers to validate that their changes do not introduce regressions. However, the execution of the test suite during each build can take a long time, potentially impacting the development process. To facilitate quicker feedback, build systems use incremental building in order to avoid the reprocessing of unmodified artifacts. This is achieved by maintaining a cache of source files, and only rebuilding artifacts that differ from their cached version. Yet, changing any part of a source file invalidates the cache, triggering the re-execution of unmodified tests. This focus over an entire file can be misleading to the build system, as it can not determine whether the actual function being tested has changed, thus invoking redundant re-testing. In this thesis, we propose a finer-grained approach to caching within build systems, by caching components within the Abstract Syntax Tree instead of entire source files. We compare their hashes on subsequent runs, in order to identify components that have changed. The potential advantage of this strategy is that re-running a specific test that has not been modified will leverage the use of caches even if the file that contains it has been modified. We implement our approach in a system called TRICORDER, and integrate it within a build system called WARP. TRICORDER works by analyzing RUST source code in order to identify the test cases that have not been changed, such as through the addition of comments, or modifications of unrelated functions. This can benefit developers by avoiding the re-execution of tests that are unmodified. We evaluate our approach against 4 notable, open-source RUST projects, targeting a set of 16 tests within them. We first analyze the accuracy with which TRICORDER detects the internal dependencies of a test function, which is needed for the code slicing done by TRICORDER, in order to cache code items related to the target test function. We then introduce artificial changes to our study subjects in order to determine whether or not TRICORDER indicates tests that need to be re-run. Finally, we analyze the ability of TRICORDER to identify real changes based on the commit history of our study subjects. Our results show that the more granular approach to caching can avoid the unnecessary recompilation and re-execution of test cases. An important direction for future work is to extend the current implementation to support the entire set of RUST features in order to evaluate TRICORDER on a larger set of study subjects. / Programvaruapplikationer byggs av utvecklingsteam som ständigt itererar över kodbasen. Programvaruprojekt förlitar sig på ett byggsystem som hanterar beroenden, kompilering, testning och implementering av programvaran. Utförande av testerna under varje byggprocess möjliggör för utvecklare att validera att deras ändringar inte introducerar regressionsfel. Dock kan utförningen av testsviten under varje byggprocess ta lång tid och potentiellt påverka utvecklingsprocessen. För att underlätta snabbare återkoppling använder byggsystemen inkrementell byggning för att undvika onödig återbearbetning av oförändrade artefakter. Detta uppnås genom att bibehålla en cache av källkodsfilerna och endast bygga om artefakter som skiljer sig från deras cachade version. Att ändra vilken del som helst av en källkodsfil invaliderar cachet och utlöser körningen av oförändrade tester. Fokuseringen på en hel fil kan vara vilseledande för byggsystemet, då det inte kan avgöra om den faktiska funktionen som testas har ändrats och därigenom påbörjar onödig omtestning. I detta projekt föreslår vi en mer detaljerad cache-strategi inom byggsystem, genom att cacha komponenter inom det abstrakta syntaxträdet istället för hela källkodsfiler. Vi jämför deras hash-värden vid senare körningar för att identifiera ändringar. Den potentiella fördelen med denna strategi är när man kör om ett specifikt test som inte har ändrats kan cachen användas även om filen som innehåller testet har modifierats. Vi implementerar vår metod i ett system som kallas TRICORDER och integrerar det i ett byggsystem som heter WARP. TRICORDER fungerar genom att analysera RUST-källkod för att identifiera testfall som inte har ändrats, till exempel genom tillägg av kommentarer eller ändringar av irrelevanta funktioner. Detta kan gynna utvecklare genom att undvika att köra om tester som inte har ändrats. Vi utvärderar vår metod mot 4 välkända öppen källkodsprojekt i RUST och riktar in oss på en uppsättning av 16 tester inom dem. Först analyserar vi noggrannheten med vilken TRICORDER identifierar de interna beroendena hos en testfunktion, vilket behövs för kodavskärningen som TRICORDER utför för att cachelagra kodenheter relaterade till måltestfunktionen. Sedan inför vi konstgjorda ändringar i våra studieobjekt för att avgöra om TRICORDER indikerar tester som behöver köras om. Slutligen analyserar vi TRICORDER förmåga att identifiera verkliga ändringar baserat på ändringshistoriken för våra studieobjekt. Våra resultat visar att den mer granulära cachelagringsmetoden kan undvika onödig omkompilering och omkörning av testfall. En viktig riktning för framtida arbete är att utöka den nuvarande implementationen för att stödja hela uppsättningen av RUST-funktioner för att utvärdera TRICORDER på en större uppsättning studieobjekt. / Aplicațiile software sunt dezvoltate de programatori care iterează constant asupra codului. Proiectele de software se bazează pe un sistem de generare care gestionează dependențele, compilarea, testarea și lansarea software-ului. Execuția testelor permite dezvoltatorilor să valideze că modificările lor nu introduc regresii. Cu toate acestea, execuția testelor în cadrul fiecărei generări poate dura mult timp, având potențialul de a incetinii dezvoltarea. Pentru a facilita o reprocesare mai rapidă, sistemele de generare utilizează construirea incrementală pentru a evita reprelucrarea a artefactelor nemodificate. Acest lucru se realizează prin menținerea unei cache și reconstruirea doar a artefactelor care diferă de cele din cache. Cu toate acestea, orice modificare a unui fișier sursă invalidează cache-ul, declanșând reprocesarea. Focalizarea asupra unui fișier întreg poate induce în eroare sistemul de generare, deoarece nu poate determina dacă funcția testată a suferit modificări, declanșând astfel teste redundante. În această teză, propunem o abordare mai detaliată a cache-ului în cadrul sistemelor de generare, prin cacharea componentelor Arborelui Sintactic Abstract, în locul întregilor fișiere sursă. Comparăm hash-urile acestora în rulările ulterioare pentru a identifica componentele modificate. Avantajul potențial al acestei strategii constă în faptul că reexecutarea unui test care nu a nemodificat va utiliza cache-urile chiar dacă fișierul a fost modificat. Implementăm abordarea noastră într-un sistem numit TRICORDER și îl integrăm într-un sistem de construire numit WARP. TRICORDER funcționează prin analizarea codului sursă RUST pentru a identifica cazurile de testare care nu au fost modificate, cum ar fi prin adăugarea de comentarii sau modificări ale funcțiilor nerelevante. Acest lucru poate fi benefic pentru dezvoltatori, evitând reexecutarea testelor care nu au fost modificate. Evaluăm abordarea noastră în raport cu 4 proiecte notabile open-source în RUST, având în vedere un set de 16 teste în cadrul acestora. Mai întâi, analizăm precizia cu care TRICORDER detectează dependențele interne ale unei funcții de testare, ceea ce este necesar pentru tăierea de cod realizată de TRICORDER, pentru a memora în cache elementele de cod legate de funcția de testare țintă. Apoi, introducem modificări artificiale în subiecții noștri de studiu pentru a determina dacă TRICORDER indică sau nu teste care trebuie reluate. În final, analizăm capacitatea TRICORDER de a identifica schimbări reale pe baza istoricului de angajări al subiecților noștri de studiu. Rezultatele noastre arată că abordarea mai granulară a memorării în cache poate evita recompilarea și reexecutarea inutilă a cazurilor de testare. O direcție importantă pentru viitor este extinderea implementării curente pentru a sprijini întregul set de caracteristici RUST, pentru a evalua TRICORDER pe un set mai mare de subiecți de studiu.