Software quality attributes are properties that reflect the quality of a software system, and Non-functional requirements (NFRs) are the specifications that define how a software system should perform to reach a desired level of goals of the quality attributes.The evaluation of quality attributes is important to show the effectiveness of a system in meeting customers' NFRs. Continuous integration (CI) environments have emerged as powerful platforms for organizations to improve software quality through automated software verification and validation.Despite this, there is a growing need for evaluating quality attributes that is often met by in-house development of metrics and tools.This highlights the importance of quality attributes for software product quality. This thesis investigates the association between quality attributes and components of a CI environment, as well as how to utilize these components for evaluating software quality attributes.The focus is on improving the knowledge of the evaluation and providing specific recommendations for companies to enhance their CI environments for higher demands of quality evaluation.The contributions of this thesis include a better understanding of the relationship between quality attributes and CI components, and a set of practical guidelines for companies to effectively leverage CI for quality attribute evaluation. The studies in this thesis utilized mixed methodologies, including a systematic literature review, a multi-case study conducted in four software development companies, and an synthesis of the collected data.The multi-case study provided a comprehensive overview of practices for quality attribute evaluation and how CI components can generate data to support the evaluation of specific attributes.The synthesis study presents a maturity model based on the collected data from both academia and industry, and the model can aid organizations in assessing their current level of maturity in utilizing CI environments to identify potential improvements.The results in these studies show the capabilities of different components of a CI environment and how these components can be used to support the evaluation of quality attributes.While the use of CI environments for the thesis topic offers benefits, it also presents several challenges, for example, the challenge to identify effective quality metrics. In conclusion, this thesis contributes to the understanding of the use of CI environments for evaluating software quality attributes.The results suggest that CI environments can be an effective approach for quality attribute evaluation, but suitable metrics need to be considered to ensure accurate and meaningful evaluation results. Furthermore, the thesis presents areas for future research, such as the use of machine learning techniques to improve the accuracy of quality assessment using CI environments.

Modeling and simulation tools are exceptionally useful for designing aerospace systems because they allow engineers to test and iterate designs before committing the massive resources required for system realization. The Horizon Simulation Framework (HSF) is a time-driven modeling and simulation tool which attempts to optimize how a modeled system could perform a mission profile. After 15 years of development, the HSF team aims to achieve a wider user and developer base by releasing the software open source. To ensure a successful release, the software required extensive testing, and the main scheduling algorithm required protections against new code breaking old functionality. The goal of the work presented in this thesis is to satisfy these requirements and officially release the software open source. The software was tested with > 80% coverage and a continuous integration pipeline which runs build and unit/integration tests on every new commit was set up. Finally, supporting documentation and user resources were created and organized to promote community adoption of the software, making Horizon ready for an open source release.

The goals with continuous delivery are to reduce the risk, cost, and time of releasing software to the stakeholders and the users. It is a process which can result in reliable releases and reducing errors in the software. Furthermore, there are some best practices to follow when conducting the continuous delivery process, involving version control, and build tools. There are however some obstacles and challenges for organizations when moving to continuous delivery. For example, complex environments, organizational problems, and lack of automated test cases. This master thesis investigates continuous delivery through a literature review, a multiple-case study, and fieldwork. The result can either be used by software engineers and organizations who are new to the continuous delivery concept. Or the result can be used by more experienced software engineers to gain more knowledge about existing obstacles and for further research.

Kubernetes is a fast emerging technological platform for developing and operating modern IT applications. The capacity to deploy new apps and change old ones at a faster rate with less chance of error is one of the key value proposition of the Kubernetes platform. A continuous integration and continuous deployment (CI/CD) pipeline is a crucial component of the technology. Such pipelines compile all updated code and do specific tests and may then automatically deploy the produced code artifacts to a running system. There is a thriving ecosystem of CI/CD tools. Tools can also be divided into two types: integrated and standalone. Integrated tools will be utilized for both pipeline phases, CI and CD. The standalone tools will be used just for one of the processes, which needs the usage of two independent programs to build up the pipeline. Some tools predate Kubernetes and may be converted to operate on Kubernetes, while others are new and designed specifically for usage with Kubernetes clusters. CD systems are classified as push-style (artifacts from outside the cluster are pushed into the cluster) or pull-style (CD tool running inside the cluster pulling built artifacts into the cluster). Pull- and push-style pipelines will have an impact on how cluster credentials are managed and if they ever need to leave the cluster. This thesis investigates the deployment time, fault tolerance, and access security of pipelines. Using a simple microservices application, a testing setup is created to measure the metrics of the pipelines. Drone, Argo Workflows, ArgoCD, and GoCD are the tools compared in this study. These tools are coupled to form various pipelines. The pipeline using Kubernetes-specific tools, Argo Workflows and ArgoCD, is the fastest, the pipeline with GoCD is somewhat slower, and the Drone pipeline is the slowest. The pipeline that used Argo Workflows and ArgoCD could also withstand failures. Theother pipelines that used Drone and GoCD were unable to recover and timed out. Pull pipelines handles the Kubernetes access differently to push pipelines as the Kubernetes cluster credentials does not have to leave the cluster, whereas push pipelines needs the cluster credentials in the external environment where the CD tool is running.

All developers understand the benefits of testing their code to ensure its functionality. Today’s market is moving further towards design principles where testing is a central or driving force during development. This puts a certain pressure on academia to supply these skills to their students.Recently the course II1302 Projects and project methods at the Royal Institute of Technology in Kista made a concerted effort to introduce the students of the course to these modern concepts. This thesis investigates how areas of testing can effectively be introduced to the students in the course, utilizing a tailored example that takes the area of testing into particular consideration and how to automate it via DevOps-tools provided by a cloud-based service. Further, it also makes an attempt to provide additional material to be used for teaching testing in conjunction with the example provided.The case study covers the development of an example application, meant to mirror a typical student project. It also covers how this was used for teaching the students about the testing areas considered. The covered testing areas include unit testing, integration testing and UI testing. With these given testing areas, the application and an associated learning module was developed for each area in question. Relevant standards, strategies and approaches was also identified for each of these areas.The thesis also presents important properties to take into consideration when developing similar examples in the future, based on the experiences obtained during the study. These include needs such as understandable by inexperienced students, applicability outside the course, adherence to established standards, tools that are simple to use and an architectural structure that allows for testing.Some improvements are also recommended: the students would benefit from learning software testing from an early stage of their studies. The content of the learning modules should also be brought to the students earlier in the course, so it can be applied in their projects at an early stage as well.Further research is also recommended to evaluate the suitability of using other cloud-based environments instead, and to evaluate the applicability of the learning modules for students of varying disabilities. / Alla utvecklare förstår fördelarna med att testa kod för att garantera dess funktionalitet. Dagens industri går i en riktning där testning spelar en central del av design under utveckling av mjukvara. Denna tendens lägger en viss press på högskolan att lära ut dessa erfarenheter till dess studenter.På senare tid har kursen II1302 Projekt och projektmetoder på Kungliga Tekniska Högskolan i Kista tagit en stor ansats för att introducera sina studenter inför dessa moderna koncept. Denna uppsats undersöker hur testningsområdet effektivt kan introduceras till studenterna inom denna kurs, genom att utnyttja ett egengjort exempel som har området i fokus, samt att automatisera detta via DevOps-verktyg tillhandahållna av molnbaserade tjänster. Dessutom görs även en ansats för att tilldela ytterligare material som kan användas för att lära ut testning av mjukvara i samband med det givna exemplet.Fallstudien omfattar utvecklingen av en exempelapplikation, som var avsedd att likna ett typiskt studentprojekt. Den täcker även hur denna användes för att lära ut de betraktade testningsområdena till studenterna. De täckta områdena av testning inkluderar enhetstestning, integrationstestning och testning av användargränssnitt. Med dessa givna testningsområden utvecklades både applikationen och dess associerade lärmoduler för vardera testningområde i fråga. Relevanta standarder, strategier och metoder var också identifierade för vardera av dessa områden.Denna uppsats presenterar även ett antal viktiga egenskaper att hålla i åtanke vid utveckling av liknande exempel i framtiden, baserat på erfarenheterna från studien. Detta inkluderar behov som tillgänglighet för mindre erfarna studenter, applicerbarhet utanför själva kursen, tillämpning av etablerade standarder, utnyttjande av lättanvända verktyg och en arkitektur som tillåter testning.Några förbättringar föreslås även: studenterna skulle gynnas av att lära sig om mjukvarutestning i ett tidigt skede av sina studier. Innehållet i lärmodulerna bör även presenteras för studenterna tidigare i kursen för att kunna appliceras i deras projekt.Ytterligare forskning rekommenderas även för att utvärdera andra lämpliga molnbaserade miljöer, samt för att utvärdera tillämpbarheten av lärmodulerna hos studenter med inlärningssvårigheter.

There is a great demand for fast deliveries of improved and updated software in different software development areas, like Internet of Things, web, and cloud, in today’s digitalized world. Software developers and organizations must adapt to be able to deliver according to customers’ wishes, to be able to retain them, and remain competitive with other organizations. Continuous integration and continuous delivery (CI/CD) are methods used within the software development world, allowing developers to automate parts of their work to develop and deliver software faster and with better quality. Tools used for CI/CD come with different benefits and performances making it difficult for developers to choose a tool. There are numerous tools to choose from, and there is a lack of performance comparisons of them. This thesis aims to give developers a performance comparison between the two well-known CI/CD tools, GitHub Actions and Jenkins, to facilitate their choice of a CI/CD tool. The research was qualitative, inductive, and comparative. A literature study and practical tests were conducted to study the performance differences between the two wellknown CI/CD tools, GitHub Actions and Jenkins. The literature study was conducted f irst and gave the necessary knowledge to perform the practical tests, and the practical tests gave the actual results. The practical tests were performed on two different software projects ,and two different tests per projec, per server were conducted. The results from both projects indicated apparent differences in performance between GitHub Actions and Jenkins, as Jenkins ran faster than GitHub Actions while running on a Windows server, and GitHub Actions ran faster than Jenkins while running on an Ubuntu server. These findings indicate that the two well-known CI/CD tools perform differently depending on the server the developers would use these tools. It can not be concluded that one of the tools has better performance than the other; instead, one tool has better performance depending on the operating system the tool is running on. If the developers were to use the tools on an Ubuntu server, GitHub Actions would be the preferred tool, and if they were to use the tool on a Windows server, Jenkins would be the preferred tool. / Det finns en stor efterfrågan på snabba leveranser av förbättrad och uppdaterad mjukvara i olika mjukvaruutvecklings områden så som Sakernas Internet, webb och moln i dagens digitaliserade värld. Mjukvaruutvecklare och organisationer måste anpassa sig för att kunna leverera till kundernas önskemål för att kunna behålla dom och förbli konkurrenskraftiga med andra organisationer. Kontinuerlig integration och kontinuerlig leverans (CI/CD) är metoder som används inom mjukvaruutvecklings världen, så att utvecklare kan automatisera delar av sitt arbete för att utveckla och leverera mjukvara snabbare och med bättre kvalité. Verktyg som används för CI/CD kommer med olika fördelar och prestanda som gör det svårt för utvecklare att välja ett verktyg. Det finns många verktyg att välja mellan och det finns en brist på prestandajämförelser av dem. Detta examensarbete syftar till att ge utvecklare en prestandajämförelse mellan de två välkända CI/CD-verktygen GitHub Actions och Jenkins, för att underlätta utvecklarens val av ett CI/CD-verktyg. En kvalitativ, induktiv och komparativ forskningsmetod användes för att genomföra denna studie. En litteraturstudie och praktiska tester genomfördes för att studera prestandaskillnader mellan de två välkända CI/CD-verktygen GitHub Actions och Jenkins. Litteraturstudien genomfördes först och gav författarna nödvändiga kunskap för att utföra dem praktiska testerna, dem praktiska testerna gav de faktiska resultaten. Praktiska testerna utfördes på två olika mjukvaruprojekt och två olika tester per projekt, en per server genomfördes. Resultaten från båda projekten visade på uppenbara skillnader i prestanda mellan GitHub Actions och Jenkins. Då Jenkins kördes snabbare än GitHub Actions när körningen kördes på en Windows server och GitHub Actions kördes snabbare än Jenkins när de kördes på en Ubuntu server. Dessa resultat tyder på att de två välkända CI/CD-verktygen fungerar olika beroende på vilken server utvecklarna skulle använda dessa verktyg på. Det går inte att dra slutsatsen att ett verktyg är bättre över det andra, i stället har ett verktyg bättre prestanda beroende på vilket operativ system verktyget körs på. Om utvecklarna skulle använda verktygen på en Ubuntu server skulle GitHub Actions vara det föredragna verktyget och om utvecklarna skulle använda verktyget på en Window server skulle Jenkins vara det föredragna verktyget.

Context: The goal of this research is to form a correlation between code packages and test cases which is done by using automated weak mutation. The correlations formed is used as the statistical test data for selecting relevant tests from the test suite which decreases the size of the test suite and speed up the process. Objectives: In this study, we have done an investigation of existing methods for reducing the computational cost of automatic mutation testing. After the investigation, we build an open source automatic mutation tool that mutates the source code to run on the test cases of the mutated code that maps the failed test to the part of the code that was changed. The failed test cases give the correlation between the test and the source code which is collected as data for future use of the test selection. Methods: Literature review and Experimentation is chosen for this research. It was a controlled experiment done at the Swedish ICT company to mutate the camera codes and test them using the regression test suite. The camera codes provided are from the continuous integration of historical data. We have chosen experimentation as our research because as this method of research is more focused on analyzing the data and implementing a tool using historical data. A literature review is done to know what kind of mutation testing reduces the computational cost of the testing process. The implementation of this process is done by using experimentation Results: The comparative results obtained after mutating the source code with regular mutants and weak mutants we have found that regular mutants and weak mutants are compared with their correlation accuracy and we found that on regular mutation operators we got 62.1% correlation accuracy and coming to weak mutation operators we got 85% of the correlation accuracy. Conclusions: This research on experimentation to form the correlations in generating test selection statistics using automated mutation testing in the continuous integration environment for improving test cases selection in regression testing

When the traditional approach to system development transitioned over to agile methods the demand for ways to continuously deliver valuable software increased. With the increasing interest in these methods the umbrella term DevOps were introduced in 2008. DevOps is the practice to shorten the time, as well as streamlining the process, between changes in the source code and when it reaches the customer. This study aimed at investigating the process of adopting DevOps practices and identify success factors and challenges in this transition. We conducted a quality approached case study on a DevOps team in a multinational IT-consulting firm. Four explorative interviews were conducted based on categories identified in relevant literature as well as basis provided by the case company. We discovered that an implementation of continuous practices is far more complex than we initially thought. It is related to more fields outside the technical aspect that must be considered during a transition.  Our findings include four critical success factors for a successful DevOps implementation; conscious staff, conscious organization, functional infrastructure and adapted business model. Embedded in these critical success factors we were able to identify challenging fields that should be considered when adopting DevOps. These critical success factors and challenges was the basis for our model presented in the study.

Znížená efektivita spôsobená robením repetitívnych a manuálnych prác je častým problémom v IT. Vývojári často testujú a nasadzujú svoj software manuálne, čo je nielen náročné na čas, nezáživné a náchylné k chybám. Táto práca sa snaží v rámci jedného DevOps tímu, vyriešiť tento problém pomocou zjednotenia vývojárskych a testovacích nástrojov, a pomocou aplikovania pricípov CI a CD do produkčného prostredia. Zároveň sa sústredí na software využívajúci Python, Jenkins a kontajnery. Hlavnými použitými nástrojmi sú GitLab CI, OpenShift a Tox. Vďaka tejto práci sa podarilo zvýšit počet projektov, ktoré používajú CI/CD zo 7 na 50 percent, zvrátiť rastúci trend v počte porušení štýlu v jazyku Python, opatriť kontajnery metadátami, zautomatizovať proces tvorby kontajnerov, ušetriť čas nerobením repetitívnych úloh a pod.

One approach to continuously achieve a certain data quality level is to use an integration pipeline that continuously checks and monitors the quality of a data set according to defined metrics. This approach is inspired by Continuous Integration pipelines, that have been introduced in the area of software development and DevOps to perform continuous source code checks. By investigating in possible tools to use and discussing the specific requirements for RDF data sets, an integration pipeline is derived that joins current approaches of the areas of software development and semantic web as well as reuses existing tools. As these tools have not been built explicitly for CI usage, we evaluate their usability and propose possible workarounds and improvements. Furthermore, a real world usage scenario is discussed, outlining the benefit of the usage of such a pipeline.