The purpose of this study is to continue development of a benchmarking model to help companies assess their technical maturity when it comes to adopting Continuous Integration, Continuous Delivery and DevOps in their organization. The goal of the research is to assess how to improve the quality of qualitative models. Which conclusions can be drawn from comparing companies using benchmark and to assess which actions are the most effective to take to reach higher Continuous Integration, Continuous Delivery and DevOps maturity. The benchmark consisted of a questioner of two hundred statements that were answered for level of agreement with a current situation analysis and an ought-to-be analysis to be able to draw conclusions from the possible discrepancy between these categories. The questioner was answered during an interview study with chosen clients. Conclusions drawn from this study were that a lot can be done to improve the quality of qualitative models for examining Continuous Integration, Continuous Delivery and DevOps maturity. Different actions are necessary but the most important seems to be to ask open ended questions as well ass questions about different aspects of the same problem to promote discussion. It was also showed to be important to peer review the questions in the interview material beforehand to increase quality. The study also showed that it is possible to see trends in Continuous Integration, Continuous Delivery and DevOps maturity when comparing qualitative results for research subjects. The study showed that the most effective method for increasing Continuous Integration, Continuous Delivery and DevOps maturity is to use extensive automated testing suites that covers all testing disciplines. / Syftet med studien är att vidareutveckla ett benchmarkingverktyg för att hjälpa företag att bedöma sin tekniska mognad när det gäller att anta Continuous Integration, Continuous Delivery och DevOps i sin organisation. Målet med forskningen är att bedöma hur man kan förbättra kvaliteten på kvalitativa modeller för att mäta detta, samt vilka slutsatser som kan dras av att jämföra företags resultat som nyttjat studien. Samt att undersöka vilka åtgärder som är effektivast att ta för att nå en högre mognadsgrad inom Continuous Integration, Continuous Delivery och DevOps. Benchmarken bestod av ett frågebatteri av tvåhundra påståenden som besvarades av kunden i hur mycket de instämde till ett påstående. Resultatet samanställdes till en aktuell nulägesanalys och en börlägesanalys, med målet att dra slutsatser i vilka skillnaden som fanns mellan dessa två kategorier. Kunden besvarade frågebatteriet under en intervjustudie med utvalda anställda. Slutsatser som härrör från denna studie var att mycket kan göras för att förbättra kvaliteten på kvalitativa modeller för att undersöka Continuous Integration, Continuous Delivery och DevOps mognadsgrad. Olika åtgärder är möjliga, men det viktigaste förefaller vara att fråga öppna frågor för att främja diskussion samt att ställa frågor om olika aspekter av samma problem. Samt att opponera frågorna internt i intervjuundersökningen innan det utförs hos en kund, för att öka kvaliteten. Studien visade också att det även är möjligt att se trender i Continuous Integration, Continuous Delivery och DevOps mognad hos deltagarna när man jämför de kvalitativa resultaten. Studien visade att de mest effektiva metoderna för att öka Continuous Integration, Continuous Delivery och DevOps mognadsgrad är att använda omfattande automatiserade testsviter för samtliga testmetoder.

Manual deployments of software is a tedious, repetitive and non-scaling method of deploying software.Continuous Delivery is a practice that enables automated deployment of software in a rapid fashion at the click of a button.When deciding whether to start using a new practice, software companies need to make an assessment from a cost-benefit perspective.This thesis compares automated deployments through Continuous Delivery with manual deployments from a cost perspective.The comparison is done at a small software company where two tools for Continuous Delivery are chosen based on requirements imposed by the company. The tools, Octopus Deploy and Azure DevOps, are cost efficient to different degrees.Octopus is cost efficient if several deployments per week are necessary, particularly if many deployment targets are involved.Azure DevOps is quickly cost efficient in most cases due to its pricing scheme, only needing roughly one deployment per week for few deployment targets, and a couple of deployments per year for many deployment targets.The initial cost of having a paid employee set up the tool needs to be paid off, but is easily done within a year using weekly deployments with a small number of deployment targets.

Les architectures de microservices sont considérées comme une architecture qui promet pour réaliser DevOps dans les organisations informatiques, car elles divisent les applications en services pouvant être mis à jour indépendamment. Toutefois, pour protéger les propriétés SLA (Service Level Agreement) lors de la mise à jour des microservices, les équipes de DevOps doivent gérer des scripts d'opérations complexes et sujets aux erreurs. Dans cet article, on utilise une approche basée sur l'architecture pour fournir un moyen simple et sûr pour mettre à jour les microservices. / Microservice architectures are considered really promising to achieve DevOps in IT organizations, because they split applications into services that can be updated independently from each others. But to protect SLA (Service Level Agreement) properties when updating microservices, DevOps teams have to deal with complex and error-prone scripts of management operations. In this paper, we leverage an architecture-based approach to provide an easy and safe way to update microservices.

Many organizations have difficulty adopting advanced software development practices. Some software development project managers in large organizations are not aligned with the relationship between performance expectancy, effort expectancy, social influence, and facilitating conditions, as moderated by experience, with intent to adopt the DevOps practice of continuous delivery. The purpose of this study was to examine the statistical relationships between the independent variables—performance expectancy, effort expectancy, social influence, and facilitating conditions, as moderated by experience—and the dependent variable of behavioral intent to adopt a continuous delivery system. Venkatesh, Morris, Davis, and Davis's unified theory of acceptance and use of technology provided the theoretical framework. A stepwise multiple linear regression analysis was performed on survey data from 85 technical project managers affiliated with LinkedIn project management groups. The analysis reflected that only performance expectancy was significant in predicting intent to adopt continuous delivery. The findings may contribute to social change by providing project managers with the information they need to support organizational change, collaboration, and facilitation. The knowledge gained may additionally help organizations develop operational efficiency, competitive advantage, and generate higher value to their clients and society.

Manual deploys and testing of code can be both time-consuming and error-prone. Many repetitive manual steps can lead to critical tests or necessary configuration being forgotten or being skipped due time-constraints resulting in software that doesn’t work as intended when deployed to production. The purpose of this report is to examine whether continuous delivery(CD) could lead to any positive effects and if there are any obstacles for CD in an Episerver project at Sigma ITC. The study was done by implementing a CD pipeline for a project similar to a real project at Sigma then letting the developers work with it and interviewing them about their current workflow, their attitude towards the different steps involved and if they saw any problems with CD for their project. Even if the developers, in general, where positive to CD they had some reservations about it in their current projects. The main obstacle the developers saw where the time/cost which would affect the customer and also some uncertainty around the complexity in testing Episerver. The results show that there could be positive effects of CD even if the project type is not optimal for reaping all the CD benefits, it all depends on people involved seeing a value in testing and the questions around testing in Episerver are straightened out.

This thesis is dealing with continuous delivery of web applications. The theoretical part contains description and explanation of the basic concepts of the continuous delivery. The objectives, benefits, specifications and basic procedure of implementation of this process are summarized in this part of the thesis. Tools for final implementation of continuous delivery in software company are designed and implemented in the practical part of the thesis.

This thesis has been carried out at the Swedish Board of Agriculture. The client wants to orientate towards a development environment and architecture that allows for more frequent software deliveries than in the current situation, to increase the business benefits of the development work carried out. The purpose of this thesis is to develop a process to iteratively move towards an architecture and development environment that enable Continuous Delivery. Continuous delivery aims to a very high extent to treat a developer's code as part of a potential release candidate. This in turn causes high demands on being able to secure the reliability of both the infrastructure and the individual developers’ contributions. The work has been carried out in cooperation with developers, infrastructure engineers, architects and team leaders on The Swedish Board of Agriculture. Theories have been tested within the IT organization to ensure their reliability and applicability in the organization. A process has been developed with the limitation that it has not been tested in a production environment because of the limited timeframe available. However, it has been demonstrated that the process is feasible for the systems that acted as the main testing candidates during the thesis. / Detta examensarbete har utförts vid Statens Jordbruksverk. Uppdragsgivaren önskar att orientera sig mer mot en utvecklingsmiljö och arkitektur som möjliggör tätare leveranser än i dagsläget, för att öka verksamhetsnyttan av det utvecklingsarbete som genomförs. Syftet med detta examensarbete är att ta fram en process för att iterativt kunna gå mot en arkitektur som möjliggör för Continuous Delivery, eller kontinuerlig leverans. Kontinuerlig leverans syftar till att i mycket hög mån behandla en utvecklares kod som en del av en potentiell releasekandidat. Detta för i sin tur med sig höga krav på att kunna säkra tillförlitligheten av både infrastruktur samt den individuelle utvecklarens bidrag. Arbetet har utförts i samarbete med utvecklare, infrastrukturtekniker, arkitekter samt teamledare på Jordbruksverket. Teorier har testats inom IT-organisationen för att se dess tillförlitlighet samt tillämplighet på just Jordbruksverkets organisation. Arbetet påvisar att det är möjligt att dela upp monolitiska system och gå närmare något som liknar kontinuerlig leverans, utan att behöva genomföra stora förändringar inom organisationen. En process har tagits fram med begräsning att den inte testats i produktionsmiljö på grund av tidsbrist. Det har dock påvisats att processen är gångbar för det system som varit testkandidat genom arbetets gång.

This diploma thesis deals with the delivery of software products to end users and automation of related processes. After the initial introduction to the issue of deployment, as it is defined by one of the widely used software engineering methodology RUP (Rational Unified Process), the thesis devotes to practices as Continuous Integration, Continuous Delivery and Continuous Deployment which are directly oriented to the delivery of a software product based on automated processes. The second part depicts the design and the implementation of the solution for automation of the deployment of web applications in PHP within the specific developer company. The design part itself covers the analysis of the current state of the manual deploying process of business applications along with possible ways of automation its parts (updating the source code and database schema, application configuration, etc.), the definition of essential requirements for the solution and analysis of available tools which is the base for the selection of the most suitable tools for each component forming the resulting solution. The thesis also depicts interesting or nontrivial parts of implementation of each component, the practical case of usage of the system, the subsequent evaluation of the resulting solution along with estimated benefits and suggestions for the future expansion of its functionality.

Context: Digital transformation poses new challenges to the organization. Market needs and competition in business have changed the continuous testing environment and continuous delivery in software organizations. There is a great possibility of conflict between the development and testing operations because they are a separate entity in large organizations. Organizations where testers, developers, and operation teams do not communicate well and lack collaboration can have their productivity affected negatively. This leads to defects and errors at the early stage of the development process. DevOps’ approach enhances the integration, delivery, performance,and communication between the developers, testers, and operational members. Organizations are reluctant to apply DevOps practices because there is a lack of agreement on DevOps characteristics. The most difficult part of a large organization is DevOpsadaptation and its implementation because of its legacy structure. It is required to get an understanding of DevOps implementation in organizations before they start transforming. Objectives: The thesis aims to identify the challenges in organizations towards continuous delivery and provide a list of techniques or strategies to overcome continuous testing and DevOps challenges. This thesis also identifies the communication challenges between continuous testing and delivery teams during the COVID-19 pandemic and the software architecture effect on testing in the DevOps environment. Methods: To achieve the research goal a multiple research method techniques are applied. A systematic literature review is conducted to identify the literature and to meet the research goal. A survey is conducted for the verification of the data from SLR. An interview is used as a data collection method in Survey and explores the actual process of continuous testing and delivery in large DevOps companies. Results: A list of challenges to large organizations towards continuous delivery is generated. A list of strategies and solutions towards the challenges of continuous testing and DevOps is generated. A list of post COVID-19 communication challenges between testing and delivery groups in DevOps is created. A list of software architecture and production environment effects on testing is also generated. After analyzing the SLR results, a survey is conducted to validate the results from software practitioners. Thematic analysis is performed on the results. "Finally", the findings from the SLR and Survey are compared. Conclusions: This research’s findings can help researchers, industry practitioners, and someone who wants to investigate further the possible effects with the continuous testing and delivery during DevOps transformation. We observed that industry practitioners could enhance their communication channels by reviewing the post-COVID-19 communication challenges between testing and delivery teams. We also observed that there is more research required to continue on this topic.

Continuous delivery is a software development methodology that aims to reduce development cycle time by putting a strong emphasis on automation, quality and rapid feedback. This thesis develops an automated method for detecting performance regressions as part of a continuous delivery deployment pipeline. The chosen method is based on control charts, a tool commonly used within statistical process control. This method is implemented as part of a continuous delivery deployment pipeline and its ability to detect performance regressions is then evaluated by injecting various performance bottlenecks in a sample application. The results from this thesis show that using a control chart based approach is a viable option when trying to automate verification of load test results in the context of continuous delivery. / Kontinuerlig leverans är en utvecklingsmetodik för mjukvara med målet att reducera ledtid genom att fokusera på automatisering, kvalitet och snabb återkoppling. I det här examensarbetet utvecklas en automatiserad metod för att upptäcka försämringar i prestanda i en deployment pipeline för kontinuerlig leverans. Den valda metoden baseras på kontrolldiagram, ett verktyg som ofta används inom statistisk processkontroll. Metoden implementeras som en del av en deployment pipeline för kontinuerlig leverans och dess förmåga att upptäcka prestandaförsämringar utvärderas genom att olika prestandarelaterade flaskhalsar implementeras i en testapplikation. Resultaten från arbetet visar att en metod baserad på kontrolldiagram är ett tänkbart alternativ för att automatisera verifiering lasttestresultat inom kontinuerlig leverans.