Information system development projecs are often considered a costly and uncertain process were projects often exceed the scheduled time and budget. By continuously integrating source code and do regular builds problems can be discovered almost directly and thus minimizing the cost to fix them. In this qualitative study we have focused on which challenges a large public organization who cooperate with an external provider can face when adopting continuous integration and continuous delivery. We have interviewed six employees within the organisation and two employees from the external provider, who are all in some way connected to the software development process. The results showed that our interviewees are interested in agile software development and to be able to deliver high quality software continuously. We also found that the software development process is complex and has a lot of barriers and handovers that slows the process down. Our conclusion is that the organization needs to change their approach to software development. To achieve this the organisation needs to adopt Devops, which means erasing the barriers between development and operations.

With a software industry that is moving at a fast pace, continuous integration and delivery is something important for many products today. Moreover, with containers being on the rise since 2013, more companies are moving their CI/CD environment into containers not only for development but also for testing. This thesis begins with giving the reader an introduction to containers, container orchestration, and Jenkins, which is a continuous integration and delivery tool. The experiment was then set up with one container based cluster and one single node machine. Two kinds of experiments were run on them, one big job and one small job. The system scalability is assessed, and with smaller clusters the memory overhead could be an issue. Performance wise, the container cluster is performing better than a single node machine, as long as it is utilizing all its nodes. Security with containers is still an issue and it could be fatal for a cluster if it is compromised.

The adoption of Continuous Development presents many challenges to users and organizations. The complexity of Continuous Development adoption is partially attributable to the diversity of the challenges faced, including technical challenges, cultural challenges, compliance regulations, and lack of understanding. In this thesis, I worked with industry partner IBM to improve their Continuous Delivery Pipeline offering to overcome adoption challenges faced by their users. Following Hevner’s Three Cycle Design Science Methodology, my research had two distinct stages: Characterizing Continuous Development Adoption Challenges and Creating Design Requirements to Aid Organizations Offering Continuous Development Services. Both stages necessitated involvement from both academic literature and industry collaboration with IBM. Industry collaboration included interviews, surveys, developer forum analysis, and collaboration with IBM’s “Continuous Delivery” teams. The design requirements I developed in this thesis addressed cultural, technical, compliance, and knowledge gap adoption challenges that were identified during the problem characterization stage. When tested within the Continuous Development community, feedback indicated that my design requirements would add value to users’ development process, enabling their Continuous Development adoption. This thesis provides a foundation of empirical research for future study and a set of guidelines for industry practitioners. / Graduate

Continuous integration (CI) is a widely used practice in modern software engineering. Unfortunately, it is also an expensive practice — Google and Mozilla estimate their CI systems in millions of dollars. In this dissertation, I propose a collection of novel build selection approaches that are able to save the cost of CI. I also propose the first exhaustive comparison of techniques to improve CI including build and test granularity approaches. I firstly design a build selection approach (SMARTBUILDSKIP) for CI cost reduction in a balanceable way. The evaluation of SMARTBUILDSKIP shows that it can save a median of 30% of builds by only incurring a median delay of 1 build in a median of 15% of failing builds under its most conservative configuration. To minimize the delayed failure observation, I then propose the second build selection approach (PRECISEBUILDSKIP) that can save cost without delaying failure observation. We find that PRECISEBUILDSKIP can save a median of 5.5% of builds while capturing the majority of failing builds (100% in median) from the evaluation. After that, I evaluate the strengths and weaknesses of 10 techniques that can improve CI including SMARTBUILDSKIP. The findings of the comparison motivate my next work to design a hybrid technique (HYBRIDBUILDSKIP) that combines these techniques to produce more cost-saving while keeping a low proportion of failing builds that are delayed in observation. Finally, I design an experiment to understand how different weights of test duration among the whole build duration can influence the cost-saving of build and test selection techniques. / Doctor of Philosophy / Modern software developing teams commonly use the continuous integration as the practice of automating and testing the integration of code changes from multiple contributors into a single software project. The best practice of continuous integration requires this process happens as frequently as possible because the bugs can be found earlier and easier before the change sets grow too large. However, continuous integration process can be time-consuming and in most cases the code change is bug-free. This means that developers may have to wait for a long time only to get a result with no actionable feedback. Thus, in this dissertation, I present multiple selection approaches to selectively execute the continuous integration process based on the prediction of the outcome - if the outcome is predicted to be passing with no actionable feedback, the approach will decide to skip the current execution. The evaluation result shows that my approaches can save the cost of continuous integration while keeping the value of it (finding bugs earlier).

Quick feedback in regards to build times is important in Continuous Integration. If builds become too long, it can hurt the rate of software development. There are multiple methods to reduce build times. One commonly suggested method is to parallelize builds. This thesis aims to investigate the effects of parallelizing builds in Continuous Integration software and provide support for whether parallelizing is a good way of reducing build times or not. We conducted an experiment consisting of running tests on different Continuous Integration software with different configurations. These configurations changed how many tests were executed and how many parallel build agents were used. The aspects that were observed and analyzed was how build time, average CPU usage and CPU time were affected. What we found was that parallelizing a Continuous Integration build drastically improves build time, while RAM usage and CPU time remains similar. This entails that there are no major consequences to parallelizing other than utilizing more threads and therefore using more of the available CPU resources.

No description available.

Företag arbetar ofta efter hårda krav från kunder där lösningar måste levereras på ett tidseffektivt sätt och samtidigt hålla en hög kvalitet. Detta i form av felfria och robusta system vilket delvis kan åstadkommas med hjälp av testning. Kraven på snabb leverans och hög kvalitet är däremot motpoler till varandra; snabb leverans genomförs ofta på bekostnad av kvalitet och tvärtom. Agila arbetsmetoder med tidiga och frekventa leveranser har ändrat på detta, men kräver samtidigt en ständig kvalitetsförsäkring under arbetets löptid. Under utveckling av mjukvara enligt dessa metoder förekommer därför en kontinuerlig kvalitetssäkringsprocess för att säkerställa att produkten dels uppfyller vad kravspecifikationen avser samt att levererad produkt håller en hög tekniskt kvalitet i form av buggfri och robust kod som innehar stor pålitlighet för framtiden. Då manuell testning är en kostnad- och resurskrävande metod har automatiserad testning blivit ett aktuellt alternativ för ökad effektivitet och hållbar utveckling. Målet med det här arbetet har varit att för Precio Systemutveckling AB utreda möjligheterna för en implementation av automatiserad testning av användargränssnitt, i en hos företaget redan existerande och etablerad utvecklingsprocess. Arbetet har genomförts med en inledande förstudie om testning med fokus på varför det är en extra viktig faktor i dagsläget. Detta följs av ett avsnitt där existerande teori och teknik för testning i generell mening avhandlats, följt av en närmare studie på hur automatiserad testning är rekommenderad att utföras ur ett perspektiv från utveckling av produkter inom Microsoft-teknologi. / Software companies work under a strict pressure from customers where solutions must be delivered in a timely manner as well as providing high quality and value. The products should be robust and without errors, which partially can be accomplished by testing during the development process. The requirement for an early delivery and a high quality does not always come hand in hand, but with the increased use of agile software development methods, this can be achieved. During agile development of software, there is a continuous process to ensure the quality of what is being developed, both to make sure that all the functional requirements are fulfilled, but also that the code behind is robust and dependable for the future. Since manual testing can be both time and resource consuming, automated testing has become a modern alternative to increase productivity and to maintain a sustainable development process. The goal of this thesis work has been to investigate the possibilities of implementing a solution for automated UI testing in an already existing development process at the company Precio Systemutveckling AB. The work has been conducted in three steps, starting with a literature study about testing in general, followed by an extensive research into suitable tools and technology for testing that exists today. After this, a deeper look was made at what the recommended solutions for implementing automated testing in a Microsoft-oriented enviroment were. The work was concluded with an actual implementation of automated testing on premise at Precio.

In traditional Software Development Life Cycle, medium and large organizations tend to divide the activities of Operations and Development into separate departments. These groups often have a troublesome relationship because of different incentives during the software delivery process. As a result, conflicts occur between development and operations personnel as they blame each other to be the cause of long lead times and inefficient software delivery processes. The concept of DevOps emerged trying to resolve the problem that arises when separating the work of Development and Operations into organizational silos. The term DevOps is a combination of the abbreviations of Development (Dev) and Operations (Ops). DevOps aim to create a coalition that spans between Development (software developers and quality assurance) and Operation (experts responsible to roll out software to production and managing the infrastructure, e.g. system, network and database administrators and technicians). The idea is to increase the speed of the software delivery process and to quickly solve critical issues, enable organizations to better serve their customers. DevOps means that development teams who previously were solely responsible for the development of their applications now have to manage and govern both development and operational responsibilities. Thus, the adoption of DevOps might introduce new type of challenges and implications for the traditional development teams. Current literature and research about DevOps focus mainly on the challenges that DevOps attempts to overcome. There is a lack of literature on the challenges that practitioners encounter during the adoption of DevOps. As more organizations and companies tend to adopt the concept of DevOps, it increases the need to understand potential challenges and effects of adopting DevOps. Therefore, the aim of this study is to investigate the challenges that development teams encounter during the adoption of DevOps. This research was conducted by an inductive research approach through a single qualitative case study, with the use of semi-structured interviews. In total, four main challenges and fourteen sub-challenges were identified in this study. The four main challenges identified was, lack of awareness, lack of support for DevOps, implementing DevOps technology and adapting organizational processes to DevOps. This study concludes that the adoption of DevOps has a profound impact on the role of a software developer, and that the traditional role of a software developer needs to be evolved. The research provides four recommendations and means to overcome the challenges identified in this research, establishing common ways of working and spreading the knowledge, building commitment and trust by smarter seating, allocate time and resources to transition and trying out with one team and one application. / I traditionell livscykel för mjukvaruutveckling tenderar medelstora och stora organisationer att dela upp verksamheten i drift och utveckling i separata avdelningar. Dessa grupper har ofta en besvärlig relation på grund av olika incitament under mjukvaruleveransprocessen. Som ett resultat uppstår konflikter mellan utvecklings- och driftpersonal eftersom de beskyller varandra för att vara orsaken till långa ledtider och ineffektiva mjukvaruleveransprocesser. Konceptet DevOps uppstod för att försöka lösa det problem som uppstår när man separerar utveckling och drift i organisationella silosar. Termen DevOps är en kombination av förkortningarna för utveckling (Dev) och drift (Ops). DevOps syftar till att skapa en koalition som sträcker sig mellan utveckling (mjukvaruutvecklare och kvalitetssäkring) och drift (system-, nätverks- och databasadministratörer och tekniker). Idén är att öka hastigheten av mjukvaruleveranser och att snabbt lösa kritiska problem för att förbättra organisationens förmåga att betjäna sina kunder. DevOps innebär att utvecklingsgrupper som tidigare enbart ansvarade för utvecklingen av sina applikationer nu även har driftansvar. Således kan antagandet av DevOps introducera nya typer av utmaningar och konsekvenser för de traditionella utvecklingsgrupperna. Aktuell litteratur och forskning kring DevOps fokuserar främst på de utmaningar som DevOps försöker övervinna. Därav finns det brist på litteratur kring de utmaningar som utövare stöter på under antagandet av DevOps. Eftersom fler organisationer och företag tenderar att adoptera begreppet DevOps ökar behovet av att förstå potentiella utmaningar och effekter av att anta DevOps. Därav är syftet med denna studie att undersöka de utmaningar som utvecklingsgrupper bemöter under antagandet av DevOps. Denna forskning utfördes genom en induktiv forskningsinriktning, en kvalitativ fallstudie och datainsamling genom halvstrukturerade intervjuer. Totalt identifierades fyra huvudutmaningar och fjorton sub utmaningar i denna studie. De fyra huvudsakliga utmaningar som identifierades var, brist på medvetenhet, brist på stöd för DevOps, implementering av DevOps-teknik och anpassning av organisationsprocesser till DevOps. Den här studien drar slutsatsen att antagandet av DevOps har en djupgående inverkan på rollen som en mjukvaruutvecklare och att den traditionella rollen som en mjukvaruutvecklare behöver utvecklas. Studien ger fyra rekommendationer och medel för att övervinna de utmaningar som identifierats, etablering av gemensamma sätt att arbeta och sprida kunskapen, bygga upp engagemang och förtroende genom smartare sittplatser, fördela tid och resurser till övergången samt prova med ett lag och en applikation.

The purpose of this study is to continue development of a benchmarking model to help companies assess their technical maturity when it comes to adopting Continuous Integration, Continuous Delivery and DevOps in their organization. The goal of the research is to assess how to improve the quality of qualitative models. Which conclusions can be drawn from comparing companies using benchmark and to assess which actions are the most effective to take to reach higher Continuous Integration, Continuous Delivery and DevOps maturity. The benchmark consisted of a questioner of two hundred statements that were answered for level of agreement with a current situation analysis and an ought-to-be analysis to be able to draw conclusions from the possible discrepancy between these categories. The questioner was answered during an interview study with chosen clients. Conclusions drawn from this study were that a lot can be done to improve the quality of qualitative models for examining Continuous Integration, Continuous Delivery and DevOps maturity. Different actions are necessary but the most important seems to be to ask open ended questions as well ass questions about different aspects of the same problem to promote discussion. It was also showed to be important to peer review the questions in the interview material beforehand to increase quality. The study also showed that it is possible to see trends in Continuous Integration, Continuous Delivery and DevOps maturity when comparing qualitative results for research subjects. The study showed that the most effective method for increasing Continuous Integration, Continuous Delivery and DevOps maturity is to use extensive automated testing suites that covers all testing disciplines. / Syftet med studien är att vidareutveckla ett benchmarkingverktyg för att hjälpa företag att bedöma sin tekniska mognad när det gäller att anta Continuous Integration, Continuous Delivery och DevOps i sin organisation. Målet med forskningen är att bedöma hur man kan förbättra kvaliteten på kvalitativa modeller för att mäta detta, samt vilka slutsatser som kan dras av att jämföra företags resultat som nyttjat studien. Samt att undersöka vilka åtgärder som är effektivast att ta för att nå en högre mognadsgrad inom Continuous Integration, Continuous Delivery och DevOps. Benchmarken bestod av ett frågebatteri av tvåhundra påståenden som besvarades av kunden i hur mycket de instämde till ett påstående. Resultatet samanställdes till en aktuell nulägesanalys och en börlägesanalys, med målet att dra slutsatser i vilka skillnaden som fanns mellan dessa två kategorier. Kunden besvarade frågebatteriet under en intervjustudie med utvalda anställda. Slutsatser som härrör från denna studie var att mycket kan göras för att förbättra kvaliteten på kvalitativa modeller för att undersöka Continuous Integration, Continuous Delivery och DevOps mognadsgrad. Olika åtgärder är möjliga, men det viktigaste förefaller vara att fråga öppna frågor för att främja diskussion samt att ställa frågor om olika aspekter av samma problem. Samt att opponera frågorna internt i intervjuundersökningen innan det utförs hos en kund, för att öka kvaliteten. Studien visade också att det även är möjligt att se trender i Continuous Integration, Continuous Delivery och DevOps mognad hos deltagarna när man jämför de kvalitativa resultaten. Studien visade att de mest effektiva metoderna för att öka Continuous Integration, Continuous Delivery och DevOps mognadsgrad är att använda omfattande automatiserade testsviter för samtliga testmetoder.

The amount of data being collected is increasing astronomically. Hence questions about privacy and data security are becoming more important than ever. A fast-changing culture is also reflected in the demands and requirements placed on software systems. Products and services need to evolve with the demands and feedback from customers to stay relevant on the market. Working methods and technologies have been refined to afford updating software continuously. However, rapidly changing software cause concern for the quality and level of security in the release. This thesis is a comprehensive literature study, reviewing the challenges of ensuring secure practises for continuously evolving software. The problem solved by the thesis is lack of an overall picture of the security concerns during continuous evolution. The findings are summarised in a checklist of areas of concern for security when maintaining and updating systems with continuous practises in cloud environments. This study shows that ensuring security, while delivering continuous releases, is a daunting task. It requires close collaboration between teams handling different aspects of software. This, in turn, entails a widening of competences to include knowledge about the work of other departments. It is concluded that personnel with this wide range of skill will be hard to acquire. / I en tid då mängden data som samlas in om individer ökar i ohindrad takt, blir frågor om integritet och informationssäkerhet viktigare än någonsin. Kraven på snabb utveckling och förändring präglar även metoderna för mjukvaruutveckling. Produkter och tjänster måste konstant anpassas efter kundernas önskemål för att förbli relevant på marknaden. Arbetssätt och teknologier har utvecklats över tid för att möjliggöra mjukvara som uppdateras kontinuerligt. Konstant föränderlig mjukvara leder dock till oro för kvalitén och säkerheten av uppdateringarna. Den här uppsatsen är en litteraturstudie som undersöker utmaningarna att säkerställa säkerhet för mjukvara som uppdateras kontinuerligt. Problemet som löses genom studien är den saknade helhetsbilden av säkerhetsproblem vid kontinuerligt föränderlig mjukvara. Resultatet sammanfattas i en checklista för områden som väcker oro för säkerheten vid arbetssätt som tillåter kontinuerliga uppdateringar i moln-miljöer. Studien visar att leverera säkra lösningar kontinuerligt är en svår uppgift. Det kräver nära samarbete mellan team som sköter olika delar av mjukvaruutveckling. Detta fordrar vida kompetenser som inkluderar förståelse av varandras arbete. Att finna personal med tillräckligt vida kompetenser uppskattas vara problematiskt.