This master's thesis describes the possibility of using a continues integration and testing in real company Logio. The result of thesis is implementation of continuous integration tools Jenkins and specific testing for the company Logio. The theoretical part deals with basic principles, techniques and tools defining continuous integration. Subsequently are describing existing tools designed for PHP programming language. The thesis also analyzes various types and possibilities of application testing. Also explains the problems in product development in company Logio, which are consequence of the need for introducing continues integration techniques. The practical part of thesis describes the implementation of a test framework PwTester and its use in continuous integration. In conclusion, the results are evaluated and outlined the possibility of extension.

With today’s focus on Continuous Integration, test cases are used to ensure the software’s reliability when integrating and developing code. Test cases that behave in an undeterministic manner are known as flaky tests, which threatens the software’s reliability. Because of flaky test’s undeterministic nature, they can be troublesome to detect and correct. This is causing companies to spend great amount of resources on flaky tests since they can reduce the quality of their products and services. The aim of this thesis was to develop a usable tool that can automatically detect flakiness in the Randomness category. This was done by initially locating and rerunning flaky tests found in public Git repositories. By scanning the resulting pytest logs from the tests that manifested flaky behaviour, noting indicators of how flakiness manifests in the Randomness category. From these findings we determined tracing to be a viable option of detecting Randomness as a cause of flakiness. The findings were implemented into our proposed tool FlakyReporter, which reruns flaky tests to determine if they pertain to the Randomness category. Our FlakyReporter tool was found to accurately categorise flaky tests into the Randomness category when tested against 25 different flaky tests. This indicates the viability of utilizing tracing as a method of categorizing flakiness.

Advancements within Large Language Models(LLMs) have opened a world of opportunities within the software development domain. This thesis, through an controlled experiment, aims to investigate how LLMs can be utilized within software testing, more specifically unit testing. The controlled experiment was performed using a Python script interfacing with the gpt-3.5-turbo model, to automatically generate unit tests for React Native components written in TypeScript. The pipeline described, performs the calls to the OpenAI Application Programming Interface(API) iterative. To evaluate and retrieve the metric code coverage, the unit tests were executed with Jest. Additionally, manual execution of failing tests, both compilable and non-compilable tests were executed and the different kind of errors with their frequency were documented. The experiment shows that LLMs can be used to generate comprehensive and accurate unit tests, with high potential of future improvements. While the amount of generated tests that compiled were low, their nature was often good, failing because of easy correctable syntax errors, faulty imports or missing dependencies. The errors found, were at large part due to project configurations while others would probably be less frequent through more extensive prompt-engineering or by the use of an newer model. The experiment also shows that the temperature affected the outcome and that the type of errors were different between compiling and non-compiling tests. A lower temperature parameter to the OpenAI API generally achieved better results, whilst a higher temperature showed greater coverage at compiled failing tests. This thesis also shows that future opportunities and improvements are widely available. Through better project optimization, newer models and better prompting, a better result is to be expected. The script could with further development be turned into a working product, making software testing faster and more efficient, saving both time and money while simultaneously improving the test case quality.

Made available in DSpace on 2014-12-17T15:48:00Z (GMT). No. of bitstreams: 1 ErnestoCBM_DISSERT.pdf: 1152535 bytes, checksum: a61c509f155d27fa9ab04bc69c4607e8 (MD5) Previous issue date: 2012-02-10 / Coordena??o de Aperfei?oamento de Pessoal de N?vel Superior / Formal methods and software testing are tools to obtain and control software quality. When used together, they provide mechanisms for software specification, verification and error detection. Even though formal methods allow software to be mathematically verified, they are not enough to assure that a system is free of faults, thus, software testing techniques are necessary to complement the process of verification and validation of a system. Model Based Testing techniques allow tests to be generated from other software artifacts such as specifications and abstract models. Using formal specifications as basis for test creation, we can generate better quality tests, because these specifications are usually precise and free of ambiguity. Fernanda Souza (2009) proposed a method to define test cases from B Method specifications. This method used information from the machine s invariant and the operation s precondition to define positive and negative test cases for an operation, using equivalent class partitioning and boundary value analysis based techniques. However, the method proposed in 2009 was not automated and had conceptual deficiencies like, for instance, it did not fit in a well defined coverage criteria classification. We started our work with a case study that applied the method in an example of B specification from the industry. Based in this case study we ve obtained subsidies to improve it. In our work we evolved the proposed method, rewriting it and adding characteristics to make it compatible with a test classification used by the community. We also improved the method to support specifications structured in different components, to use information from the operation s behavior on the test case generation process and to use new coverage criterias. Besides, we have implemented a tool to automate the method and we have submitted it to more complex case studies / M?todos formais e testes s?o ferramentas para obten??o e controle de qualidade de software. Quando utilizadas em conjunto, elas prov?em mecanismos para especifica??o, verifica??o e detec??o de falhas de um software. Apesar de permitir que sistemas sejam matematicamente verificados, m?todos formais n?o s?o suficientes pra garantir que um sistema esteja livre de defeitos, logo, t?cnicas de teste de software s?o necess?rias para completar o processo de verifica??o e valida??o de um sistema. T?cnicas de Testes Baseados em Modelos permitem que testes sejam gerados a partir de outros artefatos de software como especifica??es e modelos abstratos. Ao utilizarmos especifica??es formais como base para a cria??o de testes, podemos gerar testes de melhor qualidade pois estas especifica??es costumam ser precisas e livres de ambiguidade. Fernanda Souza (2009) prop?s um m?todo para definir casos de teste a partir de especifica??es do M?todo B. Este m?todo utilizava informa??es do invariante de uma m?quina e das pr?-condi??es de uma opera??o para definir casos de teste positivos e negativos para tal opera??o, atrav?s de t?cnicas baseadas em particionamento em classes de equival?ncia e an?lise de valor limite. No entanto, a proposta de 2009 n?o inclu?a automa??o e possu?a algumas defici?ncias conceituais como, por exemplo, n?o se encaixar exatamente em uma classifica??o de crit?rios de cobertura bem definida. Iniciamos nosso trabalho com um estudo de caso que aplicou o m?todo a um exemplo de especifica??o B proveniente da ind?stria. A partir deste estudo obtivemos subs?dios para o aperfei?o?-lo. Em nosso trabalho aperfei?oamos o m?todo proposto, reescrevendo e adicionando caracter?sticas para torn?-lo compat?vel com uma classifica??o de testes utilizada pela comunidade. O m?todo tamb?m foi melhorado para suportar especifica??es estruturadas em v?rios componentes, utilizar informa??es sobre o comportamento da opera??o durante a cria??o de casos de teste e utilizar novos crit?rios de cobertura. Al?m disso, implementamos uma ferramenta para automatiz?-lo e o submetemos a estudos de caso mais complexos

Test driven development [TDD] is an iterative technique where unit   tests are written before production code. General idea is that quality,   especially code design, improves with usage of TDD. Previous studies have had   difficulties proving these benefits. Qualitative studies about TDD are rare   and a deeper understanding how developers’ view relates to usage of TDD has   been sought after. How experience affects usage and perception about TDD is   of particular interest, and the study’s basis. Semi-structured interviews were conducted with three experienced and   three inexperienced developers in a qualitative survey study. Obvious   differences could be determined by letting those groups represent the   experience range that exists. A deductive thematic analysis was performed with   the help of a theoretical framework. Results show that   workplace’s culture greatly affects how TDD is used. TDD is not commonly used   and low adherence to protocol is also made visible. Differences in basic view   about TDD is evident as well – creation of the code’s design is starting   point for experienced developers, inexperienced use TDD as a tool in creation   of unit tests and to control system correctness. / Testdriven utveckling [TDD]   är en iterativ teknik där enhetstest skrivs före produktionskod. Tanken är   att kvalité och framförallt design av kod förbättras vid användning av TDD.   Tidigare studier visar svårigheter att bevisa dessa vinster. Kvalitativa   studier är ovanliga i forskning om TDD och en djupare förståelse hur TDD   används från utvecklarens synvinkel har eftersökts. Hur erfarenhet påverkar   användning och uppfattning om TDD är särskilt intressant, och fungerar som   studiens grund. Semistrukturerade   intervjuer utfördes med tre erfarna och tre oerfarna utvecklare i en   kvalitativ tvärsnittsstudie. Genom att låta grupperna representera det   erfarenhetsspann som existerar kunde tydliga skillnader synliggöras. En   deduktiv tematisk analys genomfördes med hjälp av ett teoretiskt ramverk. Studien visar framförallt   att arbetsplatsens kultur påverkar stort hur TDD används. TDD används inte i   särskilt stor utsträckning och låg regelmässighet blev även synliggjort.   Förutom detta visades skillnader i grundläggande syn över TDD – skapande av   kodens design är utgångspunkt för erfarna utvecklare, oerfarna använder TDD   som verktyg att skapa tester och kontrollera systemets korrekthet.

This thesis deals with the development of a client-server application for Red Hat company. Client participates in testing process of JBoss platform and gets user-defined performance data which sends during testing to the server application by platform independent communication. The server application allows to analyze the received data which can be also compared by several perspectives. These services for data analysis and comparison are accessible for server users using the web client. The server application supports the import of the performance data stored in the XML file and also their export for qVue portal. The client part of the application is implemented in Java and the server application is based on Java EE platform.

All developers understand the benefits of testing their code to ensure its functionality. Today’s market is moving further towards design principles where testing is a central or driving force during development. This puts a certain pressure on academia to supply these skills to their students.Recently the course II1302 Projects and project methods at the Royal Institute of Technology in Kista made a concerted effort to introduce the students of the course to these modern concepts. This thesis investigates how areas of testing can effectively be introduced to the students in the course, utilizing a tailored example that takes the area of testing into particular consideration and how to automate it via DevOps-tools provided by a cloud-based service. Further, it also makes an attempt to provide additional material to be used for teaching testing in conjunction with the example provided.The case study covers the development of an example application, meant to mirror a typical student project. It also covers how this was used for teaching the students about the testing areas considered. The covered testing areas include unit testing, integration testing and UI testing. With these given testing areas, the application and an associated learning module was developed for each area in question. Relevant standards, strategies and approaches was also identified for each of these areas.The thesis also presents important properties to take into consideration when developing similar examples in the future, based on the experiences obtained during the study. These include needs such as understandable by inexperienced students, applicability outside the course, adherence to established standards, tools that are simple to use and an architectural structure that allows for testing.Some improvements are also recommended: the students would benefit from learning software testing from an early stage of their studies. The content of the learning modules should also be brought to the students earlier in the course, so it can be applied in their projects at an early stage as well.Further research is also recommended to evaluate the suitability of using other cloud-based environments instead, and to evaluate the applicability of the learning modules for students of varying disabilities. / Alla utvecklare förstår fördelarna med att testa kod för att garantera dess funktionalitet. Dagens industri går i en riktning där testning spelar en central del av design under utveckling av mjukvara. Denna tendens lägger en viss press på högskolan att lära ut dessa erfarenheter till dess studenter.På senare tid har kursen II1302 Projekt och projektmetoder på Kungliga Tekniska Högskolan i Kista tagit en stor ansats för att introducera sina studenter inför dessa moderna koncept. Denna uppsats undersöker hur testningsområdet effektivt kan introduceras till studenterna inom denna kurs, genom att utnyttja ett egengjort exempel som har området i fokus, samt att automatisera detta via DevOps-verktyg tillhandahållna av molnbaserade tjänster. Dessutom görs även en ansats för att tilldela ytterligare material som kan användas för att lära ut testning av mjukvara i samband med det givna exemplet.Fallstudien omfattar utvecklingen av en exempelapplikation, som var avsedd att likna ett typiskt studentprojekt. Den täcker även hur denna användes för att lära ut de betraktade testningsområdena till studenterna. De täckta områdena av testning inkluderar enhetstestning, integrationstestning och testning av användargränssnitt. Med dessa givna testningsområden utvecklades både applikationen och dess associerade lärmoduler för vardera testningområde i fråga. Relevanta standarder, strategier och metoder var också identifierade för vardera av dessa områden.Denna uppsats presenterar även ett antal viktiga egenskaper att hålla i åtanke vid utveckling av liknande exempel i framtiden, baserat på erfarenheterna från studien. Detta inkluderar behov som tillgänglighet för mindre erfarna studenter, applicerbarhet utanför själva kursen, tillämpning av etablerade standarder, utnyttjande av lättanvända verktyg och en arkitektur som tillåter testning.Några förbättringar föreslås även: studenterna skulle gynnas av att lära sig om mjukvarutestning i ett tidigt skede av sina studier. Innehållet i lärmodulerna bör även presenteras för studenterna tidigare i kursen för att kunna appliceras i deras projekt.Ytterligare forskning rekommenderas även för att utvärdera andra lämpliga molnbaserade miljöer, samt för att utvärdera tillämpbarheten av lärmodulerna hos studenter med inlärningssvårigheter.

In recent years, artificial intelligence (AI) has become increasingly popular. An area where AI technology is used and has received much attention during the past year is chatbots. They can simulate an understanding of human language and form text responses to questions asked. Apart from generating text responses, they can also generate programming code, making them useful for tasks such as testing. Although testing is considered a crucial part of software development, many find it tedious and time-consuming. There are currently limited AI-powered tools for generating unit tests in general and even fewer for the programming language Java. The thesis tackles the problem of the lack of tools for generating unit tests in Java that explore the capabilities of AI, and a research question is introduced thereafter. The purpose of this thesis is to address the issue by creating a prototype for generating unit tests in Java based on the AI model, GPT-3.5-Turbo. The goal is to provide a basis for other professionals to create tools for generating unit tests, which was done by experimenting with different prompts and values of a randomness parameter and then suggesting the prototype JUTAI. A quantitative research method with an experimental and comparative approach was used to evaluate the results. A comparison model with three criteria was brought forward to evaluate the results. The findings reveal that JUTAI outperformed the general-purpose AI tool, ChatGPT, across all three criteria and indicate that the goal of this thesis is achieved and the research question answered. / Intresset för artificiell intelligens (AI) har ökat de senaste åren. Ett område där AI- teknologi används och som har fått mycket uppmärksamhet under det senaste året är chattbottar. De kan simulera en förståelse för mänskligt språk och svara på frågor i textformat. Utöver det kan de även generera programkod. Tack vare förmågan att generera kod kan de användas för testning. Även om testning anses vara en viktig del av mjukvaruutveckling, tycker många att det är tråkigt och tidskrävande. För närvarande finns det ett begränsat antal verktyg som kan generera enhetstester, och det finns ännu färre verktyg som kan göra detta i Java. Detta examensarbete tog sig an problemet med bristen på AI-verktyg för enhetstestning i Java genom att besvara på forskningsfrågan som ställdes. Syftet med examensarbetet är att föreslå en lösning på problemet genom att utveckla en prototyp som använder sig av AI- modellen GPT-3.5-Turbo för att generera enhetstester i Java. Målet är att ge en grund för andra yrkesverksamma att skapa verktyg för att generera enhetstester, vilket gjordes genom att experimentera med olika instruktionstrukturer och värden för en slumpmässighetsparameter, och sedan föreslå protypen JUTAI. En kvantitativ forskningsmetod tillsammans med en experimentell och jämförande ansats användes för att utvärdera resultaten. En jämförelsemodell med tre kriterier togs fram för att utvärdera resultaten. Resultaten visar att JUTAI presterade bättre än AI-verktyget ChatGPT i de tre kriterierna och indikerar att målet med detta examensarbete uppnåddes och forskningsfrågan besvarades.

The main goal of this thesis is to describe the implementation of continuous integration into the k-Wave project. The thesis focuses primarily on the version written in the C/C++ language with the usage of the OpenMP library which typically runs on supercomputers. Accordingly, many of popular workflows and approaches ought to be adapted, a few more created. The outcome of the thesis is a complete solution with real and practical usage. The author provides design, tools selection, runtime environment administration and configuration for each one of the used services. Software implementation of the basic framework is used in order to utilize running tests on the supercomputers. Furthermore, the implementation of chosen types of regression and unit tests are performed. Realisation is based on Gitlab and Jenkis services that are running on separated Docker containers.

När en grupp utvecklare jobbar med samma kodbas kan konflikter uppstå med avseende på implementationen av moduler eller delsystem som varje utvecklare individuellt jobbar på. Dessa konflikter måste snabbt lösas för att projektet ska fortskrida och inte stagnera. Utvecklare som sällan kommunicerar framför ofta okompatibla moduler eller delsystem som kan vara svåra eller omöjliga att integrera i kodbasen, detta leder ofta till s.k. “integration hell” där det kan ta väldigt lång tid att anpassa ny kod till en befintlig kodbas.En strategi som man kan ta till är “continuous integration”, ett arbetssätt som erbjuder en rad fördelar när man jobbar i grupp på en gemensam kodbas. Continuous integration är möjligt att tillämpa utan verktyg eftersom detta är ett arbetssätt. Däremot kan processen stödjas av ett s.k. “CI-system” som är något av en teknisk implementation eller påtagligt införlivande och stöd för arbetsmetoden “continuous integration”.Denna rapport syftar till att ge en inblick i vad ett CI-system är och vad den principiellt består av. Vi undersöker vad ett CI-system absolut måste bestå av genom en litteraturundersökning och en marknadsundersökning. Vi ställer upp dessa beståndsdelar som “funktionella” och “icke-funktionella” krav för ett typiskt CI-system. Vi kan på så vis kvantifiera och kategorisera olika komponenter och funktionaliteter som bör innefattas i ett typiskt CI-system. I denna rapport finns även ett bihang som visar hur man kommer igång med att bygga en egen CI-server mha. CI-systemmjukvaran “TeamCity”.Slutsatsen av vår rapport är att CI-system är ett viktigt redskap som kan underlätta mjukvaruutveckling. Med hjälp av CI-system kan man stödja utvecklingsprocessen genom att bl.a. förhindra integrationsproblem, automatisera vissa delar av arbetsprocessen (kompilering av källkod, testning av mjukvara, notifikation om stabilitet av kodbas och distribution av färdig mjukvara) samt snabbt hitta och lösa integrationsfel. / When a group of developers work on the same code base, conflicts may arise regarding the implementation of modules or subsystems that developers individually work on. These conflicts have to be resolved quickly in order for the project to advance at a steady pace. Developers who do not communicate changes or other necessary deviations may find themselves in a situation where new or modified modules or subsystems are impossible or very difficult to integrate into the mainline code-base. This often leads to so called “integration hell” where it could take huge amounts of time to adapt new code into the current state of the code-base. One strategy, which can be deployed to counteract this trend is called “continuous integration”. This practice offers a wide range of advantages when a group of developers collaborates on writing clean and stable code. Continuous integration can be put into practice without the use of any tools as it is a “way to do things” rather than an actual tool. With that said, it is possible to support the practice with a tangible tool called a CI-system.This study aims to give insight into the makings of a CI-system and what it fundamentally consists of and has to be able to do. A study of contemporary research reports regarding the subject and a survey was performed in order to substantiate claims and conclusions. Core characteristics of CI-systems are grouped into “functional requirements” and “non-functional requirements (quality attributes)”. By doing this, it is possible to quantify and categorize various core components and functionalities of a typical CI-system. This study also contains an attachment which provides instructions of how to get started with implementing your own CI-server using the CI-system software ”TeamCity”. The conclusion of this study is that a CI-system is an important tool that enables a more efficient software development process. By making use of CI-systems developers can refine the development process by preventing integration problems, automating some parts of the work process (build, test, feedback, deployment) and quickly finding and solving integration issues.