A number of logical code coverage criteria have been used throughout the years in the testing of safety-critical software. Kaminski, et al. proposed Relational Operator Replacement Global (RORG), a method to bring benefits from ROR mutation to Modified Condition / Decision Coverage (MCDC), which is widely used in the avionics industry. However, there is a lack of studies in the industry to support this method. In this thesis, we report on the results of applying RORG to avionic code, augmenting an MCDC adequate test set to satisfy RORG, evaluating its ability to find real faults in industrial software. Conclusions drawn from this thesis are: (1) Faults in relational operators in avionic code are rare, no faults were found in this study. (2) 24% of the relational operators in our study would require additional software requirements to be verified for RORG coverage. (3) 37% of the relational operators in our study were infeasible to test due to program semantics. (4) 84% of the tests added covered enumeration comparisons.

Mobile applications should work regardless of which type of wireless interface is used, and should be able to conceal unstable connections from the user to improve user experience. Therefore, network testing is important when developing mobile applications, but it is a challenge to reproduce network conditions when using real cellular networks since the test engineer has no control over the quality of the cellular network. Existing software tools can restrict bandwidth and add latency to the connection, but these tools do not accurately emulate cellular networks. This thesis proposes a system where it is possible to shape the network traffic for connected devices to mimic the network patterns of a real cellular connection when running on a WiFi connection. The design presented in this thesis is intended for testing mobile applications under diverse 3G connection parameters, such as latency, bandwidth and other characteristics. This thesis was conducted at Spotify, a company that provides a music streaming service which is a frequent user of network data traffic. The 3G emulator was evaluated using the Spotify Android application by measuring the correlation between packet traces from a real 3G connection and the 3G emulator. This correlation was compared to the correlation between packet traces from a real 3G connection and the current network emulator at Spotify. The evaluation shows that the proposed 3G emulator outperforms the current network emulator when performing tests on the Spotify application for Android. By using this emulator, we expect the network testing to become more effective as any 3G condition can be tested with repeatable results.

Coordenação de Aperfeiçoamento de Pessoal de Nível Superior / This work presents an approach of game-based learning using a serious game called Software Testing Team Game. It is a three-dimensional multiplayer game developed in the virtual world OpenSim. This game simulates the software testing division of a software development company. It aims to provide knowledge related to software testing strategies, motivate students, and encourage skills such as communication and problem solving. The main difference between this game and others is its integration with the Learning Management System Moodle, which allows teacher to edit game challenges throughout each of the levels, using the Questionnaire tool, so that the game adapts to different undergraduate computing curricula. The game also enables students to export test cases created by them in the game to the test management tool TestLink, in order that they are given the opportunity to interact with a tool used by professionals and also to directly apply the knowledge acquired. The assessment of the game was performed in a class of Computer Engineering undergraduate students in the Software Engineering Discipline. This assessment showed positive results, considering that learners succeed to achieve learning objectives. Students enjoyed playing the game more than learning in a traditional way, which in this study corresponds to a list of exercises. So, we conclude that the use of the game can collaborate on software testing education. These positive results got in the assessment motivate other experiments. / Este trabalho apresenta uma abordagem de aprendizagem baseada em jogos utilizando o jogo sério denominado Jogo da Equipe de Teste de Software (JETS), que é um jogo em três dimensões, multiplayer e foi desenvolvido no mundo virtual OpenSim. Esse jogo simula o setor de Teste de Software de uma empresa de desenvolvimento de sistemas e visa a proporcionar conhecimentos relacionados com Estratégias de Teste de Software, motivar os estudantes e estimular habilidades como comunicação e resolução de problemas. O principal diferencial do JETS é sua integração com o Ambiente Virtual de Ensino-Aprendizagem (AVEA) Moodle, que permite ao professor editar os desafios das fases por meio da ferramenta questionário, de modo que o jogo se adapte aos diferentes currículos dos cursos de graduação em computação. O JETS também permite a exportação dos casos de teste criados no jogo para a ferramenta de gerenciamento de Teste de Software TestLink, proporcionando aos estudantes o contato com uma ferramenta utilizada por profissionais e a aplicação direta dos conhecimentos adquiridos. A avaliação do JETS foi realizada em uma turma do curso de Engenharia da Computação na disciplina de Engenharia de Software. Essa avaliação teve resultados qualitativos muito positivos, pois, segundo os próprios estudantes, os objetivos de aprendizagem foram atingidos. Além disso, os estudantes relataram, em sua maioria, gostar de jogar o JETS e disseram, ainda, preferir esse tipo de atividade (jogo sério) à tradicional neste trabalho, representada por uma lista de exercícios. Portanto, foi possível perceber que a utilização do JETS pode ser uma alternativa promissora no processo de ensino-aprendizagem do conteúdo de Teste de Software. De modo que os resultados positivos obtidos na avaliação do JETS incentivam a realização de outros experimentos.

The topic of this thesis is the issue of software testing. The thesis places main emphasis on tools to support test management, manual and automated functional testing and last but not least the tools for defect tracking. The aim of this thesis is introduce readers with software testing, especially with tools that can be used to support testing. The aim is offer an overview of the basic commercial and freely distributed tools for test management, manual testing, automated functional testing and defect tracking. Another aim is design criteria that simplify selection of tool. The second aim of this thesis is practical example of the configuration and description of the basic work with tools IBM Rational - ClearQuest, ClearCase, Manual Tester and Functional Tester. The aims of this thesis were achieved by studying available sources and by own practical experience with the tools to support testing. The contribution of this thesis lies in the characteristics of the selected tools to support testing and especially in design of criteria by which tools can be selected. The last part of thesis provides practical instruction how to configure and work with the tools to support testing of IBM Rational. The thesis is conceived in three main parts. The first part attends to basic terms, which can be found in the area of testing, and to overview of types of tests. There are also described two most famous life-cycle models and methodology RUP. The second part attends to overview of tools to support testing, attention is given to areas: test management, manual testing, defect tracking, and automated functional testing. For each category of tools has been defined criteria according to which tools can be selected. The last part attends to practical example of setting and basic work with the selected tools to support testing of IBM Rational.

Context: Exploratory testing (ET) is an approach to test software with a strong focus on personal skills and freedom of the tester. ET emphasises the simultaneous design and execution of tests with minimal test documentation. Test practitioners often claim that their choice to use ET as an important alternative to scripted testing is based on several benefits ET exhibits over the scripted testing. However, these claims lack empirical evidence as there is little research done in this area. Moreover, ET is usually considered an ad-hoc way of doing testing as everyone does it differently. There have been some attempts in past to provide structure to ET. Session based test management (SBTM) is an approach that attempts to provide some structure to ET and gives some basic guidelines to structuring the test sessions. However, these guidelines are still very abstract and are very open to individuals' interpretation. Objective: The main objective of this doctoral thesis is to support practitioners in their decisions about choosing exploratory versus scripted testing. Furthermore, it is also aimed to investigate the empirical evidence in support of ET and find ways to structure ET and classify different levels of exploration that drive the choices made by exploratory testers. Another objective of this thesis is to provide a decision support system to select levels of exploration in overall test process. Method: The findings presented in this thesis are obtained through a controlled experiment with participants from industry and academia, exploratory surveys, interviews and focus groups conducted at different companies including Ericsson AB, Sony Mobile Communications, Axis Communications AB and Softhouse Consulting Baltic AB. Results: Using the exploratory survey, we found three test techniques to be most relevant in context of testing software systems and in particular heterogeneous systems. The most frequently used technique mentioned by the practitioners is ET which is not a much researched topic. We also found many interesting claims about ET in grey literature produced by practitioners in the form of informal presentations and blogs but these claims lacked any empirical evidence. Therefore, a controlled experiment was conducted with students and industry practitioners to compare ET with scripted testing. The experiment results show that ET detects significantly more critical defects compared to scripted testing and is more time efficient. However, ET has its own limitations and there is not a single way to use it for testing. In order to provide structure to ET, we conducted a study where we propose checklists to support test charter design in ET. Furthermore, two more industrial focus group studies at four companies were conducted that resulted in a taxonomy of exploration levels in ET and a decision support method for selecting exploration levels in ET. Lastly, we investigated different problems that researchers face when conducting surveys in software engineering and have presented mitigation strategies for these problems. Conclusion: The taxonomy for levels of exploration in ET, proposed in this thesis, provided test practitioners at the companies a better understanding of the underlying concepts of ET and a way to structure their test charters. A number of influence factors elicited as part of this thesis also help them prioritise which level of exploration suits more to their testing in the context of their products. Furthermore, the decision support method provided the practitioners to reconsider their current test focus to test their products in a more effective way.

This thesis deals with software testing and its possible automation and compares the pros and cons of automated testing with manual testing. In the automated approach is introduced tool for automated testing HP QuickTest Professional (HP QTP). The main objective is to analyze the advantages of using this tool in conditions of a real project, based on predefined criteria. In the theoretical section I introduce basic terms within the field of software testing and identify testing in the software development process. A comparison of manual and automated approach to software testing and introduction of tool HP QTP for automated tests follows. Target of practical part is to analyze the advantages of automation by using HP QTP on a real project according to defined criteria and recommend for what conditions the automation in this case worthwhile. In the practical part is also created manual which describing the basic functionality of HP QTP, which is demonstrated on examples. Manual, which is also the output of this work, has to help testers to quickly understand the work with this tool and therefore better plan process of tests automation.

The main objective of this diploma thesis is to design an efficient methodology for testing and evaluating user experience of smaller web applications. The proposed methodology is based on understanding the issues of user experience testing of small projects, exploring the principles of current methodologies and analysis methods for testing and evaluating user experience. Based on the analysis, appropriate methods are selected and the methodology is assembled. Part of this thesis is verifying the methodology on a real project in practice in order to reveal its potential shortcomings. This thesis is composed of two parts. The first, theoretical part includes analysis of user experience issues and its testing, description of the methodology principles and the analysis of methods themselves. The second part includes the designed methodology and its verification in practice. Proposed methodology should be beneficial for small development teams or companies that have a need for testing the user experience, but do not have enough resources to develop their own methodology or purchase the methodology developed by another company. In practice, I was not able to find freely available methodology for testing the user experience, which would also be designed specifically for small projects.

The subject of this thesis is testing and quality assurance during software development. The theoretical part explains the meaning of software quality and then describes the metrics used to evaluate software quality. The following part explains the differences between software quality assurance in agile and traditional software development methodologies, including criteria on how to compare the methodologies. Throughout the thesis, there are briefly summarized basic concepts which then include the differences between stat-ic/dynamic testing and manual/automatic testing, as well as a role of quality assurance en-gineer in software development. The practical section extends to an existing software development methodology for small software projects (MMSP) in its testing area. New testing activities, artifacts, and roles are introduced in order to align with real requirements for software testing. They will also function in the methodology when used in the testing area for development of more robust applications in bigger teams. Test management tools and test automation tools are described and followed with recommendations for methodol-ogy usage for only a selected few.

O ensino das disciplinas de fundamentos de programação e teste de software não é trivial. Várias iniciativas têm sido investigadas a fim de amenizar os problemas associados a essa atividade. Uma dessas iniciativas é o ensino conjunto de conceitos básicos de programação e de teste de software em disciplinas introdutórias dos cursos de Ciências da Computação. Este trabalho insere-se nessa perspectiva, tendo como principal objetivo fornecer subsídios para o ensino integrado de fundamentos de programação OO e teste de software. Para atingir esse objetivo, foi desenvolvido um módulo educacional integrado de fundamentos de programação OO e teste de software, utilizando uma abordagem integrada para modelagem de conteúdos educacionais. Ênfase é dada na modelagem dos aspectos conceituais, instrucionais e didáticos dos domínios de conhecimento envolvidos. Além disso, também foi desenvolvido um ambiente, denominado ProgTest, baseado na Web e na concepção de software livre, para submissão e avaliação de trabalhos práticos dos alunos. O ambiente foi integrado com ferramentas de teste de modo que a avaliação dos trabalhos seja realizada com base em atividades de teste. Os programas e os resultados são avaliados com base em padrões de consistência, eficácia e eficiência uniformes. Os professores também podem ter uma redução na sua carga de trabalho, uma vez que o sistema avalia automaticamente tanto os casos de teste quanto o código do programa. / The teaching of fundamentals of programming and software testing is not trivial. Many attempts have been investigated in order to reduce the problems associated to this activity. One of these initiatives is the teaching of basic concepts and techniques of programming in conjunction with software testing in introductory courses of the Computer Science. In this sense, this work aims at providing mechanisms to the integrated teaching of fundamentals of OO programming and software testing. To achieve this goal, we developed an integrated educational module of fundamentals of OO programming and software testing, using an integrated approach to model the educational contents. We focus on conceptual, instructional and didactic modeling of the knowledge domains. Furthermore, an open source Web-based environment, named ProgTest, was also developed for the submission and evaluation of practical assignments to the students. The environment was integrated with testing tools, so the evaluation of the assignments is performed based on testing activities. The assignments and the results are evaluated based on the uniform standards in terms of consistence, efficacy and efficiency. The professors can have their workload reduced with this environment, once the system automatically evaluates both the code of the assignment as well as its test cases.

High-level language virtual machines (HLL VMs) have been playing a key role as a mechanism for implementing programming languages. Languages that run on these execution environments have many advantages over languages that are compiled to native code. These advantages have led HLL VMs to gain broad acceptance in both academy and industry. However, much of the research in this area has been devoted to boosting the performance of these execution environments. Few eorts have attempted to introduce features that automate or facilitate some software engineering activities, including software testing. This research argues that HLL VMs provide a reasonable basis for building an integrated software testing environment. To this end, two software testing features that build on the characteristics of a Java virtual machine (JVM) were devised. The purpose of the rst feature is to automate weak mutation. Augmented with mutation support, the chosen JVM achieved speedups of as much as 95% in comparison to a strong mutation tool. To support the testing of concurrent programs, the second feature is concerned with enabling the deterministic re-execution of Java programs and exploration of new scheduling sequences / Máquinas virtuais de linguagens de programação têm desempenhado um papel importante como mecanismo para a implementação de linguagens de programação. Linguagens voltadas para esses ambientes de execução possuem várias vantagens em relação às linguagens compiladas. Essas vantagens fizeram com que tais ambientes de execução se tornassem amplamente utilizados pela indústria e academia. Entretanto, a maioria dos estudos nessa area têm se dedicado a aprimorar o desempenho desses ambientes de execução e poucos têm enfocado o desenvolvimento de funcionalidades que automatizem ou facilitem a condução de atividades de engenharia de software, incluindo atividades de teste de software. Este trabalho apresenta indícios de que máquinas virtuais de linguagens de programação podem apoiar a criação de ambientes de teste de software integrado. Para tal, duas funcionalidades que tiram proveito das características de uma máquina virtual Java foram desenvolvidas. O propósito da primeira funcionalidade e automatizar a condução de atividades de mutação fraca. Após a implementação de tal funcionalidade na máquina virtual Java selecionada, observou-se um desempenho até 95% melhor em relação a uma ferramenta de mutação forte. Afim de apoiar o teste de programas concorrentes, a segunda funcionalidade permite reexecutá-los de forma determinística além de automatizar a exploração de que novas sequências de escalonamento