Software Testing plays a key role in Software Development. There are two approaches to software testing i.e. Manual Testing and Automated Testing which are used to detect the faults. There are numbers of automated software testing tools with different purposes but it is always a problem to select a software testing tool according to the needs. In this research, the author compares two software testing tools i.e. Apache JMeter and HP LoadRunner to determine their usability and efficiency. To compare the tools, different parameters were selected which guide the tool evaluation process. To complete the objective of the research, a scenario-based survey is conducted and two different web applications were tested. From this research, it is found that Apache JMeter got an edge over HP Loadrunner in different aspects which include installation, interface and learning.

Most software today is concurrent and are used in everything from cell- phones, washing machines, cars to aircraft control systems. The reliability of the concurrent software may be more or less critical, depending on which a.o. domain it is functioning in. Irrespective of domain, the concurrent software must be sufficiently reliable. It is therefore interesting to study how adaptable test methods for sequential software are to test concurrent software. Novel test methods for concurrent software can be developed by adapting test methods for sequential software. In this dissertation, adaptability factors have been identified by conducting a literature survey over state-of-the-art test methods. Directions taken in the research of concurrent software testing is described by the survey. The survey also demonstrates differences and similarities between test methods. Three research contributions has been achieved by this dissertation. First, this dissertation presents a survey over state-of-the-art-test methods. The second contribution is the identified adaptability factors that should be added to a test method for sequential software, that will be adapted to test concurrent software. Finally, the third contribution to the field of concurrent software testing is the identified future work in areas where test methods for concurrent software has not been researched much or at all.

O teste de software é considerado uma importante atividade na garantia da qualidade de produtos de software. No entanto, há uma carência de profissionais qualificados nessa área. Isso pode ser ocasionado pela dificuldade de ensinar teste de software por meio de abordagens que utilizem apenas aulas teóricas e ferramentas de teste tradicionais. Além disso, há uma desmotivação decorrente do ambiente de trabalho e das estratégias de alocação e responsabilidade desses profissionais nas equipes de desenvolvimento e teste. Para amenizar esses problemas, têm sido utilizado outras abordagens de apoio ao ensino de teste de software, tais como: jogos educacionais, ensino de teste com programação, módulos educacionais, entre outras. O objetivo deste projeto de mestrado foi desenvolver um jogo educacional, denominado Testing Game, para auxiliar o ensino de teste de software, especificamente: teste funcional, teste estrutural e teste de mutação. Para auxiliar o desenvolvimento do Testing Game, foi realizado um mapeamento sistemático para selecionar um motor de jogos. Na primeira versão do jogo foi utilizado o motor de jogos Cocos2D e na segunda versão foi utilizado o Construct 2. Para avaliar a eficiência do Testing Game, realizou-se um estudo de viabilidade com o intuito de avaliar a qualidade com relação à motivação, experiência do usuário e aprendizagem sob o ponto de vista dos estudantes. Além disso, avaliou-se a usabilidade do Testing Game. Aproximadamente 85,64% das pessoas que participaram do estudo avaliaram a qualidade do jogo de forma positiva com relação à motivação, experiência do usuário e aprendizagem sob o ponto de vista dos estudantes. Quanto à usabilidade do jogo, foram identificados poucos problemas, o que possibilita a liberação do jogo. Por meio deste trabalho, percebeu-se que o jogo Testing Game poderia ser utilizado como um recurso complementar de apoio ao ensino de teste de software, e sua efetividade ser avaliada. / Software testing is a relevant activity to provide evidences of qualifty of software products. However, there is a lack of qualified professionals in this area. This can be caused due to difficulty in teaching software testing through approaches that use only theoretical classes traditional tools. In addition, there is a lack of motivation due to the work environment and the strategies of allocation and responsibility of these professionals in development and testing teams. To mitigate these problems, approaches have been used to support software testing education, such as: educational games, integrated teaching of software testing with programming, educational modules, among others. The objective of this masters thesis was to develop an educational game named Testing Game, addressing the following topics: functional testing, structural testing and mutation testing. To support the development of the Testing Game, we performed a systematic mapping aiming at selecting a game engine. In the first game version, we used Cocos2D and in the second one we used Construct 2. To evaluate the efficiency of the game, we conducted a feasibility study to evaluate the quality regarding motivation, user experience and learning from the point of view of the students. Moreover, we also evaluate the usability of the Testing Game. Approximately 85.64% of people who participated in the study assessed the quality of the game in a positive perspective regarding motivation, user experience and learning from the point of view of the students. Regarding the usability of the game, students identified minor problems were identified, which allows the release of the game. Through this work, we realize that the game Testing Game can be used as a complementary resource to support software testing education, and its effectiveness be evaluated.

In software testing process a large amount of information is required and generated. This information can be stored as knowledge that needs to be managed and maintained using principles of knowledge management. Ontologies can act as a bridge by representing this testing knowledge in an accessible and understandable way.

Context: Software testing is the process of assessing quality of a software product to determine whether it matches with the existing requirements of the customer or not. Software testing is one of the “Verification and Validation,” or V&V, software practices. The two basic techniques of software testing are Black-box testing and White box testing. Black-box testing focuses solely on the outputs generated in response to the inputs supplied neglecting the internal components of the software. Whereas, White-box testing focuses on the internal mechanism of the software of any application. To explore the feasibility of black-box and white-box testing under a given set of conditions, a proper test automation framework needs to be deployed. Automation is deployed in order to reduce the manual effort and to perform testing continuously, thereby increasing the quality of the product. Objectives: In this research, cloud hosted application is automated using TestComplete tool. The objective of this thesis is to verify the functionality of cloud application such as test appliance library through automation and to measure the impact of the automation on release cycles of the organisation. Methods: Here automation is implemented using scrum methodology which is an agile development software process. Using scrum methodology, the product with working software can be delivered to the customers incrementally and empirically with updating functionalities in it. Test appliance library functionality is verified deploying testing device thereby keeping track of automatic software downloads into the testing device and licenses updating in the testing device. Results: Automation of test appliance functionality of cloud hosted application is made using TestComplete tool and impact of automation on release cycles is found reduced. Through automation of cloud hosted application, nearly 24% of reduction in level of release cycles can be observed thereby reducing the manual effort and increasing the quality of delivery. Conclusion: Automation of a cloud hosted application provides no manual effort thereby utilisation of time can be made effectively and application can be tested continuously increasing the efficiency and / AUTOMATION OF A CLOUD HOSTED APPLICATION

published_or_final_version / abstract / Computer Science / Master / Master of Philosophy

published_or_final_version / Computer Science / Doctoral / Doctor of Philosophy

Engineers in diverse fields are able to model their design and experiment with that design to determine how it responds to the environment and how it satisfies the requirements. Design tools for software engineering have become standardized and matured to allow for formal definition of software design. This paper tests the current state of design documentation to determine the quality of design testing available at the early stage of software design. / text

Real-time Embedded Systems (RTESs) have an increasing role in controlling society infrastructures that we use on a day-to-day basis. RTES behaviour is not based solely on the interactions it might have with its surrounding environment, but also on the timing requirements it induces. As a result, ensuring that an RTES behaves correctly is non-trivial, especially after adding time as a new dimension to the complexity of the testing process. This research addresses the problem of testing RTESs from Timed Automata (TA) specification by the following. First, a new Priority-based Approach (PA) for testing RTES modelled formally as UPPAAL timed automata (TA variant) is introduced. Test cases generated according to a proposed timed adequacy criterion (clock region coverage) are divided into three sets of priorities, namely boundary, out-boundary and in-boundary. The selection of which set is most appropriate for a System Under Test (SUT) can be decided by the tester according to the system type, time specified for the testing process and its budget. Second, PA is validated in comparison with four well-known timed testing approaches based on TA using Specification Mutation Analysis (SMA). To enable the validation, a set of timed and functional mutation operators based on TA is introduced. Three case studies are used to run SMA. The effectiveness of timed testing approaches are determined and contrasted according to the mutation score which shows that our PA achieves high mutation adequacy score compared with others. Third, to enhance the applicability of PA, a new testing tool (GeTeX) that deploys PA is introduced. In its current version, GeTeX supports Control Area Network (CAN) applications. GeTeX is validated by developing a prototype for that purpose. Using GeTeX, PA is also empirically validated in comparison with some TA testing approaches using a complete industrial-strength test bed. The assessment is based on fault coverage, structural coverage, the length of generated test cases and a proposed assessment factor. The assessment is based on fault coverage, structural coverage, the length of generated test cases and a proposed assessment factor. The assessment results confirmed the superiority of PA over the other test approaches. The overall assessment factor showed that structural and fault coverage scores of PA with respect to the length of its tests were better than the others proving the applicability of PA. Finally, an Analytical Hierarchy Process (AHP) decision-making framework for our PA is developed. The framework can provide testers with a systematic approach by which they can prioritise the available PA test sets that best fulfils their testing requirements. The AHP framework developed is based on the data collected heuristically from the test bed and data collected by interviewing testing experts. The framework is then validated using two testing scenarios. The decision outcomes of the AHP framework were significantly correlated to those of testing experts which demonstrated the soundness and validity of the framework.

Since the early days of software testing, automatic test generation has been suggested as a way of allowing tests to be created at a lower cost. However, industrially useful and applicable tools for automatic test generation are still scarce. As a consequence, the evidence regarding the applicability or feasibility of automatic test generation in industrial practice is limited. This is especially problematic if we consider the use of automatic test generation for industrial safety-critical control systems, such as are found in power plants, airplanes, or trains. In this thesis, we improve the current state of automatic test generation by developing a technique based on model-checking that works with IEC 61131-3 industrial control software. We show how automatic test generation for IEC 61131-3 programs, containing both functional and timing information, can be solved as a model checking problem for both code and mutation coverage criteria.  The developed technique has been implemented in the CompleteTest tool. To evaluate the potential application of our technique, we present several studies where the tool is applied to industrial control software. Results show that CompleteTest is viable for use in industrial practice; it is efficient in terms of the time required to generate tests that satisfy both code and mutation coverage and scales well for most of the industrial programs considered. However, our results also show that there are still challenges associated with the use of automatic test generation. In particular, we found that while automatically generated tests, based on code coverage, can exercise the logic of the software as well as tests written manually, and can do so in a fraction of the time, they do not show better fault detection compared to manually created tests. Specifically, it seems that manually created tests are able to detect more faults of certain types (i.e, logical replacement, negation insertion and timer replacement) than automatically generated tests. To tackle this issue, we propose an approach for improving fault detection by using mutation coverage as a test criterion. We implemented this approach in the CompleteTest tool and used it to evaluate automatic test generation based on mutation testing. While the resulting tests were more effective than automatic tests generated based on code coverage, in terms of fault detection, they still were not better than manually created tests. In summary, our results highlight the need for improving the goals used by automatic test generation tools. Specifically, fault detection scores could be increased by considering some new mutation operators as well as higher-order mutations. Our thesis suggests that automatically generated test suites are significantly less costly in terms of testing time than manually created test suites. One conclusion, strongly supported by the results of this thesis, is that automatic test generation is efficient but currently not quite as effective as manual testing. This is a significant progress that needs to be further studied; we need to consider the implications and the extent to which automatic test generation can be used in the development of reliable safety-critical systems.