This work focuses on relative comparisons of individual method's performance. It is based on Stochastic Performance Logic, which allows to express, for example, that one method runs at most two times longer than another method. These results are more portable than absolute values. It extends standard unit tests with performance assumptions, which are evaluated during actual run-time of a released application. Dynamically added and removed instrumentation is used for automatic modification of the production code. Instrumentation part uses DiSL framework to be able to seamlessly measure even Java system classes. Methods are measured sequentially, the number of concurrently measured methods is dynamically changed and the measurement code is removed as soon as required data are obtained. The results show that this approach may bring appreciably lower overhead peaks than measuring all methods at once. The prototype was compared with JMH tool and the results show that it is able to accurately measure methods running longer than 1 ms. Powered by TCPDF (www.tcpdf.org)

Master thesis is dedicated to the testing and its set up into new environment. The theoretical part is devoted to testing and projects. Theoretical part defines what is project, tests as a subject of the project management and defines the types of tests, along with the tool selection issues for the stress tests. The practical part is focused on describing the processes necessary for the preparation and implementation of performance tests and the necessary steps in the project management of the project implementation.

This thesis aims to extend the MMSP methodology with performance testing intended for the Competence Center for Quality management on VŠE. Through general introduction to performance testing, definition of terms and metrics, detailed description of the preparation and implementation of performance testing and to it tightly connected testing known as operability. Processed functionality will be demonstrated on a selected tool intended for a performance testing and the functionality of the application will be veryfied at the Competence Centre. The methodology should bring order to the team and elevate it to a level that is requested by customers. The work is aimed both at professional audience in the field of testing, where focus is on explaining methodology for performance testing, and also at small testing teams in general, for which it describes how to approach small team testing. Access to methodology could also help centers in other areas to be able to add more added value to their services. My knowledge comes from QA teams of Sun Microsystems, VŠE Competency center on project for IBM and CZNIC registrar of domain names .CZ, which takes care of Czech domain operation.

Performance Testing is one of the crucial parts of any software cycle process. In today’s world, there is any number of network-based applications. Manual Testing and Automated Testing are the two important ways to test any type of application. For Manual Testing a mobile application known as BlekingeTrafiken is used. For Automated Testing, a web application known as Edmodo is used. Selenium is the automated tool included for automated testing. But, for each application, there are several users and because of that, there might be a decrease in performance of the application as an increase in the number of users. Performance of an application also depends on response times, mean, stability, speed, capacity, accuracy. The performance also depends on the device (memory consumption, battery, software variation) and Server/API (less no of calls) and depends on the network performance (jitters, packet loss, network speed). There are several tools for performance testing. By using these tools, we can get accurate performance results of each request.  In this thesis, we performed manual testing of a mobile application by increasing the number of users under similar network conditions, automated testing of a web application under various test cases and tested the performance of an iPad application (PLANETJAKTEN). It is a real-time gaming application used to learn mathematics for children. Apache JMeter is the tool used for performance testing. The interaction between the JMeter tool and the iPad is done through HTTP Proxy method. When any user starts using the application, we can measure the performance of each request sent by the user. Nagios is the tool used to monitor the various environments. Results show that for manual testing, the time taken for connecting to WI-FI is low compared to opening and using the application. For automated testing, it is found that the time taken to run each test case for the first time is high compared to the remaining trials. For performance testing, the experimental results show that the error percentage (the percentage of failed requests) is high for logging into the application compared to using the application.

Modern computer software systems are complicated. Developers can change the behavior of the software system through software configurations. The large number of configuration option and their interactions make the task of software tuning, testing, and debugging very challenging. Performance is one of the key aspects of non-functional qualities, where performance bugs can cause significant performance degradation and lead to poor user experience. However, performance bugs are difficult to expose, primarily because detecting them requires specific inputs, as well as specific configurations. While researchers have developed techniques to analyze, quantify, detect, and fix performance bugs, many of these techniques are not effective in highly-configurable systems. To improve the non-functional qualities of configurable software systems, testing engineers need to be able to understand the performance influence of configuration options, adjust the performance of a system under different configurations, and detect configuration-related performance bugs. This research will provide an automated framework that allows engineers to effectively analyze performance-influence configuration options, detect performance bugs in highly-configurable software systems, and adjust configuration options to achieve higher long-term performance gains. To understand real-world performance bugs in highly-configurable software systems, we first perform a performance bug characteristics study from three large-scale opensource projects. Many researchers have studied the characteristics of performance bugs from the bug report but few have reported what the experience is when trying to replicate confirmed performance bugs from the perspective of non-domain experts such as researchers. This study is meant to report the challenges and potential workaround to replicate confirmed performance bugs. We also want to share a performance benchmark to provide real-world performance bugs to evaluate future performance testing techniques. Inspired by our performance bug study, we propose a performance profiling approach that can help developers to understand how configuration options and their interactions can influence the performance of a system. The approach uses a combination of dynamic analysis and machine learning techniques, together with configuration sampling techniques, to profile the program execution, analyze configuration options relevant to performance. Next, the framework leverages natural language processing and information retrieval techniques to automatically generate test inputs and configurations to expose performance bugs. Finally, the framework combines reinforcement learning and dynamic state reduction techniques to guide subject application towards achieving higher long-term performance gains.

Moderní pokrok v oblasti technologií pro vývoj dnešních softwarových aplikací umožnil vývojářům více se soustředit na vývoj funkčnosti aplikace na úkor sledování jejího výkonu a správy zdrojů. V důsledku toho se zvýšily požadavky na nástroje pro výkonnostní testování, které by měly poskytovat vývojářům jasný a srozumitelný přehled o stavu systému z hlediska jeho výkonu a umožnit rychlou interpretaci naměřených výsledků. Tato práce zkoumá typické výkonnostní problémy dnešních aplikací a navrhuje přístupy, pomocí kterých je možné tyto anomálie automaticky rozpoznat. Pomocí statistických metod, jako je regresní a korelační analýza, je provedena analýza dat naměřených během výkonnostního testování s cílem rozpoznat ve výsledcích odchylky od normálního chování a z nich identifikovat výkonnostní problémy. Výsledkem je report o celkovém stavu systému z hlediska jeho výkonu. Implementací regresní analýzy je možné detekovat výkonnostní problémy jako je například zhoršující se reakční čas odpovědi, nízká propustnost systému či odhalit únik paměti. Navrhovaný přístup byl implementován v podobě nové komponenty v open-source nástroji pro výkonnostní testování PerfCake. Vyvinutá komponenta je schopna detekovat a reportovat potenciální výkonnostní problémy a jejich pravděpodobnost.

This thesis is concerned about performance and stress testing of Xtend product developed by Xura, Inc. Software development knowledge, theoretical knowledge of testing and testing tools are described in opening chapters together with key features and services provided by Xtend. Emphasis was put on implementation of performance and stress testing tool, which focuses on short-term and long-term testing scenarios and output of this tool serves Xtend developers. Part of this thesis also focuses on results of stress and performance tests.

No description available.

Continuous delivery is a software development methodology that aims to reduce development cycle time by putting a strong emphasis on automation, quality and rapid feedback. This thesis develops an automated method for detecting performance regressions as part of a continuous delivery deployment pipeline. The chosen method is based on control charts, a tool commonly used within statistical process control. This method is implemented as part of a continuous delivery deployment pipeline and its ability to detect performance regressions is then evaluated by injecting various performance bottlenecks in a sample application. The results from this thesis show that using a control chart based approach is a viable option when trying to automate verification of load test results in the context of continuous delivery. / Kontinuerlig leverans är en utvecklingsmetodik för mjukvara med målet att reducera ledtid genom att fokusera på automatisering, kvalitet och snabb återkoppling. I det här examensarbetet utvecklas en automatiserad metod för att upptäcka försämringar i prestanda i en deployment pipeline för kontinuerlig leverans. Den valda metoden baseras på kontrolldiagram, ett verktyg som ofta används inom statistisk processkontroll. Metoden implementeras som en del av en deployment pipeline för kontinuerlig leverans och dess förmåga att upptäcka prestandaförsämringar utvärderas genom att olika prestandarelaterade flaskhalsar implementeras i en testapplikation. Resultaten från arbetet visar att en metod baserad på kontrolldiagram är ett tänkbart alternativ för att automatisera verifiering lasttestresultat inom kontinuerlig leverans.

Målet med denna undersökning har varit att ta fram lämpligt bibliotek i programspråket C för generering och parsning av JSON (JavaScript Object Notation). Biblioteket ska användas för att bygga ett nytt API mot Svenska Spels transaktionssystem som ska underlätta för klientutvecklare att anropa systemet. Undersökningen har begränsats till utvärdering två utvalda bibliotek, YAJL (Yet Another Json Library) och Jansson. För att kunna jämföra och analysera dessa har jag implementerat ett testprogram som mäter deras prestanda vid generering och parsning av ett antal utvalda JSON objekt. Jag har även undersökt biblioteken utifrån användarvänlighet och robusthet. I uppdraget har även ingått att undersöka om det går att skapa ett eget bibliotek som har bättre prestanda än de båda utvalda. Efter utvärdering av resultaten från dessa undersökningar har jag tillslut kommit fram till vilket av biblioteken som är bäst anpassat för Svenska Spels behov. / The objective of this study was to find a suitable library in the C programming language for generating and parsing JSON (JavaScript Object Notation). The library will be used to build a new API for Svenska Spels transaction system which will make it easier for client developers to invoke the system. The study has been limited to evaluation of two selected libraries, YAJL (Yet Another Json Library) and Jansson. In order to compare and analyze them I have implemented a test program that measures the performance of generation and parsing of a selected number of JSON objects. I have also examined the libraries based on user-friendliness and robustness. The task has also been to study the feasibility of creating a library which has better performance than the two selected. After evaluating the results of these studies, I have finally come to the conclusion of which library is most adapted to Svenska Spels needs.