Performance Testing is one of the crucial parts of any software cycle process. In today’s world, there is any number of network-based applications. Manual Testing and Automated Testing are the two important ways to test any type of application. For Manual Testing a mobile application known as BlekingeTrafiken is used. For Automated Testing, a web application known as Edmodo is used. Selenium is the automated tool included for automated testing. But, for each application, there are several users and because of that, there might be a decrease in performance of the application as an increase in the number of users. Performance of an application also depends on response times, mean, stability, speed, capacity, accuracy. The performance also depends on the device (memory consumption, battery, software variation) and Server/API (less no of calls) and depends on the network performance (jitters, packet loss, network speed). There are several tools for performance testing. By using these tools, we can get accurate performance results of each request.  In this thesis, we performed manual testing of a mobile application by increasing the number of users under similar network conditions, automated testing of a web application under various test cases and tested the performance of an iPad application (PLANETJAKTEN). It is a real-time gaming application used to learn mathematics for children. Apache JMeter is the tool used for performance testing. The interaction between the JMeter tool and the iPad is done through HTTP Proxy method. When any user starts using the application, we can measure the performance of each request sent by the user. Nagios is the tool used to monitor the various environments. Results show that for manual testing, the time taken for connecting to WI-FI is low compared to opening and using the application. For automated testing, it is found that the time taken to run each test case for the first time is high compared to the remaining trials. For performance testing, the experimental results show that the error percentage (the percentage of failed requests) is high for logging into the application compared to using the application.

Software projects with a larger code base can be hard to maintain, and testing could therefore be a necessity. But how would one prioritize tests in their existing software project, if their project never had any tests? The purpose of this study is to find concrete steps of how to prioritize tests in a software project, independent of project type and development language. Said steps were found through two iterations of the Action Learning method. The results were found without any requirements specification, which lead to using intuition, together with code complexity, to prioritize test cases, after breaking down the project into its modules. As the study was conducted on only one specific software project, the result is limited to that specific software project until further research is conducted.

GUI testing is expensive to perform manually. Software systems involving a heterogeneous set of components exclude the applicability of specific GUI testing techniques. Visual GUI Testing (VGT) is a test automation technique that combines image recognition with scripts. It applies to almost any GUI driven application. VGT is proven to be cost-effective in comparison with manual testing. Still, it is expensive to maintain. This study investigates if test script design by following specific guidelines positively affects maintenance costs. A case study was conducted to identify best practices for VGT w.r.t. maintenance time. Four VGT versions were developed for each manual test case. These consisted of two design versions, with/without guidelines, for the two VGT-tools EyeAutomate and Sikuli. Data was collected using time measurements, observations, and interviews. Results highlighted differences in initial development time and maintenance time between the two design versions. In total, 44 observations were collected. 17 were related to the design versions, 17 to the VGT-tools, and 10 to VGT in general, initial development, and the system under test. The interviews collected the perceptions of VGT in general, maintenance of the different VGT versions, and guidelines. In conclusion, the combination of the guidelines did not have a positive effect on maintenance in terms of costs and experience. However, some of the individual guidelines did. A rationale why the guidelines did not give the desired result was identified. Future research is necessary to investigate other combinations of guidelines, such as those identified as beneficial.

This master’s thesis deals with the design and implementation of part of test environment for automatic testing of electronic control unit in the production. The electronic control unit is manufactured by UNIS, a.s. and is used as a control and power electronics for the hydraulic unit. The theoretical part of the thesis deals with the development and automatic testing of electronics in the field of aerospace engineering. The practical part describes the proposed automatic test environment and implementation of testing software using hardware and software products by National Instruments. The main platforms used for software development are LabVIEW and TestStand. Created test environment is finally verified by testing several control units.

Automated testing is a fast-growing requirement for many IT companies. The idea of testing is to create a better product for both the company and the customer. The goal of this study is to examine different aspects of au- tomated testing and Xray as a test management tool. The literature study considers information from several different scientific reports. It shows that the benefits of automated testing include increased productivity and reliable software but also pitfalls like a high initial cost and a maintenance cost. Research suggests that automated testing is complementary to man- ual testing. Manual testing is more suited for exploratory testing, while automated testing is better for regression testing. Using historical data manual tests can be placed into prioritised clusters. The coverage of each test within a cluster determines its priority, where a test with high cover- age has a high priority. A near-optimal solution for prioritising automated tests is the combination of two well-known strategies, the additional cov- erage strategy and the total coverage strategy. Tests are prioritised based on how much of the code they uniquely cover. Code coverage is mea- sured using statements, methods or complexity. Furthermore, this thesis demonstrates a proof of concept for how the unified algorithm can pri- oritise Xray tests. Xray is evaluated according to the ISO/IEC 25010:2011 standard together with a survey done on Xray practitioners. The evalua- tion for Xray shows that Xray provides the necessary tools and functions to manage a testing suite successfully. However, no official encryption exist for the Jira server, and Xray lacks integrated documentation.

With the development of Advanced Driving Assistance Systems (ADAS) vehicles have undergone better experience in field of safety, better driving and enhanced vehicle systems. Today these systems are one of the fastest growing in automotive domain. Physical parameters like map data, vehicle position and speed are crucial for the advancement of functionalities implemented for ADAS. All the navigation map databases are stored in proprietary format. So for the ADAS application to access this data an appropriate interface has to be defined. This is the main aim of Advance Driver Assistant Systems Interface Specifications (ADASIS) consortium. This new specification allows a coordinated effort of more than one industry to improve comfort and fuel efficiency. My research during the entire duration of my master thesis mainly focuses on two stages namely XML Comparator and CAN stream generation stages from ADASIS Test Environment that was developed in our company. In this test environment ADASIS Reconstructor of our company is tested against the parameters of Reference Reconstructor provided by ADASIS consortium. The main aim of this environment is to develop a Reconstructor which will adhere to all the specifications given in ADASIS Reconstructor. My implementation in this master thesis focuses on two stages of test environment setup which are XML Comparison and CAN Stream Generation Tool respectively. Prior to my working, these stages lacked in-depth research and usability features for further working.

Att skapa inkluderande webbupplevelser innebär att användare, oavsett förutsättningar, ska kunna uppfatta, förstå sig på, interagera med och kunna bidra till tjänster på webben. För att arbeta med webbtillgänglighet finns uppsatta standarder för att upptäcka och minimera begränsningar. Trots uppsatta standarder tillhandahåller flertalet myndigheter otillgängliga tjänster, vilket sägs bero på inkompetens och resursbrist. Studiens mål är därmed att bidra med kunskap om automatiserade tillgänglighetstestverktygs effektivitet. Verktygen är implementerade inom och för mjukvaruprocesser och har integrerats med ett End-To-End-testverktyg. Studiens mål uppnås med hjälp av en kvantitativ datainsamling inspirerad av Brajniks definition av effektivitet som mäter verktygens exekveringstid, fullständighet, specificitet och korrekthet. Utifrån mätningar på 4 av World Wide Web Consortiums demonstrationswebbsidor baserat på WCAG 2.1 enligt DIGGs tillsynsmanual visar studien att tillgänglighetstestverktygen Pa11y, QualWeb, IBM Equal Access och Google Lighthouse presterar bättre och sämre inom olika områden. Studien uppmärksammar att QualWeb i genomsnitt har kortast exekveringstid med ca 3933 millisekunder. QualWeb har även högst andel genomsnittlig fullständighet (80,94 %) och specificitet (58,26 %). Det verktyg med högst andel korrekthet är Google Lighthouse (99,02 %). Inget av verktygen anses perfekt eftersom samtliga verktyg gör felbedömningar. Studiens slutsats menar därmed att QualWeb är ett effektivare tillgänglighetstestverktyg som kräver komplettering av ytterligare testningsmetoder såsom manuella och användarcentrala tester. / Creating inclusive web experiences means that users, regardless of their circumstances, should be able to perceive, understand, interact with, and contribute to services on the web. To ensure web accessibility, there are established standards to detect and minimize limitations. However, despite these standards, most authorities provide inaccessible services, due to incompetence and resource constraints. The goal of the study is thus to contribute knowledge about automated accessibility testing tools effectiveness. These tools are implemented within and for software processes and are integrated with an End-To-End testing tool. The study achieves its goals through quantitative data collection inspired by Brajnik’s definition of efficiency, measuring the tools’ execution time, completeness, specificity, and correctness. Based on measurements taken from four of the World Wide Web Consortium’s demonstration websites, following WCAG 2.1 according to DIGG’s supervision manual, the study shows that the accessibility testing tools Pa11y, QualWeb, IBM Equal Access, and Google Lighthouse perform better and worse in different areas. The study highlights that QualWeb has the shortest average execution time at approximately 3933 milliseconds. QualWeb also has the highest average completeness (80.94%) and specificity (58.26%). The tool with the highest correctness rate is Google Lighthouse (99.02%). None of the tools are considered perfect, as all of them make mistakes. The study’s conclusion suggests that QualWeb is a more effective accessibility testing tool that requires additional testing methods such as manual, and user testing.

When designing a software, it was commonplace to use specialised techniques to ensure quality of the final product. Continuous Integration (CI) provides developers a strategy by allowing for teams to continuously add to their project daily without conflicting with prior work. An automated process would check that these new integration additions are functional so that any faults were quickly identified for correction. Continuous Integration played a pivotal role in organisations that developed software solutions for their own or their customers’ processes. It comes as no shock that an abundance of research illustrates its importance and success in software development. Automated testing was praised so much so that there was a distinct lack of negatives particularly in the professional industry. The goal of this study was to gather knowledge about automated testing in a work environment so that principles for implementing automated testing. This was conducted by a qualitative case study while looking at the developer/user perspective. This was done by conducting semi-structured interviews with employees working with automated testing in a telecommunication organisation. The study’s results presented eight themes that the informants and prior literature studies observed as issues of importance. These themes showed that the automated testing had experienced problems in practice that were not noticed in academical papers and for that, seven recommendations were presented for other organisations to be observant of.

Testing safety-critical systems, particularly those controlled by Programmable Logic Controllers (PLC), is crucial for ensuring the safe and reliable operation of industrial processes. This thesis addresses the critical need for automated testing of safety-critical PLC systems used in various industrial settings. Despite the significance of testing, current practices rely heavily on manual methods, leading to challenges in scalability and reliability. This work investigates enabling test automation for PLCs to facilitate and assist the current manual testing procedures in the industry. The thesis proposes and evaluates test automation techniques and tools tailored to PLCs, focusing on Function Block Diagram and Structured Text languages commonly used in industry. We systematically compare test automation tools for PLC programs, after which we propose a PLC to Python translation framework called PyLC to facilitate automated test generation. The experiment employing the EARS requirement engineering pattern reveals that while engineers use semi-formal notations in varied ways to create requirements, leading to completeness issues, it confirms the viability of employing EARS requirements for PLC system testing. Subsequently, the proposed automation approaches are fully implemented and evaluated using real-world PLC case studies, comparing their efficiency against manual testing procedures. The findings highlight the feasibility and benefits of automating PLC testing, offering insights into improving development and testing processes through carefully selected automation tools for the CODESYS IDE, a well-known PLC development environment. Additionally, we show that leveraging Python-based automated testing techniques and mutation analysis enhances testing effectiveness. Furthermore, incorporating best practices in requirement engineering, as demonstrated by the EARS approach, contributes to further enhancing testing efficiency and effectiveness in PLC development. / VeriDevOps, SmartDelta

This thesis presents an extensive survey of 100,000 websites as the basis for understanding the deployment of cookies across the Internet. The survey indicates cookie deployment on the Internet is approaching universal levels. The survey identifies the presence of P3P policies and dynamic web technologies as major predictors of cookie usage, and a number of significant relationships are established between the origin of the web application and cookie deployment. Large associations are identified between third-party persistent cookie usage and a countrys e-business environment. Cookie collection testing (CCT), a strategy for testing web applications, is presented. Cookies maintained in a browser are explored in light of anti random testing techniques, culminating in the definition of seeding vectors as the basis for a scalable test suite. Essentially CCT seeks to verify web application robustness against the modificationintentional or otherwiseof an application's internal state variables. Automation of CCT is outlined through the definition of test oracles and evaluation criterion. Evolutionary adaptive random (eAR) testing is proposed for application to the cookie collection testing strategy. A simulation study is undertaken to evaluate eAR against the current state-of-the-art in adaptive random testingfixed size candidate set, restricted random testing, quasi-random testing, and random testing. eAR is demonstrated to be superior to the other techniques for block pattern simulations. For fault patterns of increased complexity, eAR is shown to be comparable to the other methods. An empirical investigation of CCT is undertaken. CCT is demonstrated to reveal defects within web applications, and is found to have a substantial fault-triggering rate. Furthermore, CCT is demonstrated to interact with the underlying application, not just the technological platform upon which an application is implemented. Both seeding and generated vectors are found to be useful in triggering defects. A synergetic relationship is found to exist between the seeding and generated vectors with respect to distinct fault detection. Finally, a large significant relationship is established between structural and content similarity measures of web application responses, with a composite of the two similarity measures observed to be superior in the detection of faults. / Software Engineering and Intelligent Systems