Testing is a crucial process for organizations to identify any defects, ensure high-quality standards, enhance customer experience, mitigate risks, and comply with regulations. With proactive identification and resolution of problems, efficient testing saves time, money, and effort, ultimately contributing to the organization's overall success. The ABB Robotics Quality Management team for New Product Development projects identified that the current testing is process-oriented which results in overlapping test activities, delays, and increased costs. The lack of a holistic approach was identified as the main problem for the testing processes. Testing for New Product Development in the robotics industry is a time-consuming and costly activity. To deliver high-quality and reliable robot solutions, it is essential to test the right items at the right stage of development. The purpose of this research is to build a Test Strategy for New Product Development projects at ABB Robotics. The major objectives for the generated test strategy include identification of the right test methodologies, early identification of bugs and failures, achieving sufficient test coverage through component, module, integration, and system levels of testing, testing at the right level and stage, early verification of all requirements, better product quality, accelerated time to market, and superior customer experience. This research will focus on the mentioned aspects and won't address certain areas such as how suppliers conduct their testing, how test data is transferred from suppliers to ABB Robotics, processes after new product development projects are completed, and the testing processes in customer service. The research methodology involves a thorough literature review, an analysis of internal documents, international standards, and interviews within ABB Robotics, as well as external interviews and benchmarking with three well-known companies in the industry. With the support of the findings, recognized patterns, and analysis, a test strategy with its three major key strategic themes aligned with ABB Robotics' research objectives was generated with an implementation plan.

Este trabalho propõe dois métodos para teste de sistemas de software: o primeiro extrai ideias de teste de um modelo desenvolvido em rede de Petri hierárquica e o segundo valida os resultados após a realização dos testes utilizando um modelo em OWL-S. Estes processos aumentam a qualidade do sistema desenvolvido ao reduzir o risco de uma cobertura insuficiente ou teste incompleto de uma funcionalidade. A primeira técnica apresentada consiste de cinco etapas: i) avaliação do sistema e identificação dos módulos e entidades separáveis, ii) levantamento dos estados e transições, iii) modelagem do sistema (bottom-up), iv) validação do modelo criado avaliando o fluxo de cada funcionalidade e v) extração dos casos de teste usando uma das três coberturas de teste apresentada. O segundo método deve ser aplicado após a realização dos testes e possui cinco passos: i) primeiro constrói-se um modelo em OWL (Web Ontology Language) do sistema contendo todas as informações significativas sobre as regras de negócio da aplicação, identificando as classes, propriedades e axiomas que o regem; ii) em seguida o status inicial antes da execução é representado no modelo através da inserção das instâncias (indivíduos) presentes; iii) após a execução dos casos de testes, a situação do modelo deve ser atualizada inserindo (sem apagar as instâncias já existentes) as instâncias que representam a nova situação da aplicação; iv) próximo passo consiste em utilizar um reasoner para fazer as inferências do modelo OWL verificando se o modelo mantém a consistência, ou seja, se não existem erros na aplicação; v) finalmente, as instâncias do status inicial são comparadas com as instâncias do status final, verificando se os elementos foram alterados, criados ou apagados corretamente. O processo proposto é indicado principalmente para testes funcionais de caixa-preta, mas pode ser facilmente adaptado para testes em caixa branca. Obtiveram-se casos de testes semelhantes aos que seriam obtidos em uma análise manual mantendo a mesma cobertura do sistema. A validação provou-se condizente com os resultados esperados, bem como o modelo ontológico mostrouse bem fácil e intuitivo para aplicar manutenções. / This paper proposes two test methods for system software testing: the first one extracts test workflow processes from a model developed in Hierarchical Petri Nets and the other validates results after test execution using a domain model in OWL-S. Both processes increase the quality of the system developed by reducing the risk of having an insufficient coverage or an incomplete functionality test. The first technique consists of five steps: i) system evaluation and identification of separable sub modules and entities, ii) identification of states and transitions, iii) system modeling (bottom-up), iv) validation of the created model by evaluating the workflow for each functionality, and v) extraction of test cases using one of the three test coverage presented. The second method must be applied after the execution of the previous method and has also five steps: i) first a system model in OWL (Web Ontology Language) is built containing all significant information and business rules of the application; ii) then, the initial status before the test execution is represented in the model by the insertion of the instances (individuals) presented; iii) after the execution of test cases, the state model is updated by inserting (without deleting already existing instances) new instances to represent the domain sate after test; iv) in the next step we use a reasoner to make OWL model checking inferences to prove model consistency, that is, if there is no error in the application; finally, the initial status instances is compared with the final status in order to verify if these instances have been changed, created or deleted correctly. The process is indicated for blackbox functional tests, but can be easily adapted for white-box tests. There was obtained test cases similar to those that will be obtained in a manual analysis keeping the same test coverage. Validation has proved to be consistent compare to the expected results. Also, the ontological model has showed to be easy and intuitive for maintenance.

Este trabalho propõe dois métodos para teste de sistemas de software: o primeiro extrai ideias de teste de um modelo desenvolvido em rede de Petri hierárquica e o segundo valida os resultados após a realização dos testes utilizando um modelo em OWL-S. Estes processos aumentam a qualidade do sistema desenvolvido ao reduzir o risco de uma cobertura insuficiente ou teste incompleto de uma funcionalidade. A primeira técnica apresentada consiste de cinco etapas: i) avaliação do sistema e identificação dos módulos e entidades separáveis, ii) levantamento dos estados e transições, iii) modelagem do sistema (bottom-up), iv) validação do modelo criado avaliando o fluxo de cada funcionalidade e v) extração dos casos de teste usando uma das três coberturas de teste apresentada. O segundo método deve ser aplicado após a realização dos testes e possui cinco passos: i) primeiro constrói-se um modelo em OWL (Web Ontology Language) do sistema contendo todas as informações significativas sobre as regras de negócio da aplicação, identificando as classes, propriedades e axiomas que o regem; ii) em seguida o status inicial antes da execução é representado no modelo através da inserção das instâncias (indivíduos) presentes; iii) após a execução dos casos de testes, a situação do modelo deve ser atualizada inserindo (sem apagar as instâncias já existentes) as instâncias que representam a nova situação da aplicação; iv) próximo passo consiste em utilizar um reasoner para fazer as inferências do modelo OWL verificando se o modelo mantém a consistência, ou seja, se não existem erros na aplicação; v) finalmente, as instâncias do status inicial são comparadas com as instâncias do status final, verificando se os elementos foram alterados, criados ou apagados corretamente. O processo proposto é indicado principalmente para testes funcionais de caixa-preta, mas pode ser facilmente adaptado para testes em caixa branca. Obtiveram-se casos de testes semelhantes aos que seriam obtidos em uma análise manual mantendo a mesma cobertura do sistema. A validação provou-se condizente com os resultados esperados, bem como o modelo ontológico mostrouse bem fácil e intuitivo para aplicar manutenções. / This paper proposes two test methods for system software testing: the first one extracts test workflow processes from a model developed in Hierarchical Petri Nets and the other validates results after test execution using a domain model in OWL-S. Both processes increase the quality of the system developed by reducing the risk of having an insufficient coverage or an incomplete functionality test. The first technique consists of five steps: i) system evaluation and identification of separable sub modules and entities, ii) identification of states and transitions, iii) system modeling (bottom-up), iv) validation of the created model by evaluating the workflow for each functionality, and v) extraction of test cases using one of the three test coverage presented. The second method must be applied after the execution of the previous method and has also five steps: i) first a system model in OWL (Web Ontology Language) is built containing all significant information and business rules of the application; ii) then, the initial status before the test execution is represented in the model by the insertion of the instances (individuals) presented; iii) after the execution of test cases, the state model is updated by inserting (without deleting already existing instances) new instances to represent the domain sate after test; iv) in the next step we use a reasoner to make OWL model checking inferences to prove model consistency, that is, if there is no error in the application; finally, the initial status instances is compared with the final status in order to verify if these instances have been changed, created or deleted correctly. The process is indicated for blackbox functional tests, but can be easily adapted for white-box tests. There was obtained test cases similar to those that will be obtained in a manual analysis keeping the same test coverage. Validation has proved to be consistent compare to the expected results. Also, the ontological model has showed to be easy and intuitive for maintenance.

Submitted by Cláudia Bueno (claudiamoura18@gmail.com) on 2015-10-27T15:54:55Z No. of bitstreams: 2 Dissertação - Dianne Dias Silva - 2015.pdf: 2508002 bytes, checksum: caf4c84566bed1bb6b673f295e99b317 (MD5) license_rdf: 23148 bytes, checksum: 9da0b6dfac957114c6a7714714b86306 (MD5) / Approved for entry into archive by Luciana Ferreira (lucgeral@gmail.com) on 2015-10-28T14:09:40Z (GMT) No. of bitstreams: 2 Dissertação - Dianne Dias Silva - 2015.pdf: 2508002 bytes, checksum: caf4c84566bed1bb6b673f295e99b317 (MD5) license_rdf: 23148 bytes, checksum: 9da0b6dfac957114c6a7714714b86306 (MD5) / Made available in DSpace on 2015-10-28T14:09:40Z (GMT). No. of bitstreams: 2 Dissertação - Dianne Dias Silva - 2015.pdf: 2508002 bytes, checksum: caf4c84566bed1bb6b673f295e99b317 (MD5) license_rdf: 23148 bytes, checksum: 9da0b6dfac957114c6a7714714b86306 (MD5) Previous issue date: 2015-05-25 / Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - CAPES / The software products developed by micro and small brazilian enterprises have stood out on the world IT market which mostly are dedicated to the development, the production, the software distribution and the provision of services. However, new software products available on the market fail because most of these micro and small enterprises are more focused on product quality rather than the processes used in its development. Despite these organizations realize the importance of improving their processes and their working techniques, lack resources and knowledge so that this practice is applied. In recent decades, increasing demand and complexity of software products made that micro and small software development enterprises worry and invest more in quality assurance to provide continuous improvement of the techniques, the criteria, the methods and the tools used in the construction of software products. Through some national initiatives maturity models and international standards such as the Freetest Method, the MPT.BR and the ISO/IEC/IEEE 29119-2 can satisfy the needs and expectations of users of these products. However, the implementation and institutionalization of structured testing process maturity in these models and this standard are costly and complex for micro and small software development enterprises. This work developed a maturity model considering the limitations of micro and small software development enterprises, and an approach to implement the test procedures defined in this model. / Os produtos de software desenvolvidos pelas micro e pequenas empresas brasileiras têm se destacado no mercado mundial de TI que em sua maioria estão dedicadas ao desenvolvimento, à produção, à distribuição de software e à prestação de serviços. No entanto, os novos produtos de software disponibilizados no mercado falham devido a maioria dessas micro e pequenas empresas estarem mais centradas na qualidade do produto em vez dos processos adotados no seu desenvolvimento. Apesar dessas organizações perceberem a relevância de melhorar os seus processos e suas técnicas de trabalho, faltam recursos e conhecimento para que essa prática seja aplicada. Nas últimas décadas, a crescente demanda e a complexidade dos produtos de software fizeram com que as micro e pequenas empresas de desenvolvimento de software se preocupassem e, investissem cada vez mais na garantia da qualidade para prover a melhoria contínua das técnicas, dos critérios, dos métodos e das ferramentas empregadas na construção dos produtos de software. Através de algumas iniciativas nacionais de modelos de maturidade e normas internacionais tais como, o Método Freetest, o MPT.BR e a ISO/IEC/IEEE 29119-2 é possível satisfazer as necessidades e expectativas dos usuários desses produtos. Porém, a implementação e a institucionalização do processo de teste estruturado nesses modelos de maturidade e, nessa norma são onerosas e complexas para as micro e pequenas empresas de desenvolvimento de software. Neste trabalho foi desenvolvido um modelo de maturidade considerando as limitações das micro e pequenas empresas de desenvolvimento de software, e uma abordagem para implementar os processos de teste definidos nesse modelo.

The aim of this thesis is to improve the reliability and quality of new requested functionality, and existing modules, at Fortum HR System Solutions. This was conducted through an evaluation of the test processes by implementing principles of Software Testing and Test Management. For the study to successfully improve the testing performed at HR System Solutions, existing test processes were analyzed. The analysis was conducted by evaluating the current test processes using theoretical test evaluation styles called maturity models. The methodology of choice was the Testing Maturity Model (TMM), which was adapted to the nature of HR System Solutions requirements, experience and needs. The evaluation used qualitative methods together with pedagogical principles to conduct interviews and workshops to consolidate the theoretical evaluation. Interviews and workshops were conducted within the team of HR System Solutions, where all members of them team contributed to the thesis at some point. An external interview also took place for comparative study. Results of the evaluation, interviews and workshop were compiled and analyzed accordingly. With the analyzed results in place, flaws in the testing processes were apparent. A generalization of the flaws led to the conclusion in the form of a suggestion. The conclusive suggestion was for Fortum HR System Solutions to establish a test committee/group role within the team. Considering the current economical and organizational situation this job role would be a divided job role appointed to current members of the HR System Solutions team. The research creates a walkthrough on a potential method on understanding inefficiencies within testing processes of a company and providing a cause-based solution. / Undersökningen är inriktad på att förbättra tillförlitligheten och kvalitén på ny begärd funktionalitet, och befintliga moduler, på Fortum HR System Solutions. Detta genomfördes genom en utvärdering av testprocesser genom implementation av principer inom Software Testing samt Test Management. För att förbättra testningen som utförs på HR System Solutions var det nödvändigt att analysera de befintliga testprocesserna. Analysen genomfördes genom att utvärdera de nuvarande testprocesserna med hjälp av teoretiska utvärderingsmetoder som kallas Maturity Models. Den valda metoden var Testing Maturity Model (TMM) som tillämpades med avseende på HR System Solutions förutsättningar, erfarenheter och behov. Utvärderingen använder sig av kvalitativa metoder samt pedagogiska principer för att genomföra intervjuer och workshops stärka den teoretiska utvärderingen från TMM. Intervjuer och workshops genomfördes inom HR System Solutions arbetslag, där alla medlemmar bidrog till examensarbetet. Även en extern intervju gjordes i jämförande syfte. Resultatet av utvärderingen sammanställdes och analyserades i enlighet med de förnämnda teoretiska ramvärken. Med hjälp av analyserna visade sig bristerna hos HR System Solutions testprocesser. En generalisering av bristerna gjordes, vilket ledde till slutsatsen i form av ett förslag. Det slutliga förslaget innebar att Fortum HR System Solutions behöver upprätta en testkommitté/grupp inom arbetslaget. Undersökningen visar en potentiell metod för att förstå ineffektiviteten inom testprocesser i ett företag, och tillhandahålla en orsaks-baserad lösning.