Conditioning on the observed data is an important and flexible design principle for statistical test procedures. Although generally applicable, permutation tests currently in use are limited to the treatment of special cases, such as contingency tables or K-sample problems. A new theoretical framework for permutation tests opens up the way to a unified and generalized view. We argue that the transfer of such a theory to practical data analysis has important implications in many applications and requires tools that enable the data analyst to compute on the theoretical concepts as closely as possible. We re-analyze four data sets by adapting the general conceptual framework to these non-standard inference procedures and utilizing the coin add-on package in the R system for statistical computing to show what one can gain from going beyond the `classical' test procedures. / Series: Research Report Series / Department of Statistics and Mathematics

Integral part of all systematically managed software development or maintenance projects is emphasis on continuous quality of all project activities. Because final quality of project deliverables (new or enhanced applications, preconfigured solutions as ie. SAP) is a very big influencer of project success and therefore also important influencer of long-term relationship between customer and contractor(s), this document focuses on ways how to proactively prevent from mistakes (within the whole software development lifecycle) and on techniques helping to establish better quality control of important deliverables through systematic approach, high quality tools and suitable metrics in software testing discipline. This document gradually exposes typical project areas where it is necessary to keep control on quality of project members' outputs, perceives testing in context of typical project consequences, offers practical recommendations in testing methodology, tools as well as widely tested technologies, and explains trends and risks in testing domain. The goal of this document is not only to document a wide range of possibilities given by frequently used testing techniques or tools but also to offer a practical guidance in deployment of test discipline. This document was written by comparing author's professional experience in software quality management with knowledge gathered by reading information sources attached to this document. This document consists of concrete conclusions of this comparison.

This diploma thesis describes the position of software testing according to Lean software development approach and its appliacation in testing team. At first. Lean concept is introduced with its basic principles, then the thesis focuses on the position of testing in the Lean software development itself. At the beginning this thesis deals with agile approach to development and software testing position in Lean software development. This concept defines several principles, which should be followed by testing teams. These principles and its general application in testing teams are subsequently explained. The second part is devoted to Lean software development in a real company. I am working as a tester and I already know the development and testing processes -- that's why one of my goals is to apply this approach to our testing team. Firstly the default situation in our company is introduced. Used technics and types of used testing tools are listed together with procedures and also the disadvantages and troubles, which occurred during the original testing process. Then the following parts are devoted to Lean software development principles application in our testing team followed with description of improvements and advantages delivered by the principles.

Software projects go through a phase of maintenance and, in case of open source projects, through hard development process. Both of these phases are prone to regressions, meaning previously working parts of system do not work anymore. To avoid this behavior, systems are being tested with long test suites, which can be sometimes time consuming. For this reason, prediction models are developed to predict software regressions using historical testing data and code changes, to detect changes that can most likely cause regression and focus testing on such parts of code. But, these predictors rely on static code analysis without deeper semantic understanding of the code. Purpose of this master thesis is to create predictor, that relies not only on static code analysis, but provides decisions based on code semantics as well.

El testing del software es una tarea crucial y a la vez muy desafiante dentro del proceso de desarrollo de software. El testing permite encontrar errores y problemas del software contra la especificación del mismo y cumple un rol fundamental en el aseguramiento de la calidad del producto. Entre los tipos de pruebas que se pueden realizar al software están las pruebas de unidad, carga, integración y funcionales. Cada una de ellas tiene distintos objetivos y son realizadas en diferentes etapas del desarrollo del software. En el primer tipo mencionado, se desarrollan pruebas a componentes individuales de un sistema de software. Los desarrolladores especifican y codifican pruebas para cubrir todos o al menos una parte significativa de los posibles estados/configuraciones del artefacto o unidad de software, para simular el entorno del componente y descubrir la presencia de errores o “bugs”. Dado que escribir todas esas pruebas de forma manual es costoso, las pruebas de unidad son generalmente realizadas de manera ineficiente o simplemente dejadas de lado. El panorama es aún peor, más allá del esfuerzo, porque el testing no puede ser usado para probar la usencia de errores en el software sino tan solo la presencia. Por eso es necesario atacar el problema desde diferentes enfoques, cada uno teniendo sus fortalezas y ventajas. Actualmente existen muchas técnicas para hacer testing de software, y la mayoría de ellos se basan en la automatización de pasos o caminos de ejecución, con valores fijos o componentes predefinidos (hard-coded) o estáticos, y condiciones específicas. En este trabajo de maestría, se presenta un enfoque para pruebas de unidad en la programación orientada a objetos, basado en la generación de objetos de manera aleatoria. El fundamento básico de este enfoque propuesto es el testing aleatorio. También se presenta una herramienta de testing de unidad que usa el enfoque dicho, y que fue escrita en un lenguaje orientado a objetos de amplia difusión. El testing aleatorio (RT o random testing) como técnica no es nueva. Tampoco lo es la generación de valores aleatorios para pruebas. En el paradigma funcional, existe una herramienta muy conocida para probar especificaciones sobre funciones llamada QuickCheck. Ésta herramienta (escrita en Haskell) y sus ideas subyacentes son usadas como fundamento para la herramienta creada en este trabajo. La herramienta desarrollada en el presente trabajo cubre además características que existen en el paradigma orientado a objetos de manera inherente, tales como el estado de los objetos (en particular los objetos singleton con estado), clases abstractas e interfaces, que no existen en la programación funcional pura. La constribución de este trabajo de maestría es la presentación de una forma alternativa de realizar tests de unidad en la programación orientada a objetos (POO), basada en un trabajo anterior para el paradigma funcional. También se presenta una herramienta llamada YAQC4J que plasma esas ideas en un lenguaje orientado a objetos de amplia difusión. Finalmente se incluyen ejemplos que ilustran el uso de la herramienta, y se presenta una comparación con herramientas existentes que han intentado implementar el enfoque de testing. Este trabajo está dirigido a los desarrolladores de software interesados en conocer soluciones alternativas para el testing de unidad, y al mismo tiempo una forma complementaria a las ya existentes para pruebas de unidad.

In software development process, there can be misunderstandings, communication problems between individual members of the development team and mistakes when writing code or designing applications. These problems need to be resolved before the application is released into the production environment. To discover these errors it is necessary to properly test the software product. All these activities are engaged in discipline called Software Quality Assurance (SQA), which is an integral part of the software development process. The thesis discusses this discipline and procedures for software testing using the Microsoft Visual Studio 2008. The main objective of this work is to create comprehensive methodology for automated testing of web and Windows applications in Microsoft Visual Studio and to verify that procedures in practice. The reader will learn the theoretical basis of software testing at the beginning of the thesis. In next chapter, there are described various features that Microsoft Visual Studio 2008 offers for software testing. Following chapter describes the applications testing in IBM Rational Functional Tester, which is suitable for Windows applications testing (Microsoft Visual Studio 2008 does not support this kind of tests). In other part of the work -- Case Study -- the reader will learn web and Windows applications testing. Case study can serve as a basis for further methodological manual for teams engaged in software testing. The usage of instruments for testing is explained in real examples. At the end of the work there are set out recommendations for the team collaboration using Microsoft Team Foundation Server, and there are also compared the testing instruments and reviewed functions that will come up with Microsoft Visual Studio 2010.

This paper deals with the subject of refactoring usage in the development of object-oriented applications. It outlines the refactoring theory and the properties that influence the software project development. Prior to refactorization, the system has to be covered with tests. The actual refactoring can be applied during the development process as bug fixing, code review or the refactoring of the whole system. Presented in this work are the impacts of refactorization of OKsystem's company OKbase product and the evaluation with respect to the refactoring concepts of object-oriented applications.

Broadly, agile software development is an approach where code is frequently built, tested and shipped, leading to short release cycles. Extreme version is the DevOps approach where the development, testing and deployment pipelines are merged and software is continuously tested and updated. In this context our work focuses on identifying spots where the participants should be more aware of the performance and offers approaches and tools to improve their awareness with the ultimate goal of producing better software in shorter time. In general, the awareness is raised by testing, documenting, and monitoring the performance in all phases of the development cycle. In this thesis we (1) show a framework for writing performance tests for individual components (e.g. libraries). The tests capture and codify assumptions about the performance into runnable artifacts that simplify repeatability and automation. For evaluation of the performance tests we (2) propose new methods, which can automatically detect performance regressions. These methods are designed with inherent variation of performance data in mind and are able to filter it out in order to detect true regressions. Then we (3) reuse the performance tests to provide the developers with accurate and up-to-date performance API documentation that steer them...

This thesis is focused on ways of defining the requirements for performance testing software, both in practice (based on survey) as well as in theory. The aim of this thesis is to analyze ways of defining performance requirements, which are actually used on IT projects. In order to achieve this goal it is necessary to define concepts of performance testing, implement and evaluate the survey of the using ways of defining performance requirements in practice and then analyze ways of defining performance requirements in terms of their applicability to different types of IT projects. The contribution of this thesis is the comprehensive introduction performance testing software issues and mainly insight into ways of defining performance requirements used in practice and problems associated with them, through the survey, which was implemented and evaluated by myself. The conclusions resulting from this survey summarize which ways of defining performance requirements are applied to specific types of IT projects, which of these ways worked and which problems in certain cases that occur in practice. The thesis is divided into theoretical and practical part. The theoretical part explains the basic concepts associated with performance testing software. In this part, there is also described the methodology of defining performance requirements according to Tom Gilb. The practical part is focused on the realization and evaluation of survey of the using ways of defining performance requirements in practice and on the analysis ways of defining performance requirements with respect to certain types of projects.

Made available in DSpace on 2014-12-17T15:47:58Z (GMT). No. of bitstreams: 1 HeitorMAC_DISSERT.pdf: 3258229 bytes, checksum: 5f7856b140a636bd052147c58ff9dede (MD5) Previous issue date: 2011-03-01 / Coordena??o de Aperfei?oamento de Pessoal de N?vel Superior / Through the adoption of the software product line (SPL) approach, several benefits are achieved when compared to the conventional development processes that are based on creating a single software system at a time. The process of developing a SPL differs from traditional software construction, since it has two essential phases: the domain engineering - when common and variables elements of the SPL are defined and implemented; and the application engineering - when one or more applications (specific products) are derived from the reuse of artifacts created in the domain engineering. The test activity is also fundamental and aims to detect defects in the artifacts produced in SPL development. However, the characteristics of an SPL bring new challenges to this activity that must be considered. Several approaches have been recently proposed for the testing process of product lines, but they have been shown limited and have only provided general guidelines. In addition, there is also a lack of tools to support the variability management and customization of automated case tests for SPLs. In this context, this dissertation has the goal of proposing a systematic approach to software product line testing. The approach offers: (i) automated SPL test strategies to be applied in the domain and application engineering, (ii) explicit guidelines to support the implementation and reuse of automated test cases at the unit, integration and system levels in domain and application engineering; and (iii) tooling support for automating the variability management and customization of test cases. The approach is evaluated through its application in a software product line for web systems. The results of this work have shown that the proposed approach can help the developers to deal with the challenges imposed by the characteristics of SPLs during the testing process / Com o uso da abordagem de linhas de produto de software (LPSs), v?rios benef?cios s?o alcan?ados quando comparados aos processos de desenvolvimento convencionais que se baseiam na cria??o de um ?nico sistema por vez. O processo de desenvolvimento de uma LPS se diferencia da constru??o tradicional de software, uma vez que apresenta duas etapas essenciais: a engenharia de dom?nio - quando elementos comuns e vari?veis da LPS s?o definidos e implementados; e a engenharia de aplica??o quando uma ou mais aplica??es (produtos espec?ficos) s?o derivadas a partir do reuso dos artefatos criados na engenharia de dom?nio. Durante a elabora??o da LPS, assim como no desenvolvimento convencional de sistemas, a atividade de teste ? fundamental e tem como objetivo a detec??o de defeitos nos artefatos produzidos. Contudo, as caracter?sticas de uma LPS trazem novos desafios a essa atividade e que precisam ser considerados. Diversas abordagens foram propostas para o processo de teste de linhas de produto, mas elas se mostram limitadas ou fornecem diretrizes muito gerais. Outro fator preocupante ? a escassez de ferramentas que auxiliem na implementa??o, aplica??o e acompanhamento dos testes, bem como na ger?ncia e customiza??o de tais artefatos. Com base nesse contexto relacionado ao processo de teste de LPSs, esta disserta??o tem como objetivo propor uma abordagem sistem?tica para o teste de linhas de produto de software. A abordagem oferece: (i) estrat?gias de testes automatizados para LPSs tanto na engenharia de dom?nio quanto de aplica??o; (ii) diretrizes para a implementa??o e reuso de casos de teste automatizados nos n?veis de unidade, integra??o e sistema tanto para a engenharia de dom?nio quanto de aplica??o; e (iii) suporte ferramental para ger?ncia e customiza??o autom?tica de casos de teste usando t?cnicas de deriva??o autom?tica de software. A abordagem ? avaliada atrav?s da sua aplica??o em uma linha de produto para sistemas web. Os resultados deste trabalho mostram que a abordagem proposta pode ajudar os desenvolvedores a lidar com os desafios impostos pelas caracter?sticas das LPSs durante o processo de testes