No description available.

Programinės įrangos testavimas ir kokybės užtikrinimas yra svarbus programų sistemų inžinerijos kūrimo uždavinys, siekiant sukurti tinkamą naudojimui produktą. Yra daug skirtingų metodikų kuriamai programinei įrangai testuoti, tačiau vieningos sistemos, kuri būtų universali – nėra. Įvairūs tyrimai vykdomi programinės įrangos testavimo srityje duoda skirtingus rezultatus. Testavimo procesas taip pat svarbus ir praktikoje – be jo negali išsiversti nei vienas organizacija susijusi su programinės įrangos kūrimu ir plėtojimu. Šis darbas remiasi modeliu paremto testavimo paradigma ir simboliniu vykdymo metodika. Darbe apžvelgiamos teorinės simbolinio vykdymo galimybės, jo pritaikymas .Net platformoje ir papildomos priemonės, kurios reikalingos įgyvendinti tokią sistemą. Taip pat trumpai pristatomas magistro projektinis darbas, aprašomi sukurti inžinerinio produkto svarbiausi aspektai. Pagal teorinę medžiaga sukurtas simbolinio vykdymo variklis – Symex. Darbe nagrinėjamas praktinis tokio įrankio pritaikymas generuojant vienetų testus iš išeities kodo – eksperimentiškai tiriamos ir lyginamos simbolinio vykdymo ir atsitiktinių įėjimų vienetų testų kūrimo galimybės .Net platformoje. / Testing complex safety critical software always was difficult task. Development of automated techniques for error detection is even more difficult. Well known techniques for checking software are model checking static analysis and testing. Symbolic execution is a technique that is being used to improve security, to find bugs, and to help in debugging. A symbolic execution engine is basically an interpreter that figures out how to follow all paths in a program. It is a static code analysis technique. This work presents symbolic execution background, current state, analysis the possibilities of implementation on the .Net framework and platform. The work describes the master project – bug tracking software “Crunchbug” and the tool – Symex (symbolic execution engine) for .Net platform. Symex is white box model based automatic unit test generator and it is evaluated against two other tools – Microsoft Pex and framework that generates unit test inputs random. Detailed experiments made to cover symbolic execution possibilities with proprietary benchmarks and real code from the master project.

Because software testing is a repetitive and time-intensive task, a practical solution is to turn to automation. Test automation, however, requires programming skills. Testers, who typically know a lot about the application under test, often do not have the programming skills to automate the testing effort. Grammar-based test generation (GBTG) uses context-free grammars to generate strings in the language described by the grammar. Given a grammar, a GBTG algorithm can produce test cases for the application under test. Since testers typically have little programming skills and are not likely to develop the grammars needed for practical testing, the power of GBTG is unavailable to many testing practitioners. To help address this problem, we have developed Dervish, a graphical user interface which allows testers to use the power of GBTG. Our new tool allows testers to modify parts of a grammar, generate test cases, and visualize generation trees. To demonstrate the benefits of Dervish, we present the results of three case studies.

Les architectures des systèmes sur puce (System-on-Chip, SoC) actuelles intègrent de nombreux composants différents tels que les processeurs, les accélérateurs, les mémoires et les blocs d'entrée/sortie, certains pouvant contenir des caches. Vu que l'effort de validation basée sur la simulation, actuellement utilisée dans l'industrie, croît de façon exponentielle avec la complexité des SoCs, nous nous intéressons à des techniques de vérification formelle. Nous utilisons la boîte à outils CADP pour développer et valider un modèle formel d'un SoC générique conforme à la spécification AMBA 4 ACE récemment proposée par ARM dans le but de mettre en œuvre la cohérence de cache au niveau système. Nous utilisons une spécification orientée contraintes pour modéliser les exigences générales de cette spécification. Les propriétés du système sont vérifié à la fois sur le modèle avec contraintes et le modèle sans contraintes pour détecter les cas intéressants pour la cohérence de cache. La paramétrisation du modèle proposé a permis de produire l'ensemble complet des contre-exemples qui ne satisfont pas une certaine propriété dans le modèle non contraint. Notre approche améliore les techniques industrielles de vérification basées sur la simulation en deux aspects. D'une part, nous suggérons l'utilisation du modèle formel pour évaluer la bonne construction d'une unité de vérification d'interface. D'autre part, dans l'objectif de générer des cas de test semi-dirigés intelligents à partir des propriétés de logique temporelle, nous proposons une approche en deux étapes. La première étape consiste à générer des cas de tests abstraits au niveau système en utilisant des outils de test basé sur modèle de la boîte à outils CADP. La seconde étape consiste à affiner ces tests en cas de tests concrets au niveau de l'interface qui peuvent être exécutés en RTL grâce aux services d'un outil commercial de génération de tests dirigés par les mesures de couverture. Nous avons constaté que notre approche participe dans la transition entre la vérification du niveau interface, classiquement pratiquée dans l'industrie du matériel, et la vérification au niveau système. Notre approche facilite aussi la validation des propriétés globales du système, et permet une détection précoce des bugs, tant dans le SoC que dans les bancs de test commerciales. / State-of-the-art System-on-Chip (SoC) architectures integrate many different components, such as processors, accelerators, memories, and I/O blocks. Some of those components, but not all, may have caches. Because the effort of validation with simulation-based techniques, currently used in industry, grows exponentially with the complexity of the SoC, this thesis investigates the use of formal verification techniques in this context. More precisely, we use the CADP toolbox to develop and validate a generic formal model of a heterogeneous cache-coherent SoC compliant with the recent AMBA 4 ACE specification proposed by ARM. We use a constraint-oriented specification style to model the general requirements of the specification. We verify system properties on both the constrained and unconstrained model to detect the cache coherency corner cases. We take advantage of the parametrization of the proposed model to produce a comprehensive set of counterexamples of non-satisfied properties in the unconstrained model. The results of formal verification are then used to improve the industrial simulation-based verification techniques in two aspects. On the one hand, we suggest using the formal model to assess the sanity of an interface verification unit. On the other hand, in order to generate clever semi-directed test cases from temporal logic properties, we propose a two-step approach. One step consists in generating system-level abstract test cases using model-based testing tools of the CADP toolbox. The other step consists in refining those tests into interface-level concrete test cases that can be executed at RTL level with a commercial Coverage-Directed Test Generation tool. We found that our approach helps in the transition between interface-level and system-level verification, facilitates the validation of system-level properties, and enables early detection of bugs in both the SoC and the commercial test-bench.

Le frelatage (ou fuzzing) est l'une des approches les plus efficaces pour la détection de vulnérabilités dans les logiciels de tailles importantes et dont le code source n'est pas disponible. Malgré une utilisation très répandue dans l'industrie, les techniques de frelatage "classique" peuvent avoir des résultats assez limités, et pas toujours probants. Ceci est dû notamment à une faible couverture des programmes testés, ce qui entraîne une augmentation du nombre de faux-négatifs; et un manque de connaissances sur le fonctionnement interne de la cible, ce qui limite la qualité des entrées générées. Nous présentons dans ce travail une approche automatique de recherche de vulnérabilités logicielles par des processus de test combinant analyses avancées de code binaire et frelatage. Cette approche comprend : une technique de minimisation de suite de tests, pour optimiser le rapport entre la quantité de code testé et le temps d'exécution ; une technique d'analyse de couverture optimisée et rapide, pour évaluer l'efficacité du frelatage ; une technique d'analyse statique, pour localiser les séquences de codes potentiellement sensibles par rapport à des patrons de vulnérabilités; une technique dynamique d'analyse de teinte, pour identifier avec précision les zones de l'entrée qui peuvent être à l'origine de déclenchements de vulnérabilités; et finalement une technique évolutionniste de génération de test qui s'appuie sur les résultats des autres analyses, afin d'affiner les critères de décision et d'améliorer la qualité des entrées produites. Cette approche a été mise en œuvre à travers une chaîne d'outils intégrés et évalués sur de nombreuses études de cas fournies par l'entreprise. Les résultats obtenus montrent son efficacité dans la détection automatique de vulnérabilités affectant des applications majeures et sans accès au code source. / Fuzz testing (a.k.a. fuzzing) is one of the most effective approaches for discovering security vulnerabilities in large and closed-source software. Despite their wide use in the software industry, traditional fuzzing techniques suffer from a poor coverage, which results in a large number of false negatives. The other common drawback is the lack of knowledge about the application internals. This limits their ability to generate high quality inputs. Thus such techniques have limited fault detection capabilities. We present an automated smart fuzzing approach which combines advanced binary code analysis techniques. Our approach has five components. A test suite reduction technique, to optimize the ratio between the amount of covered code and the execution time. A fast and optimized code coverage measurement technique, to evaluate the fuzzing effectiveness. A static analysis technique, to locate potentially sensitive sequences of code with respect to vulnerability patterns. An origin-aware dynamic taint analysis technique, to precisely identify the input fields that may trigger potential vulnerabilities. Finally, an evolutionary based test generation technique, to produce relevant inputs. We implemented our approach as an integrated tool chain, and we evaluated it on numerous industrial case studies. The obtained results demonstrate its effectiveness in automatically discovering zero-day vulnerabilities in major closed-source applications. Our approach is relevant to both defensive and offensive security purposes.

Diversas pesquisas apoiam e investigam o teste de programas concorrentes, as quais objetivam, principalmente, a proposição de critérios de teste e mecanismos para execução das diferentes sincronizações entre processos ou threads. As características específicas dessas aplicações podem ocasionar diferentes tipos de defeitos, os quais, em sua maioria, não são facilmente identificados. Nesse contexto, a geração automática de dados de teste pode apoiar a atividade de teste atuando na seleção de entradas mais representativas, ou seja, aquelas com maior probabilidade de revelar defeitos. Apesar disso, poucas pesquisas abordam este tema no contexto de programas concorrentes, e as existentes não consideram aspectos importantes desse tipo de aplicação. A geração de dados de teste para programas sequenciais dispõe de uma variedade de técnicas que apoiam a seleção dos dados de teste. Essas técnicas têm sido estendidas para o contexto de programas concorrentes partindo da premissa que esses programas necessitam de abordagens mais complexas para seleção de entradas, em decorrência disso um maior custo é imposto ao teste. Considerando esse contexto, uma lacuna ainda em aberto é a avaliação das técnicas para o cenário de programas concorrentes. Neste trabalho a avaliação das técnicas foi explorada por meio da realização de estudos experimentais, os quais avaliaram diferentes técnicas de geração de dados de teste para o contexto de programas concorrentes, considerando a eficácia em revelar defeitos, cobertura de critérios e custo para atividade de teste. Os resultados obtidos demonstraram que as técnicas empregadas para programas sequenciais não atingem o mínimo esperado para este tipo aplicação. Apesar disso, as técnicas investigadas apresentaram características importantes que podem auxiliar a atividade de teste para programas concorrentes e a proposição de abordagens efetivas de geração de dados para esse contexto. / The concurrent program testing has been largely investigated with propositions of testing criteria and mechanisms, which aim mainly to testing criteria proposition and mechanisms for execution of different synchronizations. The specific characteristics of these applications can lead to the different types of faults, which, in most of cases, are not easily identified. In this context, the automatic test data generation can support the testing activity acting in selecting the most representative data tests, i.e. those most likely to reveal faults. Nevertheless, few studies address this issue in the context of concurrent programs, and the these studies do not consider important aspects of this type of application. In contrast, we can find several techniques proposed to support the test data generation for sequential programs. These techniques have been extended to the context of concurrent programs on the premise that these programs require more complex approaches for selection of test data. As a result, a greater cost for testing activity is enforced. Considering this context, a gap still open is the evaluation of techniques for the scenario of concurrent programs. In this work the evaluation of techniques was explored through experimental studies, which different techniques of test data generatoon were evaluated, considering effectiveness, testing coverage and application cost. The results showed that the generation techniques used for sequential programs used in the experimental study do not reach the minimum expected in terms of effectiveness and cost for concurrent programs. Nevertheless, the techniques investigated showed significant features that can help the proposition of effective approaches for test data generation applied to concurrent programs.

Made available in DSpace on 2014-06-12T15:57:23Z (GMT). No. of bitstreams: 2 arquivo3187_1.pdf: 2434276 bytes, checksum: df6b126c4802eed8524aba0d3cb25af9 (MD5) license.txt: 1748 bytes, checksum: 8a4605be74aa9ea9d79846c1fba20a33 (MD5) Previous issue date: 2009 / Boudox Peres, Glaucia; Cabral Mota, Alexandre. A black-box testing technique for the detection of crashes based on automated test scenarios. 2009. Dissertação (Mestrado). Programa de Pós-Graduação em Ciência da Computação, Universidade Federal de Pernambuco, Recife, 2009.

L'objectif de cette thèse est l'assistance à la génération de tests et à l'animation de spécifications orientées objet. Nous cherchons en particulier à profiter de l'état de l'art des techniques de résolution de satisfaisabilité en utilisant une représentation appropriée des données orientées objet. Alors que la génération automatique de cas de tests recherche un large ensemble de valeurs à fournir en entrée d'une application, l’animation de spécifications effectue les calculs qui sont conformes à une spécification à partir de valeurs fournies par l'utilisateur. L'animation est une technique importante pour la validation des spécifications.Comme fondement de ce travail, nous présentons des clarifications et une formalisation partielle du langage de spécification OCL (Object Constraint Language) ainsi que quelques extensions, afin de permettre la génération de tests et l'animation à partir de spécifications OCL.Pour la génération de tests, nous avons implémenté plusieurs améliorations à HOL-TestGen, outil basé sur le démonstrateur de théorème Isabelle, qui engendre des tests à partir de spécifications en Logique d’Ordre Supérieure (Higher-Order Logic ou HOL). Nous montrons comment des solveurs SMT peuvent être utilisés pour résoudre différents types de contraintes en HOL et nous présentons une approche modulaire de raisonnement par cas pour dériver des cas de tests. Cette dernière approche facilite l'introduction de règles de decomposition par cas qui sont adaptées aux spécifications orientées objet.Pour l'animation de spécifications, nous avons développé OCLexec, outil d'animation de spécifications en OCL. A partir de contrats de fonctions OCLexec produit les implémentations Java correspondantes qui appellent un solveur de contraintes SMT lors de leur exécution. / The goal of this thesis is the development of support for test generation and animation based on object-oriented specifications. We aim particularly to take advantage of state-of-the-art satisfiability solving techniques by using an appropriate representation of object-oriented data. While automated test generation seeks a large set of data to execute an implementation on, animation performs computations that comply with a specification based on user-provided input data. Animation is a valuable technique for validating specifications.As a foundation of this work, we present clarifications and a partial formalization of the Object Constraint Language (OCL) as well as some extensions in order to allow for test generation and animation based on OCL specifications.For test generation, we have implemented several enhancements to HOL-TestGen, a tool built on top of the Isabelle theorem proving system that generates tests from specifications in Higher-Order Logic (HOL). We show how SMT solvers can be used to solve various types of constraints in HOL and present a modular approach to case splitting for deriving test cases. The latter facilitates the introduction of splitting rules that are tailored to object-oriented specifications.For animation, we implemented the tool OCLexec for animating OCL specifications. OCLexec generates from operation contracts corresponding Java implementations that call an SMT-based constraint solver at runtime.

In the development process of digital circuits, it is often not possible to avoid introducing errors into systems that are being developed. Early detection of such errors saves money and time. This project deals with automation of feedback in functional verification of various data processing components. The goal of automatic feedback is not only to shorten the time needed to verify the functionality of a system, but mainly to improve verification coverage of corner cases and thus increase the confidence in the verified system. General functional and formal verification principles and practices are discussed, coverage metrics are presented, limitations of both techniques are mentioned and room for improvement of current status is identified. Design of feedback verification environment using a genetic algorithm is described in detial. The verification results are summarized and evaluated.

Achieving high software quality today involves manual analysis, test planning, documentation of testing strategy and test cases, and development of automated test scripts to support regression testing. This thesis is motivated by the opportunity to bridge the gap between current test automation and true test automation by investigating learning-based solutions to software testing. We present an approach that combines a trainable web component classifier, a test case description language, and a trainable test generation and execution system that can learn to generate new test cases. Training data was collected and hand-labeled across 7 systems, 95 web pages, and 17,360 elements. A total of 250 test flows were also manually hand-crafted for training purposes. Various machine learning algorithms were evaluated. Results showed that Random Forest classifiers performed well on several web component classification problems. In addition, Long Short-Term Memory neural networks were able to model and generate new valid test flows.