Contexto: Estudantes de cursos da área de Ciência da Computação que estão iniciando em programação normalmente têm dificuldade de assimilar os conteúdos básicos. Alguns pesquisadores propõem a utilização do desenvolvimento baseado em testes (TDD) a fim de melhorar o ensino, mas sem o foco na qualidade do conjunto de casos de teste. Objetivo: Este trabalho tem o objetivo de propor uma estratégia baseada no desenvolvimento baseado por testes em conjunto com técnicas de teste de software, guiando o aluno a desempenhar suas atividades de programação com mais qualidade e, desta forma, obter melhores resultados acadêmicos. Método: Baseado em um mapeamento sistemático da literatura e na revisão de estudos que aplicam desenvolvimento baseado em testes (TDD) em âmbito educacional, foi definida uma estratégia para integrar critérios de teste ao TDD. Para avaliar os resultados, foram definidas medidas e instrumentos para a coleta de resultados. Foi executado um experimento científico com o público alvo para a obtenção das informações com a finalidade de avaliar a hipótese definida. Resultados: Foi definida uma estratégia de utilização de técnicas de teste de software em conjunto com o TDD para alunos que estão iniciando em programação em cursos de Computação. A estratégia consiste na melhoria do conjunto de casos de teste durante a refatoração no TDD, considerando critérios estruturais baseado em fluxo de controle. Além disso, foram elaboradas ferramentas para a coleta de dados durante o desenvolvimento de programas e para análise dos dados. A estratégia permitiu a melhora do programa e dos casos de teste e foi avaliada pelos estudantes como um guia no desenvolvimento dos casos de teste e como um atalho para descoberta de defeitos. Conclusões: Com a utilização da estratégia proposta, foi possível observar melhorias no conjunto de casos de teste e no software final. A estratégia definida pode ser replicada em outros contextos e com o emprego de outras técnicas e critérios de teste de software para a avaliação da hipótese definida e generalização dos resultados. / Background: Students of Computer Science courses usually have difficulty assimilating the basic contents of introductory programming disciplines. Some researchers propose the use of Test-Driven Development (TDD) to improve teaching, but with no focus in the test set quality. Objective: The goal of this study is to define a strategy for test-driven development using software testing criteria, guiding students to improve the quality of their programming activities, and, consequently, academic results. Method: Initially, we performed a systematic mapping study on TDD in the Computing education context, and a literature review about the usage of test criteria with TDD. Considering the evidences collected, we defined a strategy to integrate the usage of test criteria into TDD. In order to evaluate this strategy, we defined measurements and instruments for data collection. An experimental study was conducted with students to evaluate the strategy and the objective satisfaction. Results: We defined a strategy to use software testing criteria with TDD for students that are learning how to program in Computing courses. The strategy consists into improving the test set while refactoring the code at TDD, considering control-flow-based structural testing criteria. Morever, we developed a set of tools to gather and analyze data produced by the students while using the strategy in the experimental study. The strategy lead to higher quality programs and test sets, and it was considered useful by the students, providing a guide for the creation of test cases, and helping them to discover errors. Conclusions: It was possible to observe improvements into the quality of programs and of test sets. The strategy can be replicated, considering different contexts and evaluting more testing criteria, fostering further evaluation of the objective and generalization of the results.

No description available.

The rapid advancement of web and mobile application technologies has recently posed numerous challenges to the Software Engineering community, including how to cost-effectively test applications that have complex event spaces. Many software testing techniques attempt to cost-effectively improve the quality of such software. This dissertation primarily focuses on that of hybrid test suite prioritization. The techniques utilize two or more criteria to perform test suite prioritization as it is often insufficient to use only a single criterion. The dissertation consists of the following contributions: (1) a weighted test suite prioritization technique that employs the distance between criteria as a weighting factor, (2) a coarse-to-fine grained test suite prioritization technique that uses a multilevel approach to increase the granularity of the criteria at each subsequent iteration, (3) the Caret-HM tool for Android user session-based testing that allows testers to record, replay, and create heat maps from user interactions with Android applications via a web browser, and (4) Android user session-based test suite prioritization techniques that utilize heuristics developed from user sessions created by Caret-HM. Each of the chapters empirically evaluate the respective techniques. The proposed techniques generally show improved or equally good performance when compared to the baselines, depending on an application under test. Further, this dissertation provides guidance to testers as it relates to the use of the proposed hybrid techniques.

Developing countries like Tanzania experience challenges towards utilization and acceptance of ICT; calling for a need to further research on the concept. Open Source (OS) usage is a potential strategy for addressing such challenges. However, the success of this strategy strongly relies on the strength of the promotional efforts. The study, therefore aims at assessing the OS promotional efforts in relation to ICT acceptance challenges in Tanzania. This study entailed a descriptive, mixed-methods research. A literature analysis, document analysis and observations of OS community activities were conducted in order to list the ICT acceptance challenges. The results formed a basis for survey and interview questions. The findings obtained were triangulated to determine the existing OS promotional activities and assess the effectiveness of the promotional efforts in addressing ICT acceptance challenges in Tanzania. The study also makes recommendations on how OS promotional efforts should be changed to improve their effectiveness. / Computing / (M.Sc. (Information Systems))

Hardware/Software (HW/SW) interfaces are pervasive in modern computer systems. Most of HW/SW interfaces are implemented by devices and their device drivers. Unfortunately, HW/SW interfaces are unreliable and insecure due to their intrinsic complexity and error-prone nature. Moreover, assuring HW/SW interface reliability and security is challenging. First, at the post-silicon validation stage, HW/SW integration validation is largely an ad-hoc and time-consuming process. Second, at the system deployment stage, transient hardware failures and malicious attacks make HW/SW interfaces vulnerable even after intensive testing and validation. In this dissertation, we present a comprehensive solution for HW/SW interface assurance over the system life cycle. This solution is composited of two major parts. First, our solution provides a systematic HW/SW co-validation framework which validates hardware and software together; Second, based on the co-validation framework, we design two schemes for assuring HW/SW interfaces over the system life cycle: (1) post-silicon HW/SW co-validation at the post-silicon validation stage; (2) HW/SW co-monitoring at the system deployment stage. Our HW/SW co-validation framework employs a key technique, conformance checking which checks the interface conformance between the device and its reference model. Furthermore, property checking is carried out to verify system properties over the interactions between the reference model and the driver. Based on the conformance between the reference model and the device, properties hold on the reference model/driver interface also hold on the device/driver interface. Conformance checking discovers inconsistencies between the device and its reference model thereby validating device interface implementations of both sides. Property checking detects both device and driver violations of HW/SW interface protocols. By detecting device and driver errors, our co-validation approach provides a systematic and ecient way to validate HW/SW interfaces. We developed two software tools which implement the two assurance schemes: DCC (Device Conformance Checker), a co-validation framework for post-silicon HW/SW integration validation; and CoMon (HW/SW Co-monitoring), a runtime verication framework for detecting bugs and malicious attacks across HW/SW interfaces. The two software tools lead to discovery of 42 bugs from four industry hardware devices, the device drivers, and their reference models. The results have demonstrated the signicance of our approach in HW/SW interface assurance of industry applications.

Software Maintenance is becoming more and more challenging due to rapidly changing customer needs, technologies and need for highly skilled labor. Many problems that existed a decade ago continue to exist or have even grown. In this context organizations find it difficult to match engineer interest, skill to particular customer problem. Thus making it difficult for organization to keep the selfish and rational engineers motivated and productive. In this thesis we have used game theory and mechanism design to model the interactions among such selfish engineers to motivate truth revelation using incentive based allocation schemes for software maintenance problems, especially Ticket Allocation Problem. Ticket allocation or problem allocation is a key problem in the software maintenance process.Tickets are usually allocated by the manager or the technical lead. In allocating a ticket, the manager or technical lead is normally guided by the complexity assessment of the ticket as provided by the maintenance engineers, who are entrusted with the responsibility of fixing the problem.The rationality of the maintenance engineers could induce them to report the complexity in an untruthfulway so as to increase their payoffs.This leads to non-optimal ticket allocation. In this thesis we first address the problem of eliciting ticket complexities in a truthfulway from each individual maintenance engineer, using a mechanism design approach. In particular, we model the problem as that of designing an incentive compatible mechanism and we offer two possible solutions.The first one, TA-DSIC, a Dominant Strategy Incentive Compatible (DSIC) solution and the second solution, TA-BIC, is a Bayesian Incentive Compatible mechanism. We show that the proposed mechanisms outperform conventional allocation protocols in the context of a representative software maintenance organization. In this thesis,we next address the incentive compatibility issue for group ticket allocation problem .Many times a ticket is also allocated to more than one engineers. This may be due to a quick customer delivery(time)deadline. The decision of such allocation is generally taken by the lead, based on customer deadlines and a guided complexity assessment from each maintenance engineer.The decision of allocation in such case should ensure that every individual reveals truth in the proposed group(or coalition) and has incentive to participate in the game as individual and in the coalition. We formulate this problem as Normal form game and propose three mechanisms, (1)Division of Labor, (2)Extended Second Price and (3)Greedy Division of Labor. We show that the proposed mechanisms are DSIC and we discuss their rationality properties.

This dissertation attacks the well-known problem of path-imprecision in static program analysis. Our starting point is an existing static program analysis that over-approximates the execution paths of the analyzed program. We then make this over-approximating program analysis more precise for automatic testing in an object-oriented programming language. We achieve this by combining the over-approximating program analysis with usage-observing and under-approximating analyses. More specifically, we make the following contributions. We present a technique to eliminate language-level unsound bug warnings produced by an execution-path-over-approximating analysis for object-oriented programs that is based on the weakest precondition calculus. Our technique post-processes the results of the over-approximating analysis by solving the produced constraint systems and generating and executing concrete test-cases that satisfy the given constraint systems. Only test-cases that confirm the results of the over-approximating static analysis are presented to the user. This technique has the important side-benefit of making the results of a weakest-precondition based static analysis easier to understand for human consumers. We show examples from our experiments that visually demonstrate the difference between hundreds of complicated constraints and a simple corresponding JUnit test-case. Besides eliminating language-level unsound bug warnings, we present an additional technique that also addresses user-level unsound bug warnings. This technique pre-processes the testee with a dynamic analysis that takes advantage of actual user data. It annotates the testee with the knowledge obtained from this pre-processing step and thereby provides guidance for the over-approximating analysis. We also present an improvement to dynamic invariant detection for object-oriented programming languages. Previous approaches do not take behavioral subtyping into account and therefore may produce inconsistent results, which can throw off automated analyses such as the ones we are performing for bug-finding. Finally, we address the problem of unwanted dependencies between test-cases caused by global state. We present two techniques for efficiently re-initializing global state between test-case executions and discuss their trade-offs. We have implemented the above techniques in the JCrasher, Check 'n' Crash, and DSD-Crasher tools and present initial experience in using them for automated bug finding in real-world Java programs.

Orientador: Mario Jino, Plinio de Sa Leitão Junior / Dissertação (mestrado) - Universidade Estadual de Campinas, Faculdade de Engenharia Eletrica e de Computação / Made available in DSpace on 2018-08-15T03:59:29Z (GMT). No. of bitstreams: 1 Cabeca_AndreaGoncalves_M.pdf: 8778522 bytes, checksum: c968246a4fb6a8fb41b47192a1d8cb15 (MD5) Previous issue date: 2009 / Resumo: O teste de aplicações de banco de dados é crucial para assegurar a alta qualidade do software, pois defeitos não detectados podem resultar em corrupção irrecuperável dos dados. SQL é a mais amplamente utilizada interface para sistemas de banco de dados. Nossa abordagem visa a alcançar testes efetivos pela seleção de bases de dados reveladoras de defeitos. Usamos a análise de mutantes em comandos SQL e discutimos dois cenários para aplicar as técnicas de mutação forte e fraca. Uma ferramenta para auxiliar na automatização da técnica foi desenvolvida e implementada. Experimentos usando aplicações reais, defeitos reais e dados reais foram conduzidos para: (i) avaliar a aplicabilidade da abordagem; e (ii) comparar bases de dados de entrada quanto à habilidade para detectar defeitos / Abstract: Testing database applications is crucial for ensuring high quality software as undetected faults can result in unrecoverable data corruption. SQL is the most widely used interface language for relational database systems. Our approach aims to achieve better tests by selecting fault-revealing databases. We use mutation analysis on SQL statements and discuss two scenarios for applying strong and weak mutation techniques. A tool to support the automatization of the technique has been developed and implemented. Experiments using real applications, real faults and real data were performed to: (i) evaluate the applicability of the approach, and (ii) compare fault-revealing abilities of input databases / Mestrado / Engenharia de Computação / Mestre em Engenharia Elétrica

Developing countries like Tanzania experience challenges towards utilization and acceptance of ICT; calling for a need to further research on the concept. Open Source (OS) usage is a potential strategy for addressing such challenges. However, the success of this strategy strongly relies on the strength of the promotional efforts. The study, therefore aims at assessing the OS promotional efforts in relation to ICT acceptance challenges in Tanzania. This study entailed a descriptive, mixed-methods research. A literature analysis, document analysis and observations of OS community activities were conducted in order to list the ICT acceptance challenges. The results formed a basis for survey and interview questions. The findings obtained were triangulated to determine the existing OS promotional activities and assess the effectiveness of the promotional efforts in addressing ICT acceptance challenges in Tanzania. The study also makes recommendations on how OS promotional efforts should be changed to improve their effectiveness. / Computing / M.Sc. (Information Systems)

Nowadays, deep neural networks based software have been widely applied in many areas including safety-critical areas such as traffic control, medical diagnosis and malware detection, etc. However, the software engineering techniques, which are supposed to guarantee the functionality, safety as well as fairness, are not well studied. For example, some serious crashes of DNN based autonomous cars have been reported. These crashes could have been avoided if these DNN based software were well tested. Traditional software testing, debugging or repairing techniques do not work well on DNN based software because there is no control flow, data flow or AST(Abstract Syntax Tree) in deep neural networks. Proposing software engineering techniques targeted on DNN based software are imperative. In this thesis, we first introduced the development of SE(Software Engineering) for AI(Artificial Intelligence) area and how our works have influenced the advancement of this new area. Then we summarized related works and some important concepts in SE for AI area. Finally, we discussed four important works of ours. Our first project DeepTest is one of the first few papers proposing systematic software testing techniques for DNN based software. We proposed neuron coverage guided image synthesis techniques for DNN based autonomous cars and leveraged domain specific metamorphic relation to generate oracle for new generated test cases to automatically test DNN based software. We applied DeepTest to testing three top performing self-driving car models in Udacity self-driving car challenge and our tool has identified thousands of erroneous behaviors that may lead to potential fatal crash. In DeepTest project, we found that the natural variation such as spatial transformations or rain/fog effects have led to problematic corner cases for DNN based self-driving cars. In the follow-up project DeepRobust, we studied per-point robustness of deep neural network under natural variation. We found that for a DNN model, some specific weak points are more likely to cause erroneous outputs than others under natural variation. We proposed a white-box approach and a black-box approach to identify these weak data points. We implemented and evaluated our approaches on 9 DNN based image classifiers and 3 DNN based self-driving car models. Our approaches can successfully detect weak points with good precision and recall for both DNN based image classifiers and self-driving cars. Most of existing works in SE for AI area including our DeepTest and DeepRobust focus on instance-wise errors, which are single inputs that result in a DNN model's erroneous outputs. Different from instance-wise errors, group-level errors reflect a DNN model's weak performance on differentiating among certain classes or inconsistent performance across classes. This type of errors is very concerning since it has been found to be related to many real-world notorious errors without malicious attackers. In our third project DeepInspect, we first introduced the group-level errors for DNN based software and categorized them into confusion errors and bias errors based on real-world reports. Then we proposed neuron coverage based distance metric to detect group-level errors for DNN based software without requiring labels. We applied DeepInspect to testing 8 pretrained DNN models trained in 6 popular image classification datasets, including three adversarial trained models. We showed that DeepInspect can successfully detect group-level violations for both single-label and multi-label classification models with high precision. As a follow-up and more challenging research project, we proposed five WR(weighted regularization) techniques to repair group-level errors for DNN based software. These five different weighted regularization techniques function at different stages of retraining or inference of DNNs including input phase, layer phase, loss phase and output phase. We compared and evaluated these five different WR techniques in both single-label and multi-label classifications including five combinations of four DNN architectures on four datasets. We showed that WR can effectively fix confusion and bias errors and these methods all have their pros, cons and applicable scenario. All our four projects discussed in this thesis have solved important problems in ensuring the functionality, safety as well as fairness for DNN based software and had significant influence in the advancement of SE for AI area.