No description available.

No description available.

The use of clinical decision support (CDS) systems has grown progressively during the past decades. CDS systems are associated with improved patient safety and outcomes, better prescription and diagnosing practices by clinicians and lower healthcare costs. Quality assurance of these systems is critical, given the potentially severe consequences of any errors. Yet, after several decades of research, there is still no consensual or standardized approach to their verification and validation (V&V). This project is a descriptive and exploratory case study aiming to provide a practical description of how Cambio CDS, a market-leading developer of CDS services, conducts its V&V process. Qualitative methods including semi-structured interviews and coding-based textual data analysis were used to elicit the description of the V&V approaches used by the company. The results showed that the company’s V&V methodology is strongly influenced by the company’s model-driven development approach, a strong focus and leveraging of domain knowledge and good testing practices with a focus on automation and test-driven development. A few suggestions for future directions were discussed.

<p> </p> <p>Distributed agreement-based systems use common distributed agreement protocols such as leader election and consensus as building blocks for their target functionality—processes in these systems may need to agree on a leader, on the members of a group, on owners of locks, or on updates to replicated data. Such distributed agreement-based systems are common and potentially permit modular, scalable verification approaches that mimic their modular design. Interestingly, while there are many verification efforts that target agreement protocols themselves, little attention has been given to distributed agreement-based systems that build on top of these protocols. </p> <p>In this work, we aim to develop a fully-automated, modular, and usable parameterized verification approach for distributed agreement-based systems. To do so, we need to overcome the following challenges. First, the fully automated parameterized verification problem, i.e, the problem of algorithmically checking if the system is correct for any number of processes, is a well-known <em>undecidable </em>problem. Second, to enable modular verification that leverages the inherently-modular nature of these agreement-based systems, we need to be able to support <em>abstractions </em>of agreement protocols. Such abstractions can replace the agreement protocols’ implementations when verifying the overall system; enabling modular reasoning. Finally, even when the verification is fully automated, a system designer still needs assistance in <em>modeling </em>their distributed agreement-based systems. </p> <p>We systematically tackle these challenges through the following contributions. </p> <p>First, we support efficient, decidable verification of distributed agreement-based systems by developing a computational model—the GSP model—for reasoning about distributed (agreement-based) systems that admits decidability and <em>cutoff </em>results. Cutoff results enable practical verification by reducing the parameterized verification problem to the verification problem of a system with a fixed, finite number of processes. The GSP model supports generalized communication primitives and global guards, both of which are essential to enable abstractions of agreement protocols. </p> <p>Then, we address the usability and modularity aspects by developing a framework, QuickSilver, tailored for modeling and modular parameterized verification of distributed agreement-based systems. QuickSilver provides an intuitive domain-specific language, called Mercury, that is equipped with two agreement primitives capable of abstracting away agreement protocols when modeling agreement-based systems; enabling modular verification. QuickSilver extends the decidability and cutoff results of the GSP model to provide fully automated, efficient parameterized verification for a large class of systems modeled in Mercury. </p> <p>Finally, we leverage synthesis techniques to further enhance the usability of our approach and propose Cinnabar, a tool that supports synthesis of distributed agreement-based systems with efficiently-decidable parameterized verification. Cinnabar allows a system de- signer to provide a sketch of their Mercury model and uses a counterexample-guided synthesis procedure to search for model completions that both belong to the efficiently-decidable fragment of Mercury and are correct. </p> <p>We evaluate our contributions on various interesting distributed agreement-based systems adapted from real-world applications, such as a data store, a lock service, a surveillance system, a pathfinding algorithm for mobile robots, and more. </p>

An emulator is a computer program that mimics the behavior of a production facility as seen by the control program. Emulation has been used as a tool for dynamic, off-line testing of control software for automated manufacturing systems. However, research efforts in emulation have focused mainly on controllers for equipment, AGVS, and workstations. This research focuses on emulation for testing shop control software. Though conceptually simple, emulator development efforts tend to be ad hoc in nature and lack a strong conceptual framework. Currently, the effort involved in developing an emulation model may outweigh the potential benefits. The approach used in this research centers around adapting a detailed simulation model, i.e., used for testing control strategies, for emulation, i.e., control software testing. This approach promotes software reuse and thus limits the emulation development task. Due to the limitations of conventional simulation languages and modeling techniques, a simulation model is not readily adapted for emulation. The main problems lie in turning off the control logic in a simulation and interfacing the model to the actual controller. An object-oriented modeling methodology was developed for systematically transitioning from simulation to emulation. Basically, the method calis for encapsulating manufacturing control logic into controller objects and modeling the system from the perspective of exchange of messages among controllers. The developed method also promotes the rapid development of a driver for verification and validation of the emulation model. A prototype system was developed to demonstrate the feasibility of the emulation development method. The MODSIM II language for object-oriented simulation was used to implement the object classes. The second issue addressed is the ability to use emulation to test shop control system in faster-than-real-time (FRT) mode. Currently it is necessary to test the system in real-time, which makes it impractical to observe extended operation of the shop. The mixed-mode emulation method, which switches time-advance between real-time and next-event modes, was developed. Issues in implementing and using the mixed-mode and the delay-scaling technique for FRT emulation were discussed. Experimental results showed that mixed-mode emulation had the potential to reduce run-times by more than 50% over real-time emulation. / Ph. D.

Effective software testing identifies potential bugs and helps correct them, producing more reliable and maintainable software. As software development processes have evolved, incremental testing techniques have grown in popularity, particularly with introduction of test-driven development (TDD). However, many programmers struggle to adopt TDD's "test a little, code a little" approach and conventional computer science classrooms neglect evaluating software development as a process. In response, we explore influences on students' testing behaviors, effects of incremental testing strategies, and describe approaches to help computer science students adopt good testing practices. First, to understand students' perspectives and adoption of testing strategies, we investigated their attitudes toward different aspects of TDD. In addition, we observed trends in when and how thoroughly students tested their code and how these choices impacted the quality of their assignments. However, with insight into why students struggle to adopt incremental testing, we identified a need to assess their behaviors during the software development process as a departure from traditional product-oriented evaluation. By building upon an existing automated grading system, we developed an adaptive feedback system to provide customized incentives to reinforce incremental testing behaviors while students solved programming assignments. We investigated how students react to concrete testing goals and hint reward mechanisms and found approaches for identifying testing behaviors and influencing short-term behavioral change. Moreover, we discovered how students incorporate automated feedback systems into their software development strategies. Finally, we compared testing strategies students exhibited through analyzing five years and thousands of snapshots of students' code during development. Even when accounting for factors such as procrastinating on assignments, we found that testing early and consistently maintaining testing throughout development helps produce better quality code and tests. By applying our findings of student software development behaviors to effective testing strategies and teaching techniques, we developed a framework for adaptively scaffolding feedback to empower students to critically reflect over their code and adopt incremental testing approaches. / Ph. D.

Software testing is important, but judging whether a set of software tests is effective is difficult. This problem also appears in the classroom as educators more frequently include software testing activities in programming assignments. The most common measures used to assess student-written software tests are coverage criteria—tracking how much of the student’s code (in terms of statements, or branches) is exercised by the corresponding tests. However, coverage criteria have limitations and sometimes overestimate the true quality of the tests. This dissertation investigates alternative measures of test quality based on how many defects the tests can detect either from code written by other students—all-pairs execution—or from artificially injected changes—mutation analysis. We also investigate a new potential measure called checked code coverage that calculates coverage from the dynamic backward slices of test oracles, i.e. all statements that contribute to the checked result of any test. Adoption of these alternative approaches in automated classroom grading systems require overcoming a number of technical challenges. This research addresses these challenges and experimentally compares different methods in terms of how well they predict defect-detection capabilities of student-written tests when run against over 36,500 known, authentic, human-written errors. For data collection, we use CS2 assignments and evaluate students’ tests with 10 different measures—all-pairs execution, mutation testing with four different sets of mutation operators, checked code coverage, and four coverage criteria. Experimental results encompassing 1,971,073 test runs show that all-pairs execution is the most accurate predictor of the underlying defect-detection capability of a test suite. The second best predictor is mutation analysis with the statement deletion operator. Further, no strong correlation was found between defect-detection capability and coverage measures. / Ph. D.

The fundamentals of software testing and related activities are often elusive in undergraduate curricula. A direct consequence of the lack of software testing efforts during education is the huge losses suffered by the software industry when applications are not sufficiently tested. Software practitioners have exhorted faculty members and institutions to teach more software testing in universities. The purpose of this research is to provide answers to the needs of such practitioners and introduce software-testing activities throughout the curriculum. The most important goal is to introduce software-testing education without requiring a significant amount of extra effort on behalf of faculty members or teaching assistants. The approach taken comprises the development of the Web-based Center for Automated Testing (Web-CAT) and the introduction of test-driven development (TDD) in courses. Web-CAT serves as a learning environment for software testing tasks and helps automatically assess student assignments. A comparison of student programs developed using Web-CAT with historical records indicated a significant decrease in the number of bugs in submitted programs. Undergraduate students also received exposure to the principles of software testing and were able to write test cases that were on an average better than those generated by an automated test case generator designed specifically for the assignment. / Master of Science

While mutation testing is an effective fault-based testing technique, it has its challenges such as being computationally expensive and requiring a large amount of effort to review surviving mutants. These problems have resulted in mutation testing mostly being restricted to academic research and not as widely adopted in the industry. In the academic context, the focus has been on maximizing the mutation score and while a high mutation score might increase the quality of the software, it is not feasible to kill all the mutants. Moreover, all mutants are not as equally important, and some can not or should not be killed. Instead, the focus should be shifted to prioritizing the productive mutants, those that further improve the test suite or the source code. This thesis investigated if some mutation operators are more suitable for certain types of software by using selective mutation. The mutation operators were evaluated based on their ability to generate productive mutants. Moreover, the mutation operators were analyzed to identify how they could be improved to reduce the number of unproductive mutants generated by them. Dextool Mutate was used to conduct mutation testing on four open-source C/C++ software that were all different types of software. It was concluded that some mutation operators are more suitable for certain types of software resulting in the proposal of a set of mutation operators for each software type. Moreover, various improvements for the mutation operators were identified that reduce the number of unproductive mutants generated. Lastly, it may be helpful to customize the implementation of mutation operators for each type of software and some software types may require additional specialized mutation operators.

The computational power and algorithms needed to create a cognitive radio are quickly becoming available. There are many advantages to having a radio operated by cognitive engine, and so cognitive radios are likely to become very popular in the future. One of the main difficulties associated with the cognitive radio is ensuring the signal transmitted will follow all FCC rules. The work presented in this thesis provides a methodology to guarantee that all signals will be legal and valid. The first part to achieving this is a practical and easy to use software testing program based on the tabu search algorithm that tests the software off-line. The primary purpose of the software testing program is to find most of the errors, specially structural errors, while the radio is not in use so that it does not affect the performance of the system. The software testing program does not provide a complete assurance that no errors exist, so to supplement this deficit, a built-in self-test (BIST) is employed. The BIST is designed with two parts, one that is embedded into the cognitive engine and one that is placed into the radio's API. These two systems ensure that all signals transmitted by the cognitive radio will follow FCC rules while consuming a minimal amount of computational power. The software testing approach based on the tabu search is shown to be a viable method to test software with improved results over previous methods. Also, the software BIST demonstrated its ability to find errors in the signal production and is dem to only require an insignificant amount of computational power. Overall, the methods presented in this paper provide a complete and practical approach to assure the FCC of the legality of all signals in order to obtain a license for the product. / Master of Science