Code faults can seriously degrade scientific software accuracy. Therefore, it is imperative that scientific software developers scrutinize their codes in an attempt to find these faults. This thesis explores, particularly, the efficacy of code testing as a method of scientific software code fault detection. Software engineers, as experts in code quality, have developed many code testing techniques, but many of these techniques cannot readily be applied to scientific codes for at least two reasons. First, scientific software testers do not usually have access to an ideal oracle. Second, scientific software outputs, by nature, can only be judged for accuracy and not correctness. Testing techniques developed under the assumption that these two problems can be ignored--as most have been--are of questionable value to computational scientists. To demonstrate the reality of these problems and to provide an example of how software engineers and scientists can begin to address them, this thesis discusses the development and application of a novel technique: Mutation Sensitivity Testing (MST). MST is based on traditional mutation testing, but--in place of a focus on mutant "killing"--MST focuses on assessing the mutation sensitivity of a test set. In this thesis, MST experiments are conducted using eight small numerical routines, four classes of mutation operators, and 1155 tests. The results are discussed and some conclusions are drawn. Two observations are of particular interest to computational scientists. First, it is found that oracles that exhibit uncertainties greater than (approximately) 80% of the expected output are of questionable value when they are used in the testing of scientific software. Second, it is found that a small number of carefully selected tests may be sufficient to falsify a code. / Thesis (Master, Computing) -- Queen's University, 2009-04-19 13:34:08.943

The train represents a complex system, where every sub-system has an important role. If a subsystem doesn’t work how it should, the correctness of whole the train can be uncertain. To ensure that system works properly, we should test each sub-system individually and integrate them together in the whole system. Each of these subsystems consists of the different modules with different functionalities what should be tested. Testing of different functionalities often requires a different approach. For some functionalities, it is necessary domain knowledge from the human expert, such as classification of signals in different use cases in Propulsion and Controls (PPC) in Bombardier Transportation. Due to this reason, we need to simulate of using experts knowledge in the certain domain. We are investigating the use of machine learning techniques for solving this cases and creating system what will automatically classify different signals using the previous human knowledge. This case study is conducted in Bombardier Transportation (BT), Västerås in departments Train Control Management System (TCMS) and Propulsion and Controls (PPC), where data is collected, analyzed and evaluated. We proposed a method for solving the oracle problem based on machine learning approach for different for certain use case. Also, we explained different steps what can be used for solving the test oracle problem where signals are part of verdict process