International Telemetering Conference Proceedings / October 20-23, 2003 / Riviera Hotel and Convention Center, Las Vegas, Nevada / The CCSDS recently developed the CCSDS File Delivery Protocol which can operate in configurations from simple point-to-point space/ground systems to complex arrangements of orbiters, landers, relaying spacecraft, and multiple ground facilities. An international interoperability test program has been developed in support of the development and fielding of the protocol. The first phase was successfully completed in the Fall of 2002. The second phase is to be completed about the time of the presentation of this paper. First phase testing involved five independent implementations of the Core Procedures of the protocol. Since these Procedures are for point-to-point file transfers, only two protocol “entities” were involved in each executed test. The second phase tests other Procedures, which involve multi-hop transferring of files. This phase will involve two separate, independent implementations, with one implementation being hosted on multiple hardware/OS platforms. Each test will involve three or more entities.

Today, interoperability is the major challenge for e-Business and e-Government domains. The fundamental solution is the standardization in different levels of business-to-business interactions. However publishing standards alone are not enough to assure interoperability between products of different vendors. In this respect, testing and certification activities are very important to promote standard adoption, validate conformance and interoperability of the products and maintain correct information exchange. In e-Business collaborations, standards need to address different layers of interoperability stack / communication layer, business document layer and business process layer. Although there have been conformance and interoperability testing tools and initiatives for each one of these categories, there is currently no support for testing an integration of the above within a test scenario which is similar to real life use cases. Together with the integration of different layers of testing, testing process should be automated so that test case execution can be done at low cost, and repeated if required. In this theses, a highly adaptable and flexible Test Execution Model and a complementary XML based Test Description Language consisting of high level test constructs which can handle or simulate different parts or layers of the interoperability stack is designed. The computer interpretable test description language allow dynamic set up of test cases and provides flexibility to design, modify, maintain and extend the test functionality in contrast to a priori designed and hard coded test cases. The work presented in this thesis is a part of the TestBATN system supported by TUBITAK, TEYDEB Project No: 7070191.

In the field of networking, testing of communication protocols is an important activity to validate protocol applications before commercialisation. Generally, the services that must be provided by a protocol are described in its specification(s). A specification is generally a standard defined by standards bodies such as ISO (International Standards Organization), IETF (Internet Engineering Task Force), ITU (International Telecommunication Union), etc. The purpose of testing is to verify that the protocol implementations work correctly and guarantee the quality of the services in order to meet customers expectations. To achieve this goal, a variety of testing methods have been developed. Among them, interoperability testing is to verify that several network components cooperate correctly and provide expected services. Conformance testing verifies that a product conforms to its specification. Robustness testing determines the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions. In this thesis, we focus on interoperability testing. The general architecture of interoperability testing involves a system under test (SUT), which consists of at least two implementations under test (IUT). The objectives of interoperability testing are to ensure that interconnected protocol implementations are able to interact correctly and, during their interaction, provide the services predefined in their specifications. In general, the methods of interoperability testing can be classified into two approaches: active and passive testing. Among them, active test is the most conventionally used technique, which aims to test the implementations (IUT) by injecting a series of test messages (stimuli) and observing the corresponding outputs. However, the intrusive nature of active testing is that the tester has the ability to control IUTS. This implies that the tester interrupts inevitably the normal operations of the system under test. In this sense, active testing is not a suitable technique for interoperability testing, which is often carried out in operational networks. In such context, it is difficult to insert arbitrary testing messages without affecting the normal behavior and the services of the system. On the contrary, passive testing is a technique based only on observation. The tester does not need to interact with the SUT. This allows the test to be carried out without disturbing the normal operations of the system under test. Besides, passive testing also has other advantages such as: for embedded systems to which the tester does not have direct access, test can still be performed by collecting the execution traces of the system and then detect errors by comparing the trace with the behavior of the system described in its specification. In addition, passive testing makes it possible to moniter a system over a long period, and report abnomality at any time.

Testing is one of the critical processes in software development life cycle. It plays key role in the success of software product by improving its quality. Web-based applications are emerging and evolving rapidly; their importance and complexity is also increasing. Heterogeneous and diverse nature of distributed components, applications; along with their multi-platform support and cooperativeness make these applications more complex and swiftly increasing in their size. Quality assurance of these applications is becoming more crucial and important; testing is one of the key processes to achieve and ensure the quality of these software or Web-based products. There are many testing challenges involved in Web-based applications. But most importantly interoperability and integration are the most critical testing challenges associated with Web-based applications. There are number of challenging factors involved in both integration and interoperability testing efforts. These integration and interoperability factors have almost 70 percent to 80 percent impact on overall quality of Web-based applications. In software industry different kind of testing approaches are used by practitioners to solve the issues associated with integration and interoperability, which are due to ever increasing complexities of Web-based applications. It is fact that both integration and interoperability are inter-related and it is very helpful to cover all the possible issues of interoperability testing that will reduce the integration testing effort. It will be more beneficial if a dedicated testing team is placed to perform the both integration and interoperability testing. / +46 73 55 955 15 +46 73 95 120 32

Large scale outages have occurred worldwide in recent decades with some impacting 15-25% of a nation’s population. The complexity of blackouts has been extensively studied but many questions remain. As there are no perfect solutions to prevent blackouts, usually caused by a complex sequence of cascading events, a number of different measures need to be undertaken to minimize impact of future disturbances. Increase in deployment of phasor measurement units (PMUs) across the grid has given power industry an unprecedented technology to study dynamic behavior of the system in real time. Integration of large scale synchronized measurements with SCADA system requires a careful roadmap and methodology. When properly engineered, tested, and implemented, information extracted from synchrophasor data streams provides realtime observability for transmission system. Synchrophasor data can provide operators with quick insight into precursors of blackout (e.g., angular divergence) which are unavailable in traditional SCADA systems. Current visualization tools and SE functions, supported by SCADA, provide some basic monitoring. Inaccuracies in measurements and system models, absence of redundancy in the measured parameters or breaker statuses in most cases, and lack of synchronization and time resolution in SCADA data result in limited functionality and precision for a typical EMS required in today’s operating environment of tighter margins that require more frequent and more precise data. Addition of synchrophasor data, typically having several orders of magnitude higher temporal resolution, (i.e., 60 to 120 measurements per second as opposed to one measurement every 4 to 8 seconds), can help detect higher speed phenomena and system oscillations. Also, time synchronization to one micro-second allows for accurate comparison of phase angles across the grid and identification of major disturbances and islanding. This dissertation proposes a more comprehensive, holistic set of criteria for optimizing PMU placement with consideration for diverse factors that can influence PMU siting decision-making process and incorporates several practical implementation aspects. An innovative approach to interoperability testing is presented and solutions are offered to address the challenges. The proposed methodology is tested to prove the concept and address real-life implementation challenges, such as interoperability among the PMUs located across a large area.

Dans le domaine des réseaux, le test de protocoles de communication est une activité importante afin de valider les protocoles applications avant de les mettre en service. Généralement, les services qu'un protocole doit fournir sont décrits dans sa spécification. Cette spécification est une norme ou un standard défini par des organismes de normalisation tels que l'ISO (International Standards Organisation), l'IETF (Internet Engineering Task Force), l'ITU (International Telecommunication Union), etc. Le but du test est de vérifier que les implémentations du protocole fonctionnent correctement et rendent bien les services prévus. Pour atteindre cet objectif, différentes méthodes de tests peuvent être utilisées. Parmi eux, le test de conformité vérifie qu'un produit est conforme à sa spécification. Le test de robustesse vérifie les comportements de l'implémentation de protocole face à des événements imprévus. Dans cette thèse, nous nous intéressons plus particulièrement au test d'interopérabilité, qui vise à vérifier que plusieurs composants réseaux interagissent correctement et fournissent les services prévus. L'architecture générale de test d'interopérabilité fait intervenir un système sous test (SUT) composé de plusieurs implémentations sous test (IUT). Les objectifs du test d'interopérabilité sont à la fois de vérifier que plusieurs implémentations (basées sur des protocoles conçus pour fonctionner ensemble) sont capables d'interagir et que, lors de leur interaction, elles rendent les services prévus dans leurs spécifications respectives. En général, les méthodes de test d'interopérabilité peuvent être classées en deux grandes approches: le test actif et le test passif. Le test actif est une technique de validation très populaire, dont l'objectif est essentiellement de tester les implémentations (IUT), en pratiquant une suite de contrôles et d'observations sur celles-ci. Cependant, une caractéristique fondamentale du test actif est que le testeur possède la capacité de contrôler les IUTs. Cela implique que le testeur perturbe le fonctionnement normal du système testé. De ce fait, le test actif n'est pas une technique appropriée pour le test d'interopérabilité, qui est souvent effectué dans les réseaux opérationnels, où il est difficile d'insérer des entrées arbitraires sans affecter les services ou les fonctionnements normaux des réseaux. A l'inverse, le test passif est une technique se basant uniquement sur les observations. Le testeur n'a pas besoin d'agir sur le SUT notamment en lui envoyant des stimuli. Cela permet au test d'être effectué sans perturber l'environnement normal du système sous test. Le test passif possède également d'autres avantages comme par exemple, pour les systèmes embarqués où le testeur n'a pas d'accès direct, de pourvoir effectuer le test en collectant des traces d'exécution du système, puis de détecter les éventuelles erreurs ou déviations de ces traces vis-à-vis du comportement du système. / In the field of networking, testing of communication protocols is an important activity to validate protocol applications before commercialisation. Generally, the services that must be provided by a protocol are described in its specification(s). A specification is generally a standard defined by standards bodies such as ISO (International Standards Organization), IETF (Internet Engineering Task Force), ITU (International Telecommunication Union), etc. The purpose of testing is to verify that the protocol implementations work correctly and guarantee the quality of the services in order to meet customers expectations. To achieve this goal, a variety of testing methods have been developed. Among them, interoperability testing is to verify that several network components cooperate correctly and provide expected services. Conformance testing verifies that a product conforms to its specification. Robustness testing determines the degree to which a system operates correctly in the presence of exceptional inputs or stressful environmental conditions. In this thesis, we focus on interoperability testing. The general architecture of interoperability testing involves a system under test (SUT), which consists of at least two implementations under test (IUT). The objectives of interoperability testing are to ensure that interconnected protocol implementations are able to interact correctly and, during their interaction, provide the services predefined in their specifications. In general, the methods of interoperability testing can be classified into two approaches: active and passive testing. Among them, active test is the most conventionally used technique, which aims to test the implementations (IUT) by injecting a series of test messages (stimuli) and observing the corresponding outputs. However, the intrusive nature of active testing is that the tester has the ability to control IUTS. This implies that the tester interrupts inevitably the normal operations of the system under test. In this sense, active testing is not a suitable technique for interoperability testing, which is often carried out in operational networks. In such context, it is difficult to insert arbitrary testing messages without affecting the normal behavior and the services of the system. On the contrary, passive testing is a technique based only on observation. The tester does not need to interact with the SUT. This allows the test to be carried out without disturbing the normal operations of the system under test. Besides, passive testing also has other advantages such as: for embedded systems to which the tester does not have direct access, test can still be performed by collecting the execution traces of the system and then detect errors by comparing the trace with the behavior of the system described in its specification. In addition, passive testing makes it possible to moniter a system over a long period, and report abnomality at any time.

Conformance to standards and interoperability is a major challenge of today`s applications in all domains. Several standards have been developed and some are still under development to address the various layers in the interoperability stack. Conformance and interoperability testing involves checking whether the applications conform to the standards so that they can interoperate with other conformant systems. Only through testing, correct information exchange among applications can be guaranteed. National Health Information System (NHIS) of Turkey aims to provide a nation-wide infrastructure for sharing Electronic Health Records (EHRs). In order to guarantee the interoperability, the Ministry of Health (MoH), Turkey, developed an Implementation/Integration/Interoperability Profile based on HL7 standards. TestBATN - Testing Business Process, Application, Transport and Network Layers - is a domain and standards independent set of tools which can be used to test all of the layers of the interoperability stack, namely, the Communication Layer, Document Content Layer and the Business Process Layer. In this thesis work, the requirements for conformance and interoperability testing of the NHIS are analyzed, a testing approach is designated, test cases for several NHIS services are developed and deployed and a test execution control and monitoring environment within TestBATN is designed and implemented through the identified testing requirements. The work presented in this thesis is part of the TestBATN system supported by the T&Uuml / BiTAK TEYDEB Project No: 7070191 in addition by the Ministry of Health, Turkey.