<p>Ericsson has a very big and expensive test environment with a lot of GSM AXE equipment. In order to decrease the cost of testing Ericsson has developed a combination of simulated and real hardware, the Hybrid Test Environment (HTE). There is no formal supervision and testing of the HTE system today and this has left the HTE system unstable and the testers have been avoiding using HTE. It is important for Ericsson that the confidence for HTE will increase. The goal of this thesis is to produce a method for testing the HTE system. An automated test tool has been implemented in order to monitor and test the HTE system. During the two weeks that the test tool has been operational it has discovered 4 servers in 3 different HTE rigs that malfunctioned. These servers were fixed and were operational before the end-users could discover any problem.</p>

<p>Within Ericsson AB, integration and verification activities is done on the network level in order to secure the functionality of the network. Protocol analysers are used to capture the traffic in the network. This results in many log files, which needs to be analysed. To do this, a protocol decoder tool called Scapy/LHC is used. Scapy/LHC is a framework that allows the users to write their own script to retrieve the data they need from the log files. The Scapy/LHC framework is incrementally developed as open source within Ericsson when there are needs for more functionality. This is often done by the users, outside normal working tasks. Because of this, there is almost no testing done to verify that old and new functionality works as expected, and there is no formal test strategy in use today.</p><p><br />The goal of this master’s thesis is to evaluate test strategies that are possible to use on the Scapy/LHC framework. To make the time needed for the testing process as short as possible, the test strategy needs to be automated. Therefore, possible test automation tools shall also be evaluated.</p><p><br />Two possible test strategies and two possible test automation tools are evaluated in this thesis. A test strategy, where the scripts that are written by the users are used, is then selected for implementation. The two test automation tools are also implemented. The evaluation of the implemented test strategy shows that it is possible to find defects in the Scapy/LHC framework in a time efficient way with help of the implemented test strategy and any of the implemented test automation tools.</p>

Within Ericsson AB, integration and verification activities is done on the network level in order to secure the functionality of the network. Protocol analysers are used to capture the traffic in the network. This results in many log files, which needs to be analysed. To do this, a protocol decoder tool called Scapy/LHC is used. Scapy/LHC is a framework that allows the users to write their own script to retrieve the data they need from the log files. The Scapy/LHC framework is incrementally developed as open source within Ericsson when there are needs for more functionality. This is often done by the users, outside normal working tasks. Because of this, there is almost no testing done to verify that old and new functionality works as expected, and there is no formal test strategy in use today. <br />The goal of this master’s thesis is to evaluate test strategies that are possible to use on the Scapy/LHC framework. To make the time needed for the testing process as short as possible, the test strategy needs to be automated. Therefore, possible test automation tools shall also be evaluated. <br />Two possible test strategies and two possible test automation tools are evaluated in this thesis. A test strategy, where the scripts that are written by the users are used, is then selected for implementation. The two test automation tools are also implemented. The evaluation of the implemented test strategy shows that it is possible to find defects in the Scapy/LHC framework in a time efficient way with help of the implemented test strategy and any of the implemented test automation tools.

Ericsson has a very big and expensive test environment with a lot of GSM AXE equipment. In order to decrease the cost of testing Ericsson has developed a combination of simulated and real hardware, the Hybrid Test Environment (HTE). There is no formal supervision and testing of the HTE system today and this has left the HTE system unstable and the testers have been avoiding using HTE. It is important for Ericsson that the confidence for HTE will increase. The goal of this thesis is to produce a method for testing the HTE system. An automated test tool has been implemented in order to monitor and test the HTE system. During the two weeks that the test tool has been operational it has discovered 4 servers in 3 different HTE rigs that malfunctioned. These servers were fixed and were operational before the end-users could discover any problem.

<p>For the majority of the estimated 70,000 industrial chemical substances available on the European market today there is not enough information to enable a reasonably complete assessment of the risks that they might pose to man and the environment. Any strategy for the generation of additional data for these substances should aim at making testing as efficient as possible taking into account environmental and health protection, time, monetary cost and animal welfare. To achieve this, appropriate priority setting rules are needed.</p><p>The main criterion currently used for regulatory priority setting for testing of industrial chemicals is production volume; the higher the production volume, the more information is required. This was also the main criterion in the former legislation, preceding REACH (Registration, Evaluation and Authorisation of Chemicals). The aim of this thesis is to evaluate other priority setting criteria and their implications for risk management, in particular classification and labelling.</p><p>The first paper in this thesis includes a study of the<i> efficiency ratio</i> for some of the tests required for the notification of new substances, i.e. the ratio between the likelihood that the test will lead to a classification, and the monetary cost of performing the test. The efficiency ratio was determined for the standard tests for acute oral toxicity, irritation, sensitisation and subacute toxicity using data from 1409 new chemicals notified in Europe between 1994 and 2004. The results of this investigation suggest that, given limited resources for testing, it is more efficient to perform acute toxicity tests on a larger number of substances rather than to perform additional subacute toxicity studies on the substances already tested for acute toxicity.</p><p>The second paper included in this thesis, reports the results from a comparative study of the bioaccumulating properties of substances being (a) classified as carcinogenic, mutagenic and/or toxic to reproduction (CMR-substances), or (b) classified as acutely toxic or (c) unclassified. The purpose of this investigation was to evaluate potential consequences of prioritising bioaccumulating chemicals for evaluation and testing, as this is one of the strategies prescribed in REACH. The results of this study suggest that bioaccumulating substances are neither over- nor underrepresented among the CMR-substances. This result lends support to the use of the bioconcentration factor for priority setting.</p><p>The studies reported in this thesis utilize existing data on classification of substances as an indicator of the outcome of the risk assessment process, relating priority setting methods to the risk management measures that they give rise to. To the best of my knowledge there are still only very few studies published that address the issue of priority setting in chemicals control using this approach, and in my view there is need for more studies of priority setting methods and a further development of priority setting strategies that are science-based.</p>

For the majority of the estimated 70,000 industrial chemical substances available on the European market today there is not enough information to enable a reasonably complete assessment of the risks that they might pose to man and the environment. Any strategy for the generation of additional data for these substances should aim at making testing as efficient as possible taking into account environmental and health protection, time, monetary cost and animal welfare. To achieve this, appropriate priority setting rules are needed. The main criterion currently used for regulatory priority setting for testing of industrial chemicals is production volume; the higher the production volume, the more information is required. This was also the main criterion in the former legislation, preceding REACH (Registration, Evaluation and Authorisation of Chemicals). The aim of this thesis is to evaluate other priority setting criteria and their implications for risk management, in particular classification and labelling. The first paper in this thesis includes a study of the efficiency ratio for some of the tests required for the notification of new substances, i.e. the ratio between the likelihood that the test will lead to a classification, and the monetary cost of performing the test. The efficiency ratio was determined for the standard tests for acute oral toxicity, irritation, sensitisation and subacute toxicity using data from 1409 new chemicals notified in Europe between 1994 and 2004. The results of this investigation suggest that, given limited resources for testing, it is more efficient to perform acute toxicity tests on a larger number of substances rather than to perform additional subacute toxicity studies on the substances already tested for acute toxicity. The second paper included in this thesis, reports the results from a comparative study of the bioaccumulating properties of substances being (a) classified as carcinogenic, mutagenic and/or toxic to reproduction (CMR-substances), or (b) classified as acutely toxic or (c) unclassified. The purpose of this investigation was to evaluate potential consequences of prioritising bioaccumulating chemicals for evaluation and testing, as this is one of the strategies prescribed in REACH. The results of this study suggest that bioaccumulating substances are neither over- nor underrepresented among the CMR-substances. This result lends support to the use of the bioconcentration factor for priority setting. The studies reported in this thesis utilize existing data on classification of substances as an indicator of the outcome of the risk assessment process, relating priority setting methods to the risk management measures that they give rise to. To the best of my knowledge there are still only very few studies published that address the issue of priority setting in chemicals control using this approach, and in my view there is need for more studies of priority setting methods and a further development of priority setting strategies that are science-based. / QC 20101115

Massenanpassung, kleine Losgrößen, hohe Variabilität der Produkttypen und ein sich während des Lebenszyklus einer industriellen Anlage änderndes Produktportfolio sind aktuelle Trends der Industrie. Durch eine zunehmende Entkopplung der Entwicklung von Software- und Hardwarekomponenten im industriellen Kontext, entstehen immer häufiger Kompatibilitätsprobleme innerhalb von industriellen Steuerungen. In dieser Arbeit wird mittels Literaturrecherche und angewandter Forschung ein Strategiekonzept zur Kompatibilitätsprüfung hergeleitet und diskutiert. Dieses vierphasige Konzept ermittelt Inkompatibilitäten zwischen Software- und Hardwarekomponenten im Umfeld von industriellen Steuerungen und ermöglicht Testingenieuren das frühzeitige Erkennen von Problemen. Durch eine automatische Durchführung der Kompatibilitätsprüfung auf einem externen Industrie PC kann die Kompatibilitätsprüfung sowohl beim Aufspielen neuer Software auf die industrielle Steuerung als auch beim Neustart der Steuerung ablaufen. Somit werden Änderungen an den Komponenten stetig erkannt und Inkompatibilitäten vermieden. Weiterhin kann durch die frühzeitige Erkennung sichergestellt werden, dass eine Anlage dauerhaft lauffähig bleibt. Anhand einer Diskussion werden Mittel festgestellt, um die Robustheit und Anwendbarkeit des vorgestellten Konzeptes zusätzlich zu festigen.:1 Motivation 1 1.1 Aufgabenanalyse 3 1.1.1 Forschungsfragen und Teilaufgaben 3 1.1.2 Aufgabenkomplexe 4 1.1.3 Eingrenzung der Aufgabenstellung 5 1.1.4 Ziel der Arbeit 6 1.1.5 Festsetzung von Formulierungen 6 2 Einführung und Stand der Technik 7 2.1 VIBN von industriellen Anlagen 7 2.1.1 Teststrategien aus der VIBN 9 2.1.1.1 Model-in-the-Loop 9 2.1.1.2 Software-in-the-Loop 9 2.1.1.3 Hardware-in-the-Loop 10 2.1.1.4 Konklusion und Forschungsbestrebungen 11 2.2 CS in industriellen Anlagen 12 2.2.1 Sicherheitsziel 13 2.2.2 Teststrategien aus der CS 13 2.2.2.1 Signaturbasierte Erkennung 14 2.2.2.2 Anomaliebasierte Erkennung 14 2.2.2.3 Konklusion und Forschungsbestrebungen 16 2.3 Interoperabilität als Kompatibilitätsmaß 16 2.4 Testautomatisierung und Test Case Generierung 17 2.5 Allgemeine Softwareteststrategien 17 2.5.1 Modellbasiertes Testen 17 2.5.2 Funktionale Tests 18 2.6 Allgemeine Hardware Teststrategien 19 2.6.1 Modellbasiertes Testen 19 2.6.2 Manuelles Testen 19 2.7 Interoperabilität in industriellen Anlagen 20 2.7.1 Definitionen der Interoperabilität 20 2.7.2 Herausforderungen der Interoperabilität 22 2.7.3 Implementierung von Interoperabilität 22 2.7.3.1 Syntaktische Interoperabilität 23 2.7.3.2 Semantische Interoperabilität 23 2.7.4 Vertikale Integration 24 2.7.5 Horizontale Integration 25 3 Anforderungsanalyse 27 3.1 Adaption von Strategien der VIBN und CS 27 3.2 Anforderungen 28 3.2.1 Anforderungen an die Kompatibilitätsprüfung 28 3.2.2 Anforderungen an die Hardwarekomponenten 29 3.2.3 Anforderungen an die Softwarekomponenten 29 4 Konzept 30 4.1 Komponenten des Teststrategiekonzeptes 30 4.1.1 SPS Selbsttest 32 4.1.2 Export & Import des Soll-Zustandes 32 4.1.3 Ermittlung des Ist-Zustandes 35 4.1.4 Vergleich des Soll- & Ist-Zustandes 35 4.2 Fehlerdetektionstabellen 36 4.3 Reaktionen auf Inkompatibilitäten 38 5 Evaluation 39 5.1 Methodik und Evaluationskriterien 39 5.2 Anwendungsbeispiel 39 5.3 Referenzsystem für Evaluation 41 5.4 Durchführung Evaluation 41 5.5 Erfüllung der Anforderungen an die Kompatibilitätsprüfung 46 6 Diskussion 48 6.1 Beantwortung der Forschungsfragen 48 6.2 Diskussion zur Forschungsmethodik 48 6.3 Bewertung des Konzeptes 49 7 Zusammenfassung und Ausblick 50 7.1 Zusammenfassung 50 7.2 Ausblick und weitere Forschungsarbeit 51 Literaturverzeichnis 52 / Mass customization, small batch sizes, high variability of product types and a changing product portfolio during the life cycle of an industrial plant are current trends in the industry. Due to an increasing decoupling of the development of software and hardware components in an industrial context, compatibility problems within industrial control systems arise more and more frequently. In this thesis, a strategy concept for compatibility testing is derived and discussed by means of literature review and applied research. This 4-phased strategy concept identifies incompatibilities between software and hardware components in the industrial control environment and enables test engineers to detect problems at an early stage. By automating the compatibility test on an external I-PC, the test can be run both when new software is installed on the industrial controller and when the controller is restarted. Thus, changes to the components are constantly detected and incompatibilities are avoided. Furthermore, early incompatibility detection can ensure that a system remains permanently operational. Based on a discussion, additionally strategies are identified to consolidate the robustness and applicability of the presented concept.:1 Motivation 1 1.1 Aufgabenanalyse 3 1.1.1 Forschungsfragen und Teilaufgaben 3 1.1.2 Aufgabenkomplexe 4 1.1.3 Eingrenzung der Aufgabenstellung 5 1.1.4 Ziel der Arbeit 6 1.1.5 Festsetzung von Formulierungen 6 2 Einführung und Stand der Technik 7 2.1 VIBN von industriellen Anlagen 7 2.1.1 Teststrategien aus der VIBN 9 2.1.1.1 Model-in-the-Loop 9 2.1.1.2 Software-in-the-Loop 9 2.1.1.3 Hardware-in-the-Loop 10 2.1.1.4 Konklusion und Forschungsbestrebungen 11 2.2 CS in industriellen Anlagen 12 2.2.1 Sicherheitsziel 13 2.2.2 Teststrategien aus der CS 13 2.2.2.1 Signaturbasierte Erkennung 14 2.2.2.2 Anomaliebasierte Erkennung 14 2.2.2.3 Konklusion und Forschungsbestrebungen 16 2.3 Interoperabilität als Kompatibilitätsmaß 16 2.4 Testautomatisierung und Test Case Generierung 17 2.5 Allgemeine Softwareteststrategien 17 2.5.1 Modellbasiertes Testen 17 2.5.2 Funktionale Tests 18 2.6 Allgemeine Hardware Teststrategien 19 2.6.1 Modellbasiertes Testen 19 2.6.2 Manuelles Testen 19 2.7 Interoperabilität in industriellen Anlagen 20 2.7.1 Definitionen der Interoperabilität 20 2.7.2 Herausforderungen der Interoperabilität 22 2.7.3 Implementierung von Interoperabilität 22 2.7.3.1 Syntaktische Interoperabilität 23 2.7.3.2 Semantische Interoperabilität 23 2.7.4 Vertikale Integration 24 2.7.5 Horizontale Integration 25 3 Anforderungsanalyse 27 3.1 Adaption von Strategien der VIBN und CS 27 3.2 Anforderungen 28 3.2.1 Anforderungen an die Kompatibilitätsprüfung 28 3.2.2 Anforderungen an die Hardwarekomponenten 29 3.2.3 Anforderungen an die Softwarekomponenten 29 4 Konzept 30 4.1 Komponenten des Teststrategiekonzeptes 30 4.1.1 SPS Selbsttest 32 4.1.2 Export & Import des Soll-Zustandes 32 4.1.3 Ermittlung des Ist-Zustandes 35 4.1.4 Vergleich des Soll- & Ist-Zustandes 35 4.2 Fehlerdetektionstabellen 36 4.3 Reaktionen auf Inkompatibilitäten 38 5 Evaluation 39 5.1 Methodik und Evaluationskriterien 39 5.2 Anwendungsbeispiel 39 5.3 Referenzsystem für Evaluation 41 5.4 Durchführung Evaluation 41 5.5 Erfüllung der Anforderungen an die Kompatibilitätsprüfung 46 6 Diskussion 48 6.1 Beantwortung der Forschungsfragen 48 6.2 Diskussion zur Forschungsmethodik 48 6.3 Bewertung des Konzeptes 49 7 Zusammenfassung und Ausblick 50 7.1 Zusammenfassung 50 7.2 Ausblick und weitere Forschungsarbeit 51 Literaturverzeichnis 52