Scenario-based testing emerges as the main approach to validate automated driving systems (ADS) and thus ensure safe road traffic. Thereby, the test scenarios used should represent the traffic event of the corresponding operational design domain (ODD) and should cover the traffic situation from normal driving to an accident. For this, the fusion of police accident data and video-based traffic observation data into one database for subsequent scenario generation is advisable. Therefore, this paper presents the Fuse4Representativity (Fuse4Rep) process model as part of the Dresden Method, which helps to fuse heterogeneous data sets into one ODD-representative database for lean, fast and comprehensive scenario generation. Hereby, statistical matching is used as the fusion approach building on probable matching variables, such as the 3-digit accident type, the collision type and the misconduct of participants. Moreover, the paper shows how the scenarios generated in this way can be hypothetically used to validate ADS, e.g. in a stochastic traffic simulation incorporating human driver behaviour models. Future studies should apply the Fuse4Rep model in practice and test its validity.

Scenario-based testing is a pillar of assessing the effectiveness of automated driving systems (ADSs). For data-driven scenario-based testing, representative traffic scenarios need to describe real road traffic situations in compressed form and, as such, cover normal driving along with critical and accident situations originating from different data sources. Nevertheless, in the choice of data sources, a conflict often arises between sample quality and depth of information. Police accident data (PD) covering accident situations, for example, represent a full survey and thus have high sample quality but low depth of information. However, for local video-based traffic observation (VO) data using drones and covering normal driving and critical situations, the opposite is true. Only the fusion of both sources of data using statistical matching can yield a representative, meaningful database able to generate representative test scenarios. For successful fusion, which requires as many relevant, shared features in both data sources as possible, the following question arises: How can VO data be collected by drones and analysed to create the maximum number of relevant, shared features with PD? To answer that question, we used the Find–Unify–Synthesise–Evaluation (FUSE) for Representativity (FUSE4Rep) process model.We applied the first (“Find”) and second (“Unify”) step of this model to VO data and conducted drone-based VOs at two intersections in Dresden, Germany, to verify our results. We observed a three-way and a four-way intersection, both without traffic signals, for more than 27 h, following a fixed sample plan. To generate as many relevant information as possible, the drone pilots collected 122 variables for each observation (which we published in the ListDB Codebook) and the behavioural errors of road users, among other information. Next, we analysed the videos for traffic conflicts, which we classified according to the German accident type catalogue and matched with complementary information collected by the drone pilots. Last, we assessed the crash risk for the detected traffic conflicts using generalised extreme value (GEV) modelling. For example, accident type 211 was predicted as happening 1.3 times per year at the observed four-way intersection. The process ultimately facilitated the preparation of VO data for fusion with PD. The orientation towards traffic conflicts, the matched behavioural errors and the estimated GEV allowed creating accident-relevant scenarios. Thus, the model applied to VO data marks an important step towards realising a representative test scenario database and, in turn, safe ADSs.

Scenario-based testing emerges as the main approach to validate automated driving systems (ADS) and thus ensure safe road traffic. Thereby, the test scenarios used should represent the traffic event of the corresponding operational design domain (ODD) and should cover the traffic situation from normal driving to an accident. For this, the fusion of police accident data and video-based traffic observation data into one database for subsequent scenario generation is advisable. Therefore, this paper presents the Fuse4Representativity (Fuse4Rep) process model as part of the Dresden Method, which helps to fuse heterogeneous data sets into one ODD-representative database for lean, fast and comprehensive scenario generation. Hereby, statistical matching is used as the fusion approach building on probable matching variables, such as the 3-digit accident type, the collision type and the misconduct of participants. Moreover, the paper shows how the scenarios generated in this way can be hypothetically used to validate ADS, e.g. in a stochastic traffic simulation incorporating human driver behaviour models. Future studies should apply the Fuse4Rep model in practice and test its validity.

Scenario-based testing is a major pillar in the development and effectiveness assessment of automated driving systems. Thereby, test scenarios address different information layers and situations (normal driving, critical situations and accidents) by using different databases. However, the systematic combination of accident and / or normal driving databases into new synthetic databases can help to obtain scenarios that are as realistic as possible. This paper shows how statistical matching (SM) can be applied to fuse different categorical accident and traffic observation databases. Hereby, the fusion is demonstrated in two use cases, each featuring several fusion methods. In use case 1, a synthetic database was generated out of two accident data samples, whereby 78.7% of the original values could be estimated correctly by a random forest classifier. The same fusion using distance-hot-deck reproduced only 67% of the original values, but better preserved the marginal distributions. A real-world application is illustrated in use case 2, where accident data was fused with over 23,000 car trajectories at one intersection in Germany. We could show that SM is applicable to fuse categorical traffic databases. In future research, the combination of hotdeck- methods and machine learning classifiers needs to be further investigated.

Traditional software testing usually comes with manual definitions of test cases. This manual process can be time-consuming, tedious, and incomplete in covering important but elusive corner cases that are hardly identifiable. Automatic generation of random test cases emerges as a strategy to mitigate the challenges associated with the manual test case design. However, the effectiveness of random test cases in fault detection may be limited, leading to increased testing costs, particularly in systems where test execution demands substantial resources and time. Leveraging the domain knowledge of test experts can guide the automatic random generation of test cases to more effective zones. In this thesis, we target quality assurance of multiagent autonomous systems and aim to automate test generation for them by applying the domain knowledge of test experts. To formalize the specification of the domain expert's knowledge, we introduce a small Domain Specific Language (DSL) for formal specification of particular locality-based constraints for grid-based multiagent systems. We initially employ this DSL for filtering randomly generated test inputs. Then, we evaluate the effectiveness of the generated test cases through an experiment on a case study of autonomous agents. Applying statistical analysis on the experiment results demonstrates that utilizing the domain knowledge to specify test selection criteria for filtering randomly generated test cases significantly reduces the number of potentially costly test executions to identify the persisting faults.  Domain knowledge of experts can also be utilized to directly generate test inputs with constraint solvers. We conduct a comprehensive study to compare the performance of filtering random cases and constraint-solving approaches in generating selective test cases across various test scenario parameters. The examination of these parameters provides criteria for determining the suitability of random data filtering versus constraint solving, considering the varying size and complexity of the test input generation constraint. To conduct our experiments, we use QuickCheck tool for random test data generation with filtering, and we employ Z3 for constraint solving. The findings, supported by observations and statistical analysis, reveal that test scenario parameters impact the performance of filtering and constraint-solving approaches differently. Specifically, the results indicate complementary strengths between the two approaches: random generation and filtering approach excels for the systems with a large number of agents and long agent paths but shows degradation in larger grid sizes and stricter constraints. Conversely, constraint solving approach demonstrates robust performance for large grid sizes and strict constraints but experiences degradation with increased agent numbers and longer paths. Our initially proposed DSL is limited in its features and is only capable of specifying particular locality-based constraints. To be able to specify more elaborate test scenarios, we extend that DSL based on a more intricate model of autonomous agents and their environment. Using the extended DSL, we can specify test oracles and test scenarios for a dynamic grid environment and agents having several attributes. To assess the extended DSL's utility, we design a questionnaire to gather opinions from several experts and also run an experiment to compare the efficiency of the extended DSL with the initially proposed one. The questionnaire results indicate that the extended DSL was successful in specifying several scenarios that the experts found more useful than the scenarios specified by the initial DSL. Moreover, the experimental results demonstrate that testing with the extended DSL can significantly reduce the number of test executions to detect system faults, leading to a more efficient testing process. / Safety of Connected Intelligent Vehicles in Smart Cities – SafeSmart