International Telemetering Conference Proceedings / October 30-November 02, 1989 / Town & Country Hotel & Convention Center, San Diego, California / Modern aircraft testing and training increasingly demand the use of multiple targets. A novel method to meet this requirement is to use the new AN/MPS-39 Multiple Object Tracking Radar (MOTR) with Vega Target Control System equipment. The AN/MPS-39 can be loosely described as the equivalent of ten AN/FPS-16 radars. This equivalency, due largely to the AN/MPS-39’s phased array antenna, immediately suggests the controlling of multiple target drones as an added capability to the radar’s basic and demonstrated function as a precision metric instrument. This paper demonstrates the adaptability of the AN/MPS-39 MOTR to the use of VTCS, thus exploiting the AN/MPS-39’s inherent capability to control multiple target drones simultaneously.

This dissertation is situated at the intersection of aviation safety, sensor technology, and computational modeling, increasing airport apron safety by developing and testing optical sensing methods for automated apron surveillance. Central to this research is the utilization of Light Detection and Ranging (LiDAR) technology combined with computer vision algorithms for automatic scene understanding, complemented by tracking, motion prediction, and accident prediction functionalities for dynamic entities. Serving as the impetus for this research, an in-depth empirical analysis of 1220 aviation ground accident reports from 2008 to 2017 exhibits that 76 % of these occurrences could have been visually observed. Notably, the data reveals that 44 % of events indicate human failure, resulting from deficiencies in situational awareness among the involved parties. These findings highlight the opportunity for increasing airport safety by integrating automated surveillance methodologies. However, the ambitious endeavor of transitioning airport surveillance tasks to an automated system presents three main challenges. First, algorithms for automatic scene understanding rely on training datasets with ground truth annotations, which refer to semantic information representing real-world conditions. Such datasets do not exist for airport apron environments. Creating a training dataset for such environments involves scanning and manually annotating every aircraft type, ground vehicle, or object from multiple perspectives in every conceivable pose, velocity, and weather condition at multiple airports. Second, developing accurate tracking algorithms for aircraft relying on LiDAR point clouds requires time-synchronized true states for validation, which are not available. Third, recognizing visual features for accident prediction requires corresponding sensor data, which cannot be acquired in sufficient quantities given aviation's high safety standards and security-related access limitations to airport airside. Thus, this dissertation addresses these challenges by developing a simulation environment that provides training data and a testing framework to develop recognition models and tracking algorithms for real-world applications, utilizing Dresden International Airport as the test field. This simulation environment includes 3D models of the test field, kinematic descriptions of aircraft ground movements, and a sensor model replicating LiDAR sensor behavior under different weather conditions. The simulation environment obviates real-world data acquisition and manual annotation by generating synthetic LiDAR scans, automatically annotated using context knowledge inherent to the simulation framework. Consequently, it enables training recognition models on synthetic data applicable to real-world data. The simulation environment can be adapted to any airport by modifying the static background elements, thus addressing the first challenge. Sensor positioning within the simulation is fully customizable. The developed motion models are formulated in a general manner, ensuring their functionality across any movement network. For validation purposes, a real LiDAR dataset was collected at the test airport and manually annotated. Competing recognition models were trained: employing real-world training data and the other leveraging synthetic training data. These models were tested on a real test dataset not seen during the training. The results show that the synthetic data-trained model achieves recognition performance comparable to, or even superior to, the real-data-trained model. Specifically, it demonstrates improved recognition of aircraft and weather-induced noise within the real test dataset. This enhanced performance is attributed to an overrepresentation of aircraft and weather effects in the synthetic training data. The semantic segmentation model assigns semantic labels to each point of the point cloud. Tracking algorithms leverage this information to estimate the pose of objects. These estimations are crucial for verifying compliance with operational rules and to predict aircraft movement. Object positioning and orientation data inherent to the simulation enables the development and evaluation of tracking algorithms, addressing the second challenge. This research introduces an adaptive point sampling method for aircraft tracking that considers the velocity and spatial relationships of the tracked object, enhancing localization accuracy compared to conventional naïve sampling strategies in a simulated test dataset. Finally, addressing the third challenge, the empirical study of accidents and incidents informing the generation of accident scenarios within the simulation environment. A kinematic motion prediction model, coupled with a deep learning architecture, is instrumental in establishing classifiers that distinguish between normal conditions and accident patterns. Evaluations conducted on a simulated test dataset have demonstrated considerable promise for accident and incident prediction while maintaining a minimal rate of false positives. The classifier has delivered lead times of up to 12 s before the precipitating event, facilitating adequate warnings for emergency braking in 80 % of the ground collision cases and 97 % of the scenarios involving infringements of holding restrictions within a test dataset. This result demonstrates a transformative potential for real-world applications, setting a new benchmark for preemptive measures in airport ground surveillance.

Η τεχνολογία της αναγνώρισης και παρακολούθησης αεροσκαφών βρίσκει ποικίλες εφαρμογές σε όλους τους τομείς της αεροναυσιπλοΐας, πολιτικούς και στρατιωτικούς, από τον έλεγχο και τη ρύθμιση της εναέριας κυκλοφορίας σε πολιτικά αεροδρόμια έως το χειρισμό και την καθοδήγηση αντιαεροπορικών όπλων για στρατιωτικούς σκοπούς (π.χ. το σύστημα TAS του αντιαεροπορικού συστήματος MIM-23B Hawk). Έως σήμερα, γνωστές μέθοδοι υλοποίησης αποτελούν οι ραδιοεντοπιστές (radar), οι υπέρυθρες και οι θερμικές κάμερες, τα οποία είναι εγκατεστημένα σε επίγειους σταθμούς, σε κινούμενες μονάδες και σε αεροσκάφη. Το σύστημα που δημιουργήθηκε και θα παρουσιαστεί στην παρούσα διπλωματική εργασία αποτελεί μια εναλλακτική μέθοδο υλοποίησης της αναγνώρισης και της παρακολούθησης ιπτάμενων αντικειμένων, που εκμεταλλεύεται το οπτικό φάσμα με τη χρήση μιας οπτικής κάμερας ενσωματωμένης σε ένα σερβοκινητήρα. Σε σημεία όπου είναι δύσκολο να εφαρμοσθεί κάποια άλλη τεχνολογία ή σε σημεία που δεν καλύπτονται από άλλες συσκευές ανίχνευσης (π.χ. radar), η συσκευή μας προσφέρει όμοιες υπηρεσίες και συμπληρώνει πιθανά χάσματα ακάλυπτων περιοχών. Συγκεκριμένα, μέσω του λογισμικού που έχει αναπτυχθεί, όταν κάποιος στόχος (αεροσκάφος) εισέλθει στο οπτικό πεδίο της κάμερας, ανιχνεύεται και αναγνωρίζεται. Στη συνέχεια ο σερβοκινητήρας παρακολουθεί τον στόχο τροφοδοτούμενος με δεδομένα της θέσης και της ταχύτητάς του, ενώ βρίσκεται σε συνεχή επικοινωνία με την κάμερα. Όλα τα παραπάνω έχουν αναπτυχθεί ώστε να λειτουργούν σε συνθήκες πραγματικού χρόνου. Παρά την απουσία μιας θεωρητικής παρουσίασης ή μιας ολοκληρωμένης λύσης οπτικής αναγνώρισης και παρακολούθησης αεροσκαφών, η αναζήτηση και μελέτη της διεθνούς βιβλιογραφίας μας έδωσε το θεωρητικό υπόβαθρο για την κατανόηση του προβλήματος και ταυτόχρονα τη δυνατότητα να συνδυάσουμε τεχνικές και μεθόδους για την επίτευξη του στόχου μας. Για την επιτυχή αναγνώριση και παρακολούθηση των στόχων δημιουργήθηκαν διάφορα μοντέλα προσομοίωσης για τον έλεγχο της συμπεριφοράς μεμονομένων χαρακτηριστικών. Συγκεκριμένα, στο υποσύστημα της αναγνώρισης του στόχου μοντελοποιήθηκε αρχικά μια μέθοδος εξαγωγής της θέσης βασισμένη στο χρώμα του στόχου σε περιβάλλον Matlab/Simulink. Στη συνέχεια η ίδια μέθοδος μεταφέρθηκε σε περιβάλλον LabVIEW για να εμπλουτισθεί με διάφορες άλλες μεθόδους βασισμένες σε ένα σύνολο από χαρακτηριστικά που θα αναλυθούν στη συνέχεια. Το τελικό μοντέλο αποτελεί συνδυασμό των μεθόδων του αθροίσματος απολύτων διαφορών, της οπτικής ροής, της εξαγωγής χρωματικών και σχηματικών χαρακτηριστικών, της κανονικοποιημένης εττεροσυσχέτισης και άλλων λογικών μεθόδων και βελτιστοποιήσεων τους. Για την επίτευξη μιας επιτυχυμένης παρακολούθησης ενός “κλειδωμένου” στόχου, δοκιμάστηκαν και έγιναν πολλές προσομοιώσεις με διαφορετικούς τύπους ελεγκτών. Συγκεκριμένα η δυναμική του μοντέλου που δημιουργήθηκε, εξαρτάται από ένα συνδυασμό ελεγκτών θέσεως, ταχύτητας και άλλων παραμέτρων. Αυτά εξασφαλίζουν ένα ευσταθές και γραμμικοποιημένο σύστημα παρακολούθησης, ικανό να παρακολουθήσει οποιοδήποτε στόχο με τη προϋπόθεση ότι τα χαρακτηριστικά του στόχου καθώς και η κατάστασή του (θέση, ταχύτητα κτλ.), ικανοποιούν τις απαιτήσεις του αλγορίθμου αναγνώρισης και είναι μέσα στις εργοστασιακές δυνατότητες του συστήματος. Το μοντέλο αυτό αναπτύχθηκε και υλοποιήθηκε σε περιβάλλον LabVIEW, όπως και οι μετρήσεις και προσομοιώσεις που έγιναν πάνω σε αυτό. Όλες οι παραπάνω μέθοδοι συνεργάζονται και είναι ικανοί να δώσουν ακριβή αποτελέσματα θέσης πραγματικών στόχων κατά τη διάρκεια της ημέρας ακόμα και κάτω από δύσκολες συνθήκες (όπως συννεφιά, χαμηλή φωτεινότητα, παρεμβολή αντικειμένων) σε πραγματικό χρόνο. Η ακραία μεταβολή των περιβαλλοντικών συνθηκών θα επηρρέαζε οποιοδήποτε οπτικό σύστημα, συνεπώς και το παρόν. Περιγραφή των παραγόντων που επηρρεάζουν το σύστημά μας θα γίνει στη συνέχεια. / The technology of aircraft recognition and tracking applies in various applications in all areas of air navigation, civil and military, from air traffic control and regulation at civilian airports to anti-aircraft weapon handling and guidance for military purposes (e.g the TAS system of MIM-23B Hawk anti-aircraft system). To date, known methods of implementation are the radar, infrared and thermal cameras, which are installed at ground stations, in moving plants and aircrafts. The system that was created and is presented in this thesis is an alternative implementation of identifying and tracking flying objects, which operates in the optical spectrum using an optical camera built into a servomotor (pan-tilt unit – PTU). In regions where is difficult for one technology to be applied or in areas that are not covered by other detection devices (e.g. radar), our device offers similar services and complements potential gaps that arise by uncovered areas. Specifically, through the software we developed, when a target (aircraft) enters the field of view of our camera, it is detected and identified. Then the PTU, fed with data of target position and velocity, tracks the aircraft while keeps in constant communication with the camera. All the above have been developed to operate in real time. Despite the lack of a theoretical presentation or a complete solution of optical aircraft recognition and tracking, search and study of literature has given us the theoretical background for understanding the problem and making it possible to combine techniques and methods to achieve our goal. For the successful identification and monitoring of the targets, various simulation models were created to control the behavior of isolated features. Specifically, for the target recognition subsystem a method for extraction of the position based on the color of the target was initially modeled in Matlab/Simulink environment. Then the same method was implemented in LabVIEW to be enriched with several other methods based on a set of features that will be discussed below. The final model is a combination of the sum of absolute differences between two images, the extraction of color and shape profiles, the normalized cross-correlation and other logical methods and their optimizations. In order a successful tracking of a “locked” target to be achieved, there have been many tests and carried out many simulations with different types of controllers. Specifically, the dynamic of the model which was created, depends on a combination of position/velocity controllers and other parameters. These provide a stable and linearized tracking system, capable to follow any target under the condition that the characteristics of the target and its current status (position, speed, etc.) meet the requirements of the recognition algorithm and is within the capabilities of the system. The model was developed and implemented in the LabVIEW environment, as well as measurements and simulations were carried out in it. All these methods work and are able to give accurate results of the position of real targets during the day, even under difficult circumstances (such as clouds, decreased sky brightness etc) in real time. The extreme variation of environmental conditions would affect any optical system and hence could affect ours as well. Description of the factors that affect our system will be presented.

The combination of a video camera with an acoustic vector sensor (AVS) opens new possibilities in environment awareness applications. The goal of this thesis is the design of an algorithm for detection and tracking of low-flying aircraft using a multi-coordinate VAUDEO system. A commercial webcam placed in line with an AVS in a ground array are used to record real low-flying aircraft data at Teuge international airport. Each frame, the algorithm analyzes a matrix of three orthogonal acoustic particle velocity signals and one acoustic pressure signal using the Singular Value Decomposition to estimate the Direction of Arrival, DoA of propeller aircraft sound. The DoA data is then applied to a Kalman filter and its output is used later on to narrow the region of video frame processed. Background subtraction is applied followed by a Gaussian-weighted intensity mask to assign high priority to moving objects which are closer to the sound source estimated position. The output is applied to another Kalman filter to improve the accuracy of the aircraft location estimation. The performance evaluation of the algorithm proved that it is comparable to the performances of state-of-the-art video alone based algorithms. In conclusion, the combination of video and directional audio increases the accuracy of propeller aircraft detection and tracking comparing to reported previous work using audio alone. / +593 980826278