The automation of navigation for different kinds of vehicles is a research problem of great interest. This problem has applications with unmanned aerial vehicles (UAVs) as well as manned vehicles such as cars and planes. The goal of an autonomous vehicle is to navigate safely from one point to another given a set of high-level instructions and data from a set of sensors. This thesis explores an implementation of a modular approach for autonomously driving taxiing planes before proposing methods for object tracking using a LIDAR sensor which can be incorporated into the autonomous driving pipeline. The taxiing algorithm regresses waypoints for the plane to follow given a high-level driving goal such as ”turn left” or ”go straight”, along with RGB images taken from the cockpit and wings. Waypoints are then used with a separate control system to taxi the plane. The training and testing of this autonomous aircraft is done in a photo-realistic simulator which has been adapted for this task. The policy developed in this fashion is capable of learning how to go straight and how to turn. However, the driving policy is not trained to react to other moving objects. To address this issue, and due to the superior reliability of LIDAR over RGB sensors, an object tracking method using only LIDAR point clouds is proposed. The proposed method uses a novel 3D Siamese network to obtain a similarity score between a model and candidate object point clouds. This similarity score is shown to work for tracking by applying it using an exhaustive search and obtaining improved performances when compared with simple baselines. For a realistic application, the similarity score is applied using candidates provided by a search on the BEV projection of the LIDAR point cloud. This method is shown to provide improved tracking results over other search strategies when using a lower number of candidates.

In this document, a low-cost, portable, non-invasive method of collecting the 3D trajectories of flying bats is first presented. An array of commercially available camera and light components is used alongside a number of well-established calibration and triangulation techniques to resolve the motion of agents through a 3D volume. It is shown that this system is capable of accurately capturing the bats' flight paths in a field experiment. The use of non-visible illumination ensures that a natural cave environment is disturbed as little as possible for behavioral experiments. Following is a transfer entropy analysis approach applied to the 3D paths of bats flying in pairs. The 3D trajectories are one-dimensionally characterized as inverse curvature time series to allow for entropy calculations. In addition to a traditional formulation of information flow between pair members, a path coupling hypothesis is pursued with time-delay modifications implemented in such a way as to not change the Markovianity of the process. With this modification, trends are found that suggest a leader-follower interaction between the front bat and the rear bat, although statistical significance is not reached due to the small number of pairs considered. / Master of Science

Zusammenfassung Heutige Ansätze des 3D-Trackings für die Registrierung in der realen Welt zum Einsatz von Augmented Reality lassen sich in modellbasierte und umgebungsbasierte Verfahren unterteilen. Umgebungsbasierte Verfahren nutzen den SLAM-Algorithmus zur Erzeugung dreidimensionaler Punktwolken der Umgebung in Echtzeit. Modellbasierte Verfahren finden Ihren Ursprung im Canny edge detector und nutzen aus den CAD-Modellen abgeleitete Kantenmodelle. Wird das modellbasierte Verfahren über Kantendetektion und das umgebungsbasierte Verfahren über 3DPunktewolken kombiniert, ergibt sich ein robustes, hybrides 3D-Tracking. Die entsprechenden Algorithmen der verschiedenen Verfahren sind in heute verfügbaren AR-Frameworks bereits implementiert. Der vorliegende Betrag zeigt zwar, welche Effizienz das hybride 3D-Tracking aufweist, jedoch auch die Problematik der erforderlichen geometrischen Ähnlichkeit von idealem CAD-Modell, bzw. Kantenmodell, und realem Objekt. Bei unterschiedlichen Montagestufen an verschiedenen Montagestationen und mit wechselnden Anwendern ist beispielsweise eine erneute Initialisierung erforderlich. Somit bedingt das hybride 3D-Tracking zahlreiche Kantenmodell, die zuvor aus der jeweiligen Montagestufe abgeleitet werden müssen. Hinzu kommen geometrische Abweichungen durch die Fertigung, die je nach Größe der branchenspezifischen Toleranzen keine hinreichend hohe Übereinstimmung mit den abgeleiteten Kantenmodellen aus den idealen CAD-Modellen aufweisen. Die Autoren schlagen daher den Einsatz parametrisch aufgebauter Mastermodelle vor, welche durch eine interaktive Initialisierung geometrisch Instanziiert werden. Zum Einsatz kommt hier ein mobiler Tiefensensor für Smart Devices, welcher mit Hilfe des Anwenders eine Relation der realen geometrischen Merkmale mit den Idealen des CAD-Modells ermöglicht. Des Weiteren wird in dem dargestellten Konzept die Nutzung von speziellen Suchalgorithmen basierend auf geometrischen Ähnlichkeiten vorgeschlagen, sodass eine Registrierung und Instanziierung auch ohne hinterlegtes Mastermodell ermöglicht wird. Der Beitrag fokussiert sich bei der Validierung auf die interaktive Initialisierung anhand eines konkreten anwendungsnahen Beispiels, da die Initialisierung die Grundlage für die weitere Entwicklung des Gesamtkonzeptes darstellt.

This thesis develops and describes methods for real-time tracking, segmentation and 3-dimensional (3D) model acquisition, in the context of developing games for stroke patients that are rehabilitating at home. Real-time tracking and reconstruction of a stroke patient's feet, hands and the control objects that they are touching can enable not only the graphical visualization of the virtual avatar in the rehabilitation games, but also permits measurement of the patient's performs. Depth or combined colour and depth imagery from a Kinect sensor is used as input data. The 3D signed distance function (SDF) is used as implicit shape representation, and a series of probabilistic graphical models are developed for the problem of model-based 3D tracking, simultaneous 3D tracking and reconstruction and 3D tracking of multiple objects with identical appearance. The work is based on the assumption that the observed imagery is generated jointly by the pose(s) and the shape(s). The depth of each pixel is randomly and independently sampled from the likelihood of the pose(s) and the shape(s). The pose(s) tracking and 3D shape reconstruction problems are then cast as the maximum likelihood (ML) or maximum a posterior (MAP) estimate of the pose(s) or 3D shape. This methodology first leads to a novel probabilistic model for tracking rigid 3D objects with only depth data. For a known 3D shape, optimization aims to find the optimal pose that back projects all object region pixels onto the zero level set of the 3D shape, thus effectively maximising the likelihood of the pose. The method is extended to consider colour information for more robust tracking in the presence of outliers and occlusions. Initialised with a coarse 3D model, the extended method is also able to simultaneously reconstruct and track an unknown 3D object in real time. Finally, the concept of `shape union' is introduced to solve the problem of tracking multiple 3D objects with identical appearance. This is formulated as the minimum value of all SDFs in camera coordinates, which (i) leads to a per-pixel soft membership weight for each object thus providing an elegant solution for the data association in multi-target tracking and (ii) it allows for probabilistic physical constraints that avoid collisions between objects to be naturally enforced. The thesis also explore the possibility of using implicit shape representation for online shape learning. We use the harmonics of 2D discrete cosine transform (DCT) to represent 2D shapes. High frequency harmonics are decoupled from low ones to represent the coarse information and the details of the 2D shape. A regression model is learnt online to model the relationship between the high and low frequency harmonics using Locally Weighted Projection Regression (LWPR). We have demonstrated that the learned regression model is able to detect occlusion and recover them to the complete shape.

In this thesis 3D diamond detectors were fabricated using an ultrafast femtosecond (120 fs) pulse length laser, with a 800nm wavelength, to induce a phase change of diamond to graphite to form electrodes in the diamond bulk. Graphitic electrodes, with diameters of O(um), were fabricated using a known processing technique and were enhanced further through the use of a Spatial Light Modulator (SLM), which is a new technology in this field. These detectors were subsequently characterised through the use of particle beams, and this work also presents methods for characterising such detectors: A pair of crossed polarisers to determine the stress induced by the electrodes on the diamond bulk; Raman spectroscopy to assess the relative quantity of diamond:graphite formed; Scanning Electron Microscopy (SEM) to image the starting (seed) and finishing (exit) sides of electrode formation; and current-voltage (I-V) measurements to calculate the electrical properties of the electrodes. These characterisation methods (alongside the use of particle beams) serve as a means to compare the two fabrication techniques and to determine the optimum fabrication parameters to produce 3D diamond detectors for use as tracking detectors in high luminosity environments such as those in the Large Hadron Collider (LHC). This work shows that using a higher beam energy and translation speed of the focal spot results in electrodes of lower electrical resistivity, which is an ideal characteristic for a tracking detector. These higher processing parameters also result in more graphitic structure on the seed and exit sides of the diamond, determined separately via Raman spectroscopy and SEM. An increased beam energy also results in larger electrode diameters, reducing the active area of the detector and inducing more stress in the diamond bulk. These measurements therefore indicate an upper limit on the fabrication parameters. A further study into these processing parameters shows the translation speed scales with the pulse repetition rate of the laser and allows for fast fabrication of 3D diamond detectors. Two devices were fabricated with and without the use of an SLM, with a more uniform detector response (through characterisation by particle beams), lower electrical resistivity, and more graphitic material observed for SLM-fabricated electrodes. The benefits of square and hexagonal cell structures were also investigated with both structures showing a similar response to particle beams. A lower charge sharing region is observed in hexagonal cells and indicates potentially different applications for these cell geometries. Transient Current Technique (TCT) measurements were also taken on both detectors, where faster charge collection and higher quality data were seen for the SLM-fabricated device. These measurements indicate a preference in the use of an SLM for the future fabrication of 3D diamond tracking detectors. These TCT measurements were then compared to simulations to extract the charge carrier properties in diamond. Only qualitative agreement was obtained, motivating further work in this area to fully understand the charge carrier dynamics and demonstrate the future viability of 3D diamond detectors.

Zusammenfassung Heutige Ansätze des 3D-Trackings für die Registrierung in der realen Welt zum Einsatz von Augmented Reality lassen sich in modellbasierte und umgebungsbasierte Verfahren unterteilen. Umgebungsbasierte Verfahren nutzen den SLAM-Algorithmus zur Erzeugung dreidimensionaler Punktwolken der Umgebung in Echtzeit. Modellbasierte Verfahren finden Ihren Ursprung im Canny edge detector und nutzen aus den CAD-Modellen abgeleitete Kantenmodelle. Wird das modellbasierte Verfahren über Kantendetektion und das umgebungsbasierte Verfahren über 3DPunktewolken kombiniert, ergibt sich ein robustes, hybrides 3D-Tracking. Die entsprechenden Algorithmen der verschiedenen Verfahren sind in heute verfügbaren AR-Frameworks bereits implementiert. Der vorliegende Betrag zeigt zwar, welche Effizienz das hybride 3D-Tracking aufweist, jedoch auch die Problematik der erforderlichen geometrischen Ähnlichkeit von idealem CAD-Modell, bzw. Kantenmodell, und realem Objekt. Bei unterschiedlichen Montagestufen an verschiedenen Montagestationen und mit wechselnden Anwendern ist beispielsweise eine erneute Initialisierung erforderlich. Somit bedingt das hybride 3D-Tracking zahlreiche Kantenmodell, die zuvor aus der jeweiligen Montagestufe abgeleitet werden müssen. Hinzu kommen geometrische Abweichungen durch die Fertigung, die je nach Größe der branchenspezifischen Toleranzen keine hinreichend hohe Übereinstimmung mit den abgeleiteten Kantenmodellen aus den idealen CAD-Modellen aufweisen. Die Autoren schlagen daher den Einsatz parametrisch aufgebauter Mastermodelle vor, welche durch eine interaktive Initialisierung geometrisch Instanziiert werden. Zum Einsatz kommt hier ein mobiler Tiefensensor für Smart Devices, welcher mit Hilfe des Anwenders eine Relation der realen geometrischen Merkmale mit den Idealen des CAD-Modells ermöglicht. Des Weiteren wird in dem dargestellten Konzept die Nutzung von speziellen Suchalgorithmen basierend auf geometrischen Ähnlichkeiten vorgeschlagen, sodass eine Registrierung und Instanziierung auch ohne hinterlegtes Mastermodell ermöglicht wird. Der Beitrag fokussiert sich bei der Validierung auf die interaktive Initialisierung anhand eines konkreten anwendungsnahen Beispiels, da die Initialisierung die Grundlage für die weitere Entwicklung des Gesamtkonzeptes darstellt.

Thesis (MSc (Mathematics))--University of Stellenbosch, 2010. / ENGLISH ABSTRACT: Extracting data from video streams and using the data to better understand the observed world allows many systems to automatically perform tasks that ordinarily needed to be completed by humans. One such problem with a wide range of applications is that of detecting and tracking people in a video sequence. This thesis looks speci cally at the problem of estimating the positions of players on a sports eld, as observed by a multi-view camera setup. Previous attempts at solving the problem are discussed, after which the problem is broken down into three stages: detection, 2D tracking and 3D position estimation. Possible solutions to each of the problems are discussed and compared to one another. Motion detection is found to be a fast and e ective solution to the problem of detecting players in a single view. Tracking players in 2D image coordinates is performed by implementing a hierarchical approach to the particle lter. The hierarchical approach is chosen as it improves the computational complexity without compromising on accuracy. Finally 3D position estimation is done by multiview, forward projection triangulation. The components are combined to form a full system that is able to nd and locate players on a sports eld. The overall system that is developed is able to detect, track and triangulate player positions. The components are tested individually and found to perform well. By combining the components and introducing feedback between them the results of the individual components as well as those of the overall system are improved. / AFRIKAANSE OPSOMMING: Deur data uit 'n video-stroom te ontrek, en die data te gebruik om die wêreld wat waargeneem word beter te verstaan, kan baie rekenaarstelsels take outomaties voltooi wat voorheen deur 'n mens sou gedoen moes word. Een so 'n probleem wat 'n wye toepassingsveld het, is om mense te vind en te volg in 'n video. Hierdie tesis kyk spesi ek daarna om die posisie van spelers op 'n sportveld te vind, gegee 'n klomp kameras wat na die veld kyk. Daar word na vorige stelsels wat hierdie probleem probeer oplos gekyk, waarna die probleem in drie dele opgedeel word: vind die spelers, volg die spelers in 2D en skat die posisie van die spelers in 3D. Moontlike oplossings vir elk van hierdie dele word bespreek en vergelyk met mekaar. Daar word gevind dat om beweging te identi seer 'n eenvoudige manier is om die spelers te vind. Hulle word dan gevolg in 2D beeldkoördinate deur gebruik te maak van 'n hiërargiese implementasie van die partikel- lter. Die hiërargiese implementering word gekies omdat dit die spoed van die partikel- lter verbeter, sonder om die akkuraatheid te verswak. Laastens word die 3D posisie gevind deur multi-sigpunt, voorwaartse projeksie triangulering. Die verskillende komponente word kombineer om 'n volledige stelsel te vorm wat spelers kan vind en plaas op 'n veld. Die volledige stelsel wat ontwikkel is, is in staat om spelers te vind, volg en hulle posisies te bepaal. Elk van die individuele komponente word getoets, en daar word gevind dat hulle goed op hulle eie werk. Deur die komponente te kombineer en terugvoer tussen verskillende komponente te bewerkstellig word die resultate van die individuele komponente, sowel as dié van die volledige stelsel nog verbeter.

In surgical education, two of the current primary assessment tools for technical skills are the OSATS checklist and electromagnetic (EM) hand tracking. There are few bench models available for open procedures. The first two chapters of this thesis introduce these concepts further and provide detailed background knowledge. Chapter 3 explores interference that may from the concurrent use of EM tracking and cautery, finding that monopolar cautery interferes, while bipolar cautery does not. Chapter 4 discusses the validation of an assessment tool for open cholecystectomies (OCs) consisting of a porcine bench model and a Wii remote based infrared (IR) hand tracking system. The assessment tool is found to have construct validity and the face validity of the OC model is established. Chapter 5 concludes the thesis and presents several avenues of future research for the improvement of both the OC model and Wii remote based hand tracking system.

In surgical education, two of the current primary assessment tools for technical skills are the OSATS checklist and electromagnetic (EM) hand tracking. There are few bench models available for open procedures. The first two chapters of this thesis introduce these concepts further and provide detailed background knowledge. Chapter 3 explores interference that may from the concurrent use of EM tracking and cautery, finding that monopolar cautery interferes, while bipolar cautery does not. Chapter 4 discusses the validation of an assessment tool for open cholecystectomies (OCs) consisting of a porcine bench model and a Wii remote based infrared (IR) hand tracking system. The assessment tool is found to have construct validity and the face validity of the OC model is established. Chapter 5 concludes the thesis and presents several avenues of future research for the improvement of both the OC model and Wii remote based hand tracking system.

Augmented Reality (AR) has become a popular area in computer Science where research studies and technological innovations are extensive. Research in AR first began in the early 1990s and thenceforth, a number of dierent tracking algorithms and methods have been developed. Tracking systems have a critical importance for AR and marker based vision tracking systems became the mostly used tracking systems due to their low cost and ease of use. Basically, marker systems consist of special patterns that are placed in the environment and detected by simple cameras. In this thesis, we propose a new marker system based on topological tracking where markers are detected and identified using their topology trees. In our proposed marker system, we create topology trees of markers as region adjacency graphs that are obtained from binary images of the markers. Similarly, camera frames are also converted into binary images and corresponding topology trees are created. We used left heavy depth sequences as the canonical convention for the topology trees. For marker tracking, we used a simplified version of subgraph isomorphism algorithm that searches marker topology trees in a frame topology tree. Finally, we tested our proposed system for performance in using spatially distinct marker parts / occlusion resolving / detection rate with respect to marker size, camera-marker angle, false positive marker detection / performance with respect marker library size. Our system achieved 90% marker detection success with 50 pixels marker size and an average of 1.1 false positive marker detection in ten dierent test videos. We made all tests in comparison with the widely used ARToolkit library. Our system surpasses the ARToolkit library for all tests performed. In addition, our system enables spatially distinct placement of marker parts and permits occlusion unless the topology of the marker is not corrupted, but the ARToolkit library does not have these features.