Annotations for 500 of the 690 images used for training.

In recent years, deep learning based computer vision technology has developed rapidly. This is not only due to the improvement of computing power, but also due to the emergence of high-quality datasets. The combination of object detectors and drones has great potential in the field of rescue and disaster relief. We created an image dataset specifically for vision applications on drone platforms. The dataset contains 5000 images, and each image is carefully labeled according to the PASCAL VOC standard. This specific dataset will be very important for developing deep learning algorithms for drone applications. In object detection models, loss function plays a vital role. Considering the uneven distribution of large and small objects in the dataset, we propose adjustment coefficients based on the frequencies of objects of different sizes to adjust the loss function, and finally improve the accuracy of the model.

No description available.

Techniques for improving the information quality present in imagery for feature extraction are proposed in this thesis. Specifically, two methods are presented: soft feature extraction and improved Evolution-COnstructed (iECO) features. Soft features comprise the extraction of image-space knowledge by performing a per-pixel weighting based on an importance map. Through soft features, one is able to extract features relevant to identifying a given object versus its background. Next, the iECO features framework is presented. The iECO features framework uses evolutionary computation algorithms to learn an optimal series of image transforms, specific to a given feature descriptor, to best extract discriminative information. That is, a composition of image transforms are learned from training data to present a given feature descriptor with the best opportunity to extract its information for the application at hand. The proposed techniques are applied to an automatic explosive hazard detection application and significant results are achieved.

In a three-dimensional world, for perception of the objects around us, we not only wish to classify them, but also know where these objects are. The task of object detection combines both classification and localization. In addition to predicting the object category, we also predict where the object is from sensor data. As it is not known ahead of time how many objects that we have interest in are in the sensor data and where are they, the output size of object detection may change, which makes the object detection problem difficult. In this dissertation, I focus on the task of object detection, and use fusion to improve the detection accuracy and robustness. To be more specific, I propose a method to calculate measure of conflict. This method does not need external knowledge about the credibility of each source. Instead, it uses the information from the sources themselves to help assess the credibility of each source. I apply the proposed measure of conflict to fuse independent sources of tracking information from various stereo cameras. Besides, I propose a computational intelligence system for more accurate object detection in real--time. The proposed system uses online image augmentation before the detection stage during testing and fuses the detection results after. The fusion method is computationally intelligent based on the dynamic analysis of agreement among inputs. Comparing with other fusion operations such as average, median and non-maxima suppression, the proposed methods produces more accurate results in real-time. I also propose a multi--sensor fusion system, which incorporates advantages and mitigate disadvantages of each type of sensor (LiDAR and camera). Generally, camera can provide more texture and color information, but it cannot work in low visibility. On the other hand, LiDAR can provide accurate point positions and work at night or in moderate fog or rain. The proposed system uses the advantages of both camera and LiDAR and mitigate their disadvantages. The results show that comparing with LiDAR or camera detection alone, the fused result can extend the detection range up to 40 meters with increased detection accuracy and robustness.

Lightning strokes create powerful electromagnetic pulses that routinely cause very low frequency (VLF) waves to propagate across hemispheres along geomagnetic field lines. VLF antenna receivers can be used to detect these whistler waves generated by these lightning strokes. The particular time/frequency dependence of the received whistler wave enables the estimation of electron density in the plasmasphere region of the magnetosphere. Therefore the identification and characterisation of whistlers are important tasks to monitor the plasmasphere in real time and to build large databases of events to be used for statistical studies. The current state of the art in detecting whistler is the Automatic Whistler Detection (AWD) method developed by Lichtenberger (2009) [1]. This method is based on image correlation in 2 dimensions and requires significant computing hardware situated at the VLF receiver antennas (e.g. in Antarctica). The aim of this work is to develop a machine learning based model capable of automatically detecting whistlers in the data provided by the VLF receivers. The approach is to use a combination of image classification and localisation on the spectrogram data generated by the VLF receivers to identify and localise each whistler. The data at hand has around 2300 events identified by AWD at SANAE and Marion and will be used as training, validation, and testing data. Three detector designs have been proposed. The first one using a similar method to AWD, the second using image classification on regions of interest extracted from a spectrogram, and the last one using YOLO, the current state of the art in object detection. It has been shown that these detectors can achieve a misdetection and false alarm rate, respectively, of less than 15% on Marion's dataset. It is important to note that the ground truth (initial whistler label) for data used in this study was generated using AWD. Moreover, SANAE IV data was small and did not provide much content in the study.

Environment perception within autonomous driving aims to provide a comprehensive and accurate model of the surrounding environment based on information from sensors. For the model to be comprehensive it must provide the kinematic state of surrounding objects. The existing approaches of object detection and tracking (estimation of kinematic state) are developed for dense 3D LiDAR data from a sensor mounted on a car. However, it is a challenge to design a robust detection and tracking algorithm for sparse 3D LiDAR data. Therefore, in this thesis we propose a framework for detection and tracking of unknown objects using sparse VLP-16 LiDAR data which is mounted on a heavy duty vehicle. Experiments reveal that the proposed framework performs well detecting trucks, buses, cars, pedestrians and even smaller objects of a size bigger than 61x41x40 cm. The detection distance range depends on the size of an object such that large objects (trucks and buses) are detected within 25 m while cars and pedestrians within 18 m and 15 m correspondingly. The overall multiple objecttracking accuracy of the framework is 79%. / Miljöperception inom autonom körning syftar till att ge en heltäckande och korrekt modell av den omgivande miljön baserat på information från sensorer. För att modellen ska vara heltäckande måste den ge information om tillstånden hos omgivande objekt. Den befintliga metoden för objektidentifiering och spårning (uppskattning av kinematiskt tillstånd) utvecklas för täta 3D-LIDAR-data från en sensor monterad på en bil. Det är dock en utmaning att designa en robust detektions och spårningsalgoritm för glesa 3D-LIDAR-data. Därför föreslår vi ett ramverk för upptäckt och spårning av okända objekt med hjälp av gles VLP-16-LIDAR-data som är monterat på ett tungt fordon. Experiment visar att det föreslagna ramverket upptäcker lastbilar, bussar, bilar, fotgängare och även mindre objekt om de är större än 61x41x40 cm. Detekteringsavståndet varierar beroende på storleken på ett objekt så att stora objekt (lastbilar och bussar) detekteras inom 25 m medan bilar och fotgängare detekteras inom 18 m respektive 15 m på motsvarande sätt. Ramverkets totala precision för objektspårning är 79%.

Detecting curved objects against cluttered backgrounds is a hard problem in computer vision. We present new low-level and mid-level features to function in these environments. The low-level features are fast to compute, because they employ an integral image approach, which makes them especially useful in real-time applications. The mid-level features are built from low-level features, and are optimized for curved object detection. The usefulness of these features is tested by designing an object detection algorithm using these features. Object detection is accomplished by transforming the mid-level features into weak classifiers, which then produce a strong classifier using AdaBoost. The resulting strong classifier is then tested on the problem of detecting heads with shoulders. On a database of over 500 images of people, cropped to contain head and shoulders, and with a diverse set of backgrounds, the detection rate is 90% while the false positive rate on a database of 500 negative images is less than 2%.

Recent years have seen several projects across the globe using drones to detect sharks, including several high profile projects around alerting beach authorities to keep people safe. However, so far many of these attempts have used cloud-based machine learning solutions for the detection component, which complicates setup and limits their use geographically to areas with internet connection. An on-device (or on-controller) shark detector would offer greater freedom for researchers searching for and tracking sharks in the field, but such a detector would need to operate under reduced resource constraints. To this end we look at SSD MobileNet, a popular object detection architecture that targets edge devices by sacrificing some accuracy. We look at the results of SSD MobileNet in detecting sharks from a data set of aerial images created by a collaboration between Cal Poly and CSU Long Beach’s Shark Lab. We conclude that SSD MobileNet does suffer from some accuracy issues with smaller objects in particular, and we note the importance of customized anchor box configuration.

The significant advancements and breakthroughs achieved in Machine Learning (ML) have revolutionized the field of Computer Vision (CV), where numerous real-world applications are now utilizing state-of-the-art advancements in the field. Advanced video surveillance and analytics, entertainment, and autonomous vehicles are a few examples that rely heavily on reliable and accurate perception systems. Deep learning usage in Computer Vision has come a long way since it sparked in 2012 with the introduction of Alexnet. Convolutional Neural Networks (CNN) have evolved to become more accurate and reliable. This is attributed to the advancements in GPU parallel processing, and to the recent availability of large scale and high quality annotated datasets that allow the training of complex models. However, ML models can only be as good as the data they train on and the data they receive in production. In real-world environments, a perception system often needs to be able to operate in different environments and conditions (weather, lighting, obstructions, etc.). As such, it is imperative for a perception system to utilize information from different types of sensors to mitigate the limitations of individual sensors. In this dissertation, we focus on studying the efficacy of using thermal sensors to enhance the robustness of perception systems. We focus on two common vision tasks: object detection and multiple object tracking. Through our work, we prove the viability of thermal sensors as a complement, and in some scenarios a replacement, to RGB cameras. For their important applications in autonomous vehicles and surveillance, we focus our research on pedestrian and vehicle perception. We also introduce the world's first (to the best of our knowledge) large scale dataset for pedestrian detection and tracking including thermal and corresponding RGB images.