Machine vision is hot research topic with findings being published at a high pace and more and more companies currently developing automated vehicles. Robotic lawn mowers are also increasing in popularity but most mowers still use relatively simple methods for cutting the lawn. No previous work has been published on machine learning networks that improved between cutting sessions by automatically collecting data and then used it for training. A data acquisition pipeline and neural network architecture that could help the mower in avoiding collision was therefor developed. Nine neural networks were tested of which a convolutional one reached the highest accuracy. The performance of the data acquisition routine and the networks show that it is possible to design a object detection model that improves between runs.

For human-robot interaction with the intent to grip a human arm, it is necessary that the ideal gripping location can be identified. In this work, the gripping location is situated on the arm and thus it can be extracted using the position of the wrist and elbow joints. To achieve this human pose estimation is proposed as there exist robust methods that work both in and outside of lab environments. One such example is OpenPose which thanks to the COCO and MPII datasets has recorded impressive results in a variety of different scenarios in real-time. However, most of the images in these datasets are taken from a camera mounted at chest height on people that for the majority of the images are oriented upright. This presents the potential problem that prone humans which are the primary focus of this project can not be detected. Especially if seen from an angle that makes the human appear upside down in the camera frame. To remedy this two different approaches were tested, both aimed at creating a rotation-invariant 2D pose estimation method. The first method rotates the COCO training data in an attempt to create a model that can find humans regardless of orientation in the image. The second approach adds a RotationNet as a preprocessing step to correctly orient the images so that OpenPose can be used to estimate the 2D pose before rotating back the resulting skeletons.

This master thesis studies the learning of dense feature descriptors where camera poses are the only supervisory signal. The use of camera poses as a supervisory signal has only been published once before, and this thesis expands on this previous work by utilizing a couple of different techniques meant increase the robustness of the method, which is particularly important when not having access to ground-truth correspondences. Firstly, an adaptive robust loss is utilized to better differentiate inliers and outliers. Secondly, statistical properties during training are both enforced and adapted to, in an attempt to alleviate problems with uncertainties introduced by not having true correspondences available. These additions are shown to slightly increase performance, and also highlights some key ideas related to prediction certainty and robustness when working with camera poses as a supervisory signal. Finally, possible directions for future work are discussed.

The up to date mapping data is of great importance in social services and disaster relief as well as in city planning. The vast amounts of data and the constant increase of geographical changes lead to large loads of continuous manual analysis. This thesis takes the process of updating maps and breaks it down to the problem of discovering buildings by comparing different machine learning methods to automate the finding of buildings. The chosen methods, YOLOv3 and Mask R-CNN, are based on Region Convolutional Neural Network(R-CNN) due to their capabilities of image analysis in both speed and accuracy. The image data supplied by Lantmäteriet makes up the training and testing data; this data is then used by the chosen machine learning methods. The methods are trained at different time limits, the generated models are tested and the results analysed. The results lay ground for whether the model is reasonable to use in a fully or partly automated system for updating mapping data from aerial imagery. The tested methods showed volatile results through their first hour of training, with YOLOv3 being more so than Mask R-CNN. After the first hour and until the eight hour YOLOv3 shows a higher level of accuracy compared to Mask R-CNN. For YOLOv3, it seems that with more training, the recall increases while precision decreases. For Mask R-CNN, however, there is some trade-off between the recall and precision throughout the eight hours of training. While there is a 90 % confidence interval that the accuracy of YOLOv3 is decreasing for each hour of training after the first hour, the Mask R-CNN method shows that its accuracy is increasing for every hour of training,however, with a low confidence and can therefore not be scientifically relied upon. Due to differences in setups the image size varies between the methods, even though they train and test on the same areas; this results in a fair evaluation where YOLOv3 analyses one square kilometre 1.5 times faster than the Mask R-CNN method does. Both methods show potential for automated generation of footprints, however, the YOLOv3 method solely generates bounding boxes, leaving the step of polygonization to manual work while the Mask R-CNN does, as the name implies, create a mask of which the object is encapsulated. This extra step is thought to further automate the manual process and with viable results speed up the updating of map data. / Uppdaterad kartdata är av stor betydelse för sociala tjänster och katastrofhjälp såväl som inom stadsplanering. De enorma mängderna data och den ständiga ökningen av geografiska förändringar leder till mycket arbete för kontinuerlig manuell analys. Denna avhandling kommer att behandla detta problem med att uppdatera kartor, bryta ned det till det specifika problemet att upptäcka byggnader och ur den synvinkelen jämföra olika maskininlärningsmetoder för automatisera detektering av byggnader. De valda metoderna, YOLOv3 och Mask R-CNN, är baserade på Region Convolutional Neural Network (R-CNN) på grund av dess förmåga av bildanalys i både hastighet och träffsäkerhet. Bildmaterial från Lantmäteriet utgör tränings- och testdatan, denna data används sedan av de utvalda maskininlärningmetoderna. Metoderna tränas med olika tidsgränser och de genererade modellerna testas och resultaten analyseras. Resultaten lägger grund för huruvida modellen är rimlig att använda i ett helt eller delvis automatiserat system för uppdatering av kartdata från flygbilder. De testade metoderna visade varierande resultat under sin första timmes träning, med YOLOv3 mer så än Mask R-CNN. Efter den första timmen fram till den åttonde timmen visar YOLOv3 en högre nivå av precision jämfört med Mask R-CNN. För YOLOv3 ser det ut som att mer träning ökar recall samtidigt som precision minskar. För Mask R-CNN är det emellertid en avvägning mellan recall och precision under de åtta timmarnas träning. Medan det finns en 90 % konfidens att accuracy minskar med YOLOv3 för varje timmes träning efter första timmen så visar Mask R-CNN-metoden att dess accuracy ökar för varje timmes träning, det är dock med låg konfidens och har därmed inte vetenskapligt stöd. På grund av skillnader i konfigurationer varierar bildstorleken mellan metoderna, de tränar och testar dock på samma områden för att ge en rättvis jämförelse. I dessa test analyserar YOLOv3 en kvadratkilometer 1.5 gånger snabbare än Mask R-CNN. Båda metoderna visar potential för en automatiserad generering av footprints. Dock så genererar YOLOv3-metoden endast en bounding box, vilket gör att polygoniseringen återstår för manuellt arbete medan Mask R-CNN, som namnet antyder, skapar en mask som objektet inkapslas i. Detta extrasteg är tänkt att automatisera den manuella processen och med rimliga resultat påskynda uppdateringen av kartdata.

Event-based vision offers high dynamic range, time resolution and lower latency than conventional frame-based vision sensors. These attributes are useful in varying light condition and fast motion. However, there are no neural network models and training protocols optimized for object detection with event data, and conventional artificial neural networks for frame-based data are not directly suitable for that task. Spiking neural networks are natural candidates but further work is required to develop an efficient object detection architecture and end-to-end training protocol. For example, object detection in varying light conditions is identified as a challenging problem for the automation of construction equipment such as earth-moving machines, aiming to increase the safety of operators and make repetitive processes less tedious. This work focuses on the development and evaluation of a neural network for object detection with data from an event-based sensor. Furthermore, the strengths and weaknesses of an event-based vision solution are discussed in relation to the known challenges described in former works on automation of earth-moving machines. A solution for object detection with event data is implemented as a modified YOLOv3 network with spiking convolutional layers trained with a backpropagation algorithm adapted for spiking neural networks. The performance is evaluated on the N-Caltech101 dataset with classes for airplanes and motorbikes, resulting in a mAP of 95.8% for the combined network and 98.8% for the original YOLOv3 network with the same architecture. The solution is investigated as a proof of concept and suggestions for further work is described based on a recurrent spiking neural network.

CNN-based (Convolutional Neural Network) visual object detectors often reach human level of accuracy but need to be trained with large amounts of manually annotated data. Collecting and annotating this data can frequently be time-consuming and financially expensive. Using generative models to augment the data can help minimize the amount of data required and increase detection per-formance. Many state-of-the-art generative models are Generative Adversarial Networks (GANs). This thesis investigates if and how one can utilize image data to generate new data through GANs to train a YOLO-based (You Only Look Once) object detector, and how CAD (Computer-Aided Design) models can aid in this process. In the experiments, different models of GANs are trained and evaluated by visual inspection or with the Fréchet Inception Distance (FID) metric. The data provided by Ericsson Research consists of images of antenna and baseband equipment along with annotations and segmentations. Ericsson Research supplied the YOLO detector, and no modifications are made to this detector. Finally, the YOLO detector is trained on data generated by the chosen model and evaluated by the Average Precision (AP). The results show that the generative models designed in this work can produce RGB images of high quality. However, the quality reduces if binary segmentation masks are to be generated as well. The experiments with CAD input data did not result in images that could be used for the training of the detector. The GAN designed in this work is able to successfully replace objects in images with the style of other objects. The results show that training the YOLO detector with GAN-modified data compared to training with real data leads to the same detection performance. The results also show that the shapes and backgrounds of the antennas contributed more to detection performance than their style and colour.

The Micro Aerial Vehicless (MAVs) are gaining attention in numerous applications asthese platforms are cheap and can do complex maneuvers. Moreover, most of the commer-cially available MAVs are equipped with a mono-camera. Currently, there is an increasinginterest to deploy autonomous mono-camera MAVs with obstacle avoidance capabilitiesin various complex application areas. Some of the application areas have moving obstaclesas well as stationary, which makes it more challenging for collision avoidance schemes.This master thesis set out to investigate the possibility to avoid moving and station-ary obstacles with a single camera as the only sensor gathering information from thesurrounding environment.One concept to perform autonomous obstacle avoidance is to predict the time near-collision based on a Convolution Neural Network (CNN) architecture that uses the videofeed from a mono-camera. In this way, the heading of the MAV is regulated to maximizethe time to a collision, resulting in the avoidance maneuver. Moreover, another interestingperspective is when due to multiple dynamic obstacles in the environment there aremultiple time predictions for different parts of the Field of View (FoV). The method ismaximizing time to a collision by choosing the part with the largest time to collision.However, this is a complicated task and this thesis provides an overview of it whilediscussing the challenges and possible future directions. One of the main reason was thatthe available data set was not reliable and was not provide enough information for theCNN to produce any acceptable predictions.Moreover, this thesis looks into another approach for avoiding collisions, using objectdetection method You Only Lock Once (YOLO) with the mono-camera video feed. YOLOis a state-of-the-art network that can detect objects and produce bounding boxes in real-time. Because of YOLOs high success rate and speed were it chosen to be used in thisthesis. When YOLO detects an obstacle it is telling where in the image the object is,the obstacle pixel coordinates. By utilizing the images FoV and trigonometry can pixelcoordinates be transformed to an angle, assuming the lens does not distort the image.This position information can then be used to avoid obstacles. The method is evaluated insimulation environment Gazebo and experimental verification with commercial availableMAV Parrot Bebop 2. While the obtained results show the efficiency of the method. To bemore specific, the proposed method is capable to avoid dynamic and stationary obstacles.Future works will be the evaluation of this method in more complex environments with multiple dynamic obstacles for autonomous navigation of a team of MAVs. A video ofthe experiments can be viewed at:https://youtu.be/g_zL6eVqgVM.

Unmanned aircraft systems (UASs) are an important future technology with early generations already being used in many areas of application encompassing both military and civilian domains. This thesis proposes a number of integration techniques for combining control-based navigation with more abstract path planning functionality for UASs. These techniques are empirically tested and validated using an RMAX helicopter platform used in the UASTechLab at Linköping University. Although the thesis focuses on helicopter platforms, the techniques are generic in nature and can be used in other robotic systems. At the control level a navigation task is executed by a set of control modes. A framework based on the abstraction of hierarchical concurrent state machines for the design and development of hybrid control systems is presented. The framework is used to specify  reactive behaviors and for sequentialisation of control modes. Selected examples of control systems deployed on UASs are presented. Collision-free paths executed at the control level are generated by path planning algorithms.We propose a path replanning framework extending the existing path planners to allow dynamic repair of flight paths when new obstacles or no-fly zones obstructing the current flight path are detected. Additionally, a novel approach to selecting the best path repair strategy based on machine learning technique is presented. A prerequisite for a safe navigation in a real-world environment is an accurate geometrical model. As a step towards building accurate 3D models onboard UASs initial work on the integration of a laser range finder with a helicopter platform is also presented. Combination of the techniques presented provides another step towards building comprehensive and robust navigation systems for future UASs.

To infer the hidden states from the noisy observations and make predictions based on a set of input states and output observations are two challenging problems in many research areas. Examples of applications many include position estimation from various measurable radio signals in indoor environments, self-navigation for autonomous cars, modeling and predicting of the traffic flows, and flow pattern analysis for crowds of people. In this thesis, we mainly use the Bayesian inference framework for position estimation in an indoor environment, where the radio propagation is uncertain. In Bayesian inference framework, it is usually hard to get analytical solutions. In such cases, we resort to Monte Carlo methods to solve the problem numerically. In addition, we apply Bayesian nonparametric modeling for trajectory learning in sport analytics. The main contribution of this thesis is to propose sequential Monte Carlo methods, namely particle filtering and smoothing, for a novel indoor positioning framework based on proximity reports. The experiment results have been further compared with theoretical bounds derived for this proximity based positioning system. To improve the performance, Bayesian non-parametric modeling, namely Gaussian process, has been applied to better indicate the radio propagation conditions. Then, the position estimates obtained sequentially using filtering and smoothing are further compared with a static solution, which is known as fingerprinting. Moreover, we propose a trajectory learning framework for flow estimation in sport analytics based on Gaussian processes. To mitigate the computation deficiency of Gaussian process, a grid-based on-line algorithm has been adopted for real-time applications. The resulting trajectory modeling for individual athlete can be used for many purposes, such as performance prediction and analysis, health condition monitoring, etc. Furthermore, we aim at modeling the flow of groups of athletes, which could be potentially used for flow pattern recognition, strategy planning, etc.

With the wide application of machine learning technologies in advanced driver assistance systems of vehicles, object classification on obstacles has attracted much attention. Ultrasonic sensors are mainly used to measure the distance to obstacles under the condition of low-speed movement. The existing ultrasonic sensor is low-cost, and its good performance and robustness are sufficient for obstacle avoidance. Recent progress on ultrasonic has attempted to classify obstacles with the combination of ultrasonic sensors and machine learning. It shows that deep neural networks are able to classify objects using only ultrasonic sensors. In the thesis, we focus on the object classification on sizes of obstacles and expect our proposed neural networks model can solve the classification task under the simulation environment, thus contributing to the application of ultrasonic sensors in vehicles. The ultrasonic sensor arrays are built in COMSOL Multiphysics and can provide ultrasonic data with different objects. After many simulation experiments, the ultra-sonic data from objects are labeled and stored in datasets. Then we process the ultrasonic data from datasets and feed them to the proposed neural networks. The ultrasonic data obtained by experiments are examined by the distribution of reflected ultrasonic rays in simulation. The analysis results are in line with our expectations. The trained neural networks are divided into two groups. The first networks group is trained with data from cubes and shows over 80% accuracy on five object categories. The second group of networks is trained with data from cubes and S1 objects. There is an approximate 5% drop in classification performance as the difficulty of the classification task increases.