1 Title: Convolutional neural networks and their application in object detection Author: Matej Hrinčár Department: Department of Theoretical Computer Science and Mathematical Logic Supervisor: doc. RNDr. Iveta Mrázová, CSc. Supervisor's e-mail address: Iveta.Mrazova@mff.cuni.cz Abstract: Nowadays, it has become popular to enhance live sport streams with an augmented reality like adding various statistics over the hockey players. To do so, players must be automatically detected first. This thesis deals with such a challenging task. Our aim is to deliver not only a sufficient accuracy but also a speed because we should be able to make the detection in real time. We use one of the newer model of neural network which is a convolutional network. This model is suitable for proces- sing image data a can use input image without any preprocessing whatsoever. After our detailed analysis we choose this model as a detector for hockey players. We have tested several different architectures of the networks which we then compared and choose the one which is not only accurate but also fast enough. We have also tested the robustness of the network with noisy patterns. Finally we assigned detected pla- yers to their corresponding teams utilizing K-mean algorithm using the information about their jersey color. Keywords:...

Alex Krizhevsky and his colleagues changed the world of machine vision and image processing in 2012 when their deep learning model, named Alexnet, won the Im- ageNet Large Scale Visual Recognition Challenge with more than 10.8% lower error rate than their closest competitor. Ever since, deep learning approaches have been an area of extensive research for the tasks of object detection, classification, pose esti- mation, etc...This thesis presents a comprehensive analysis of different deep learning models and architectures that have delivered state of the art performances in various machine vision tasks. These models are compared to each other and their strengths and weaknesses are highlighted. We introduce a new approach for human head and shoulder detection from RGB- D data based on a combination of image processing and deep learning approaches. Candidate head-top locations(CHL) are generated from a fast and accurate image processing algorithm that operates on depth data. We propose enhancements to the CHL algorithm making it three times faster. Different deep learning models are then evaluated for the tasks of classification and detection on the candidate head-top loca- tions to regress the head bounding boxes and detect shoulder keypoints. We propose 3 different small models based on convolutional neural networks for this problem. Experimental results for different architectures of our model are highlighted. We also compare the performance of our model to mobilenet. Finally, we show the differences between using 3 types of inputs CNN models: RGB images, a 3-channel representation generated from depth data (Depth map, Multi-order depth template, and Height difference map or DMH), and a 4 channel input composed of RGB+D data.

No description available.

Context. Image enhancement algorithms can be used to enhance the visual effects of images in the field of human vision. So can image enhancement algorithms be used in the field of computer vision? The convolutional neural network, as the most powerful image classifier at present, has excellent performance in the field of image recognition. This paper explores whether image enhancement algorithms can be used to improve the performance of convolutional neural networks. Objectives. The purpose of this paper is to explore the effect of image enhancement algorithms on the performance of CNN models in deep learning and transfer learning, respectively. The article selected five different image enhancement algorithms, they are the contrast limited adaptive histogram equalization (CLAHE), the successive means of the quantization transform (SMQT), the adaptive gamma correction, the wavelet transform, and the Laplace operator. Methods. In this paper, experiments are used as research methods. Three groups of experiments are designed; they respectively explore whether the enhancement of grayscale images can improve the performance of CNN in deep learning, whether the enhancement of color images can improve the performance of CNN in deep learning and whether the enhancement of RGB images can improve the performance of CNN in transfer learning?Results. In the experiment, in deep learning, when training a complete CNN model, using the Laplace operator to enhance the gray image can improve the recall rate of CNN. However, the remaining image enhancement algorithms cannot improve the performance of CNN in both grayscale image datasets and color image datasets. In addition, in transfer learning, when fine-tuning the pre-trained CNN model, using contrast limited adaptive histogram equalization (CLAHE), successive means quantization transform (SMQT), Wavelet transform, and Laplace operator will reduce the performance of CNN. Conclusions. Experiments show that in deep learning, using image enhancement algorithms may improve CNN performance when training complete CNN models, but not all image enhancement algorithms can improve CNN performance; in transfer learning, when fine-tuning the pre- trained CNN model, image enhancement algorithms may reduce the performance of CNN.

Visual object detection is a popular computer vision task that has been intensively investigated using deep learning on real data. However, data from virtual environments have not received the same attention. A virtual environment enables generating data for locations that are not easily reachable for data collection, e.g. aerial environments. In this thesis, we study the problem of object detection in virtual environments, more specifically an aerial virtual environment. We use a simulator, to generate a synthetic data set of 16 different types of vehicles captured from an airplane. To study the performance of existing methods in virtual environments, we train and evaluate two state-of-the-art detectors on the generated data set. Experiments show that both detectors, You Only Look Once version 3 (YOLOv3) and Single Shot MultiBox Detector (SSD), reach similar performance quality as previously presented in the literature on real data sets. In addition, we investigate different fusion techniques between detectors which were trained on two different subsets of the dataset, in this case a subset which has cars with fixed colors and a dataset which has cars with varying colors. Experiments show that it is possible to train multiple instances of the detector on different subsets of the data set, and combine these detectors in order to boost the performance.

Continuous authentication using behavioral biometrics can provide an additional layer of protection against online account hijacking and fraud. Mouse dynamics classification is the concept of determining the authenticity of a user through the use of machine learning algorithms on mouse movement data. This thesis investigates the viability of state of the art deep learning technologies in mouse dynamics classification by designing convolutional neural network classifiers taking mouse movement images as input. For purposes of comparison, classifiers using the random forest algorithm and engineered features inspired by related works are implemented and tested on the same data set as the neural network classifier. A technique for lowering bias toward the on-screen location of mouse movement images is introduced, although its effectiveness is questionable and requires further research to thoroughly investigate. This technique was named 'centering', and is used for the deep learning-based classification methods alongside images not using the technique. The neural network classifiers yielded single action classification accuracies of 66% for centering, and 78% for non-centering. The random forest classifiers achieved the average accuracy of 79% for single action classification, which is very close to the results of other studies using similar methods. In addition to single action classification, a set based classification is made. This is the method most suitable for implementation in an actual authentication system as the accuracy is much higher. The neural network and random forest classifiers have different strengths. The neural network is proficient at classifying mouse actions that are of similar appearance in terms of length, location, and curvature. The random forest classifiers seem to be more consistent in these regards, although the accuracy deteriorates for especially long actions. As the different classification methods in this study have different strengths and weaknesses, a composite classification experiment was made where the output was determined by the least ambiguous output of the two models. This composite classification had an accuracy of 83%, meaning it outperformed both the individual models.

Segmentation of mammograms pertains to assigning a meaningful label to each pixel found in the image. The segmented mammogram facilitates both the function of Computer Aided Diagnosis Systems and the development of tools used by radiologists during examination. Over the years many approaches to this problem have been presented. A surge in the popularity of new methods to image processing involving deep neural networks present new possibilities in this domain, and this thesis evaluates mammogram segmentation as an application of a specialized neural network architecture, U-net. Results are produced on publicly available datasets mini-MIAS and CBIS-DDSM. Using these two datasets together with mammograms from Hologic and FUJI, instances of U-net are trained and evaluated within and across the different datasets. A total of 10 experiments are conducted using 4 different models. Averaged over classes Pectoral, Breast and Background the best Dice scores are: 0.987 for Hologic, 0.978 for FUJI, 0.967 for mini-MIAS and 0.971 for CBIS-DDSM.

While railroad trespassing is a dangerous activity with significant security and safety risks, regular patrolling of potential trespassing sites is infeasible due to exceedingly high resource demands and personnel costs. There is thus a need to design an automated trespass detection and early warning prediction tool leveraging state-of-the-art machine learning techniques. Leveraging video surveillance through security cameras, this thesis designs a novel approach called ARTS (Automated Railway Trespassing detection System) that tackles the problem of detecting trespassing activity. In particular, we adopt a CNN-based deep learning architecture (Faster-RCNN) as the core component of our solution. However, these deep learning-based methods, while effective, are known to be computationally expensive and time consuming, especially when applied to a large amount of surveillance data. Given the sparsity of railroad trespassing activity, we design a dual-stage deep learning architecture composed of an inexpensive prefiltering stage for activity detection followed by a high fidelity trespass detection stage for robust classification. The former is responsible for filtering out frames that show little to no activity, this way reducing the amount of data to be processed by the later more compute-intensive stage which adopts state-of-the-art Faster-RCNN to ensure effective classification of trespassing activity. The resulting dual-stage architecture ARTS represents a flexible solution capable of trading-off performance and computational time. We demonstrate the efficacy of our approach on a public domain surveillance dataset.

Understanding road surface conditions is an important component in active vehicle safety. Estimations can be achieved through image classification using increasingly popular convolutional neural networks (CNNs). In this paper, we explore the effects of multi-task learning by creating CNNs capable of simultaneously performing the two tasks road surface condition classification (RSCC) and road scene semantic segmentation (RSSS). A multi-task network, containing a shared feature extractor (VGG16, ResNet-18, ResNet-101) and two taskspecific network branches, is built and trained using the Road-Conditions and Cityscapes datasets. We reveal that utilizing task-dependent homoscedastic uncertainty in the learning process improvesmulti-task model performance on both tasks. When performing task adaptation, using a small set of additional data labeled with semantic information, we gain considerable RSCC improvements on complex models. Furthermore, we demonstrate increased model generalizability in multi-task models, with up to 12% higher F1-score compared to single-task models.

<div>Convolutional neural networks lie at the heart of nearly every object recognition system today. While their performance continues to improve through new architectures and techniques, some of their deciencies have not been fully addressed to date. Two of these deciencies are their inability to distinguish the spatial relationships between features taken from the data, as well as their need for a vast amount of training data. Capsule networks, a new type of convolutional neural network, were designed specically to address these two issues. In this work, several capsule network architectures are utilized to classify objects taken from overhead satellite imagery. These architectures are trained and tested on small datasets that were constructed from the xView dataset, a comprehensive collection of satellite images originally compiled for the task of object detection. Since the objects in overhead satellite imagery are taken from the same viewpoint, the transformations exhibited within each individual object class consist primarily of rotations and translations. These spatial relationships are exploited by capsule networks. As a result it is shown that capsule networks achieve considerably higher accuracy when classifying images from these constructed datasets than a traditional convolutional neural network of approximately the same complexity.</div>