1 Title: Convolutional neural networks and their application in object detection Author: Matej Hrinčár Department: Department of Theoretical Computer Science and Mathematical Logic Supervisor: doc. RNDr. Iveta Mrázová, CSc. Supervisor's e-mail address: Iveta.Mrazova@mff.cuni.cz Abstract: Nowadays, it has become popular to enhance live sport streams with an augmented reality like adding various statistics over the hockey players. To do so, players must be automatically detected first. This thesis deals with such a challenging task. Our aim is to deliver not only a sufficient accuracy but also a speed because we should be able to make the detection in real time. We use one of the newer model of neural network which is a convolutional network. This model is suitable for proces- sing image data a can use input image without any preprocessing whatsoever. After our detailed analysis we choose this model as a detector for hockey players. We have tested several different architectures of the networks which we then compared and choose the one which is not only accurate but also fast enough. We have also tested the robustness of the network with noisy patterns. Finally we assigned detected pla- yers to their corresponding teams utilizing K-mean algorithm using the information about their jersey color. Keywords:...

In the field of autonomous vehicles, Advanced Driver Assistance Systems (ADAS)play a key role. Their applications vary from aiding with critical safety systems to assisting with trivial parking scenarios. To optimize the use of resources, trivial ADAS applications are often limited to make use of low-cost sensors. As a result, sensors such as Cameras and UltraSonics are preferred over LiDAR (Light Detection and Ranging) and RADAR (RAdio Detection And Ranging) in assisting the driver with parking. In a parking scenario, to ensure the safety of people in and around the car, the sensors need to detect objects around the car in real-time. With the advancements in Deep Learning, Deep Neural Networks (DNN) are becoming increasingly effective in detecting objects with real-time performance. Therefore, the thesis aims to investigate the viability of Deep Neural Networks using Fisheye cameras to detect pedestrians around the car. To achieve the objective, an experiment was conducted on a test vehicle equipped with multiple Fisheye cameras. Three Deep Neural Networks namely, YOLOv3 (You Only Look Once), its faster variant Tiny-YOLOv3 ND ResNet-50 were chosen to detect pedestrians. The Networks were trained on Fisheye image dataset with the help of transfer learning. After training, the models were also compared to pre-trained models that were trained to detect pedestrians on normal images. Our experiments have shown that the YOLOv3 variants have performed well but with a difficulty of localizing the pedestrians. The ResNet model has failed to generate acceptable detections and thus performed poorly. The three models produced detections with a real-time performance for a single camera but when scaled to multiple cameras, the detection speed was not on par. The YOLOv3 variants could detect pedestrians successfully on dewarped fish-eye images but the pipeline still needs a better dewarping algorithm to lessen the distortion effects. Further, the models need to be optimized in order to generate detections with real-time performance on multiple cameras and also to fit the model on an embedded system.

This thesis describes how we utilize machine learning and image preprocessing to create a system that can extract a license plate number by taking a picture of a car with an Android smartphone. This project was provided by ÅF at the behalf of one of their customers who wanted to make the workflow of their employees more efficient. The two main techniques of this project are object detection to detect license plates and optical character recognition to then read them. In between are several different image preprocessing techniques to make the images as readable as possible. These techniques mainly includes skewing and color distorting the image. The object detection consists of a convolutional neural network using the You Only Look Once technique, trained by us using Darkflow. When using our final product to read license plates of expected quality in our evaluation phase, we found that 94.8% of them were read correctly. Without our image preprocessing, this was reduced to only 7.95%.

Multiple object tracking is the process of assigning unique and consistent identities to objects throughout a video sequence. A popular approach to multiple object tracking, and object tracking in general, is to use a method called tracking-by-detection. Tracking-by-detection is a two-stage procedure: an object detection algorithm first detects objects in a frame, these objects are then associated with already tracked objects by a tracking algorithm. One of the main concerns of this thesis is to investigate how different object detection algorithms perform on surveillance video supplied by National Forensic Centre. The thesis then goes on to explore how the stand-alone alone performance of the object detection algorithm correlates with overall performance of a tracking-by-detection system. Finally, the thesis investigates how the use of visual descriptors in the tracking stage of a tracking-by-detection system effects performance.  Results presented in this thesis suggest that the capacity of the object detection algorithm is highly indicative of the overall performance of the tracking-by-detection system. Further, this thesis also shows how the use of visual descriptors in the tracking stage can reduce the number of identity switches and thereby increase performance of the whole system.

A detecção de objetos, pertencentes a uma determinada classe, em vídeos é de uma atividade amplamente estudada devido às aplicações potenciais que ela implica. Por exemplo, para vídeos obtidos por uma câmera estacionária, temos aplicações como segurança ou vigilância do tráfego, e por uma câmera dinâmica, para assistência ao condutor, entre outros. Na literatura, há diferentes métodos para tratar indistintamente cada um dos casos mencionados, e que consideram só imagens obtidas por um único tipo de câmera para treinar os detectores. Isto pode levar a uma baixa performance quando se aplica a técnica em vídeos de diferentes tipos de câmeras. O estado da arte na detecção de objetos de apenas uma classe, mostra uma tendência pelo uso de histogramas, treinamento supervisionado e, basicamente, seguem a seguinte estrutura: construção do modelo da classe de objeto, detecção de candidatos em uma imagem/quadro, e aplicação de uma medida sobre esses candidatos. Outra desvantagem observada é o uso de diferentes modelos para cada linha de visada de um objeto, gerando muitos modelos e, em alguns casos, um classificador para cada linha de visada. Nesta dissertação, abordamos o problema de detecção de objetos, usando um modelo da classe do objeto criada com um conjunto de dados de imagens estáticas e posteriormente usamos o modelo para detectar objetos na seqüência de imagens (vídeos) que foram coletadas a partir de câmeras estacionárias e dinâmicas, ou seja, num cenário totalmente diferente do usado para o treinamento. A criação do modelo é feita em uma fase de aprendizagem off-line, utilizando o conjunto de imagens PASCAL 2007. O modelo baseia-se em uma mistura de modelos baseados em partes deformáveis (MDPM), originalmente proposto por Felzenszwalb et al. (2010b) no âmbito da detecção de objetos em imagens. Não limitamos o modelo para uma determinada linha de visada. Foi elaborado um conjunto de experimentos que exploram o melhor número de componentes da mistura e o número de partes do modelo. Além disso, foi realizado um estudo comparativo de MDPMs simétricas e assimétricas. Testamos esse método para detectar objetos como pessoas e carros em vídeos obtidos por câmera estacionária e dinâmica. Nossos resultados não mostram apenas o bom desempenho da MDPM e melhores resultados que o estado da arte na detecção de objetos em vídeos obtidos por câmeras estacionárias ou dinâmicas, mas também mostram o melhor número de componentes da mistura e as partes para o modelo criado. Finalmente, os resultados mostram algumas diferenças entre as MDPMs simétricas e assimétricas na detecção de objetos em diferentes vídeos. / The problem of detecting objects that belong to a specific class of objects, in videos is a widely studied activity due to its potential applications. For example, for videos that have been taken from a stationary camera, we can mention applications such as security and traffic surveillance; when the video have been taken from a dynamic camera, a possible application is autonomous driving. The literature, presents several different approaches to treat indiscriminately with each of the cases mentioned, and only consider images obtained from a stationary or dynamic camera to train the detectors. These approaches can lead to poor performaces when the tecniques are used in sequences of images from different types of camera. The state of the art in the detection of objects that belong to a specific class shows a tendency to the use of histograms, supervised training and basically follows the structure: object class model construction, detection of candidates in the image/frame, and application of a distance measure to those candidates. Another disadvantage is that some approaches use several models for each point of view of the car, generating a lot of models and, in some cases, one classifier for each point of view. In this work, we approach the problem of object detection, using a model of the object class created with a dataset of static images and we use the model to detect objects in videos (sequence of images) that were collected from static and dynamic cameras, i.e., in a totally different setting than used for training. The creation of the model is done by an off-line learning phase, using an image database of cars in several points of view, PASCAL 2007. The model is based on a mixture of deformable part models (MDPM), originally proposed by Felzenszwalb et al. (2010b) for detection in static images. We do not limit the model for any specific viewpoint. A set of experiments was elaborated to explore the best number of components of the integration, as well as the number of parts of the model. In addition, we performed a comparative study of symmetric and asymmetric MDPMs. We evaluated the proposed method to detect people and cars in videos obtained by a static or a dynamic camera. Our results not only show good performance of MDPM and better results than the state of the art approches in object detection on videos obtained from a stationary, or dynamic, camera, but also show the best number of components of the integration and parts or the created object. Finally, results show differences between symmetric and asymmetric MDPMs in the detection of objects in different videos.

A detecção de objetos é uma área de extrema importância para sistemas de visão computacional. Em especial, dado o aumento constante da utilização de dispositivos móveis, torna-se cada vez mais importante o desenvolvimento de métodos e aplicações capazes de serem utilizadas em tais aparelhos. Neste sentido, neste trabalho propõe-se o estudo e implementação de um aplicativo para dispositivos móveis capaz de detectar, em tempo real, objetos existentes em ambientes internos com uma aplicação para auxiliar um usuário a se localizar dentro do local. O aplicativo depende somente das capacidades do próprio aparelho e, portanto, procura ser mais flexível e sem restrições. A detecção de objetos é realizada por casamento de grafos-chave entre imagens de objetos pré-escolhidas e a imagem sendo capturada pela câmera do dispositivo. Os grafos-chave são uma generalização do método de detecção de pontos-chave tradicional e, por levarem em consideração um conjunto de pontos e suas propriedades estruturais, são capazes de descrever e detectar os objetos de forma robusta e eficiente. Para realizar a localização, optou-se por detectar placas existentes no próprio local. Após cada detecção, aplica-se um simples, mas bastante eficaz, sistema de localização baseado na comparação da placa detectada com uma base de dados de imagens de todo o ambiente. A base foi construída utilizando diversas câmeras colocadas sobre uma estrutura móvel, capturando sistematicamente imagens do ambiente em intervalos regulares. A implementação é descrita em detalhes e são apresentados resultados obtidos por testes reais no ambiente escolhido utilizando um celular Nokia N900. Tais resultados são avaliados em termos da precisão da detecção e da estimativa de localização, bem como do tempo decorrido para a realização de todo o processo. / Object detection is an area of extreme importance for computer vision systems. Specially because of the increasing use of mobile devices, it becomes more and more important to develop methods and applications that can be used in such devices. In this sense, we propose the study and implementation of an application for mobile devices that is able to detect, in real time, existing indoor objects with an application to help a user in localization in the environment. The application depends solely on the device capabilities and hence, it is flexible and unconstrained. Object detection is accomplished by keygraph matching between images of previously chosen signs and the image currently being captured by the camera device. Keygraphs are a generalization of the traditional keypoints method and, by taking into consideration a set of points and its structural properties, are capable of describing the objects robustly and efficiently. In order to perform localization, we chose to detect signs existing in the environment. After each detection, we apply a simple, but very effective, localization method based on a comparison between the detected sign and a dataset of images of the whole environment. The dataset was built using several cameras atop a mobile structure, systematically capturing images of the environment at regular intervals. The implementation is described in details and we show results obtained from real tests in the chosen environment using a Nokia N900 cell phone. Such results are evaluated in terms of detection and localization estimation precision, as well as the elapsed time to perform the whole process.

<p>Quick and accurate segmentation and object detection of the biomedical image is the starting point of most disease analysis and understanding of biological processes in medical research. It will enhance drug development and advance medical treatment, especially in cancer-related diseases. However, identifying the objects in the CT or MRI images and labeling them usually takes time even for an experienced person. Currently, there is no automatic detection technique for nucleus identification, pneumonia detection, and fetus brain segmentation. Fortunately, as the successful application of artificial intelligence (AI) in image processing, many challenging tasks are easily solved with deep convolutional neural networks. In light of this, in this thesis, the deep learning based object detection and segmentation methods were implemented to perform the nucleus segmentation, lung segmentation, pneumonia detection, and fetus brain segmentation. The semantic segmentation is achieved by the customized U-Net model, and the instance localization is achieved by Faster R-CNN. The reason we choose U-Net is that such a network can be trained end-to-end, which means the architecture of this network is very simple, straightforward and fast to train. Besides, for this project, the availability of the dataset is limited, which makes U-Net a more suitable choice. We also implemented the Faster R-CNN to achieve the object localization. Finally, we evaluated the performance of the two models and further compared the pros and cons of them. The preliminary results show that deep learning based technique outperforms all existing traditional segmentation algorithms. </p>

Perceiving the dynamics of moving objects in complex scenarios is crucial for smart monitoring and safe navigation, thus a key enabler for intelligent supervision and autonomous driving. A variety of research has been developed to detect and track moving objects from data collected by optical sensors and/or laser scanners while most of them concentrate on certain type of objects or face the problem of lacking motion cues. In this thesis, we present a data-driven, model-free detection-based tracking approach for tracking moving objects in urban scenes from time sequential point clouds obtained via state-of-art Doppler LiDAR, which can not only collect spatial information (e.g. point clouds) but also Doppler images by using Doppler-shifted frequencies. In our approach, we first use Doppler images to detect moving points and determine the number of moving objects, which are then completely segmented via a region growing technique. The detected objects are then input to the tracking session which is based on Multiple Hypothesis Tracking (MHT) with two innovative extensions. One extension is that a new point cloud descriptor, <i>Oriented Ensemble of Shape Function (OESF)</i>, is proposed to evaluate the structure similarity when doing object-to-track association in MHT. Another extension is that speed information from Doppler images is used to predict the dynamic state of the moving objects, which is integrated into MHT to improve the estimation of dynamic state of moving objects. The proposed approach has been tested on datasets collected by a terrestrial Doppler LiDAR and a mobile Doppler LiDAR <a>separately</a>. The quantitative evaluation of detection and tracking results shows the unique advantages of the Doppler LiDAR and the effectiveness of the proposed detection and tracking approach.<br>

Rip current images are useful for assisting in climate studies but time consuming to manually annotate by hand over thousands of images. Object detection is a possible solution for automatic annotation because of its success and popularity in identifying regions of interest in images, such as human faces. Similarly to faces, rip currents have distinct features that set them apart from other areas of an image, such as more generic patterns of the surf zone. There are many distinct methods of object detection applied in face detection research. In this thesis, the best fit for a rip current object detector is found by comparing these methods. In addition, the methods are improved with Haar features exclusively created for rip current images. The compared methods include max distance from the average, support vector machines, convolutional neural networks, the Viola-Jones object detector, and a meta-learner. The presented results are compared for accuracy, false positive rate, and detection rate. Viola-Jones has the top base-line performance by achieving a detection rate of 0.88 and identifying only 15 false positives in the test image set of 53 rip currents. The described meta-learner integrates the presented Haar features, which are developed in accordance with the original Viola-Jones algorithm. Ada-Boost, a feature ranking algorithm, shows that the newly presented Haar features extract more meaningful data from rip current images than some of the current features. The meta-classifier improves upon the stand-alone Viola-Jones when applying these features by reducing its false positives by 47% while retaining a similar computational cost and detection rate.

In the e-commerce industry, importing data from third party clothing brands require validation of this data. If the validation step of this data is done manually, it is a tedious and time-consuming task. Part of this task can be replaced or assisted by using computer vision to automatically find clothing types, such as T-shirts and pants, within imported images. After a detection of clothing type is computed, it is possible to recommend the likelihood of clothing products correlating to data imported with a certain accuracy. This was done alongside a prototype interface that can be used to start training, finding clothing types in an image and to mask annotations of products. Annotations are areas describing different clothing types and are used to train an object detector model. A model for finding clothing types is trained on Mask R-CNN object detector and achieves 0.49 mAP accuracy. A detection take just above one second on an Nvidia GTX 1070 8 GB graphics card. Recommending one or several products based on a detection take 0.5 seconds and the algorithm used is k-nearest neighbors. If prediction is done on products of which is used to build the model of the prediction algorithm almost perfect accuracy is achieved while products in images for another products does not achieve nearly as good results.