Deep learning, a sub-topic of machine learning inspired by biology, have achieved wide attention in the industry and research community recently. State-of-the-art applications in the area of computer vision and speech recognition (among others) are built using deep learning algorithms. In contrast to traditional algorithms, where the developer fully instructs the application what to do, deep learning algorithms instead learn from experience when performing a task. However, for the algorithm to learn require training, which is a high computational challenge. High Performance Computing can help ease the burden through parallelization, thereby reducing the training time; this is essential to fully utilize the algorithms in practice. Numerous work targeting GPUs have investigated ways to speed up the training, less attention have been paid to the Intel Xeon Phi coprocessor. In this thesis we present a parallelized implementation of a Convolutional Neural Network (CNN), a deep learning architecture, and our proposed parallelization scheme, CHAOS. Additionally a theoretical analysis and a performance model discuss the algorithm in detail and allow for predictions if even more threads are available in the future. The algorithm is evaluated on an Intel Xeon Phi 7120p, Xeon E5-2695v2 2.4 GHz and Core i5 661 3.33 GHz using various architectures and thread counts on the MNIST dataset. Findings show a 103.5x, 99.9x, 100.4x speed up for the large, medium, and small architecture respectively for 244 threads compared to 1 thread on the coprocessor. Moreover, a 10.9x - 14.1x (large to small) speed up compared to the sequential version running on Xeon E5. We managed to decrease training time from 7 days on the Core i5 and 31 hours on the Xeon E5, to 3 hours on the Intel Xeon Phi when training our large network for 15 epochs

We present a set of easy-to-use methods and tools to analyze human attention, behaviour, and physiological responses. A potential application of our work is evaluating user interfaces being used in a natural manner. Our approach is designed to be scalable and to work remotely on regular personal computers using expensive and noninvasive equipment. The data sources our tool processes are nonintrusive, and captured from video; i.e. eye tracking, and facial expressions. For video data retrieval, we use a basic webcam. We investigate combinations of observation modalities to detect and extract affective and mental states. Our tool provides a pipeline-based approach that 1) collects observational, data 2) incorporates and synchronizes the signal modality mentioned above, 3) detects users' affective and mental state, 4) records user interaction with applications and pinpoints the parts of the screen users are looking at, 5) analyzes and visualizes results. We describe the design, implementation, and validation of a novel multimodal signal fusion engine, Deep Temporal Credence Network (DTCN). The engine uses Deep Neural Networks to provide 1) a generative and probabilistic inference model, and 2) to handle multimodal data such that its performance does not degrade due to the absence of some modalities. We report on the recognition accuracy of basic emotions for each modality. Then, we evaluate our engine in terms of effectiveness of recognizing basic six emotions and six mental states, which are agreeing, concentrating, disagreeing, interested, thinking, and unsure. Our principal contributions include the implementation of a 1) multimodal signal fusion engine, 2) real time recognition of affective and primary mental states from nonintrusive and inexpensive modality, 3) novel mental state-based visualization techniques, 3D heatmaps, 3D scanpaths, and widget heatmaps that find parts of the user interface where users are perhaps unsure, annoyed, frustrated, or satisfied.

The aim of this master thesis is to design and implement a system for the detection, tracking and classification of vehicles from streams or records from traffic cameras in language C++. The system runs on the platform Robot Operating System and uses the OpenCV, FFmpeg, TensorFlow and Keras libraries. For detection cascade classifier is used, for tracking Kalman filter and for classification of the convolutional neural network. Out of a total of 627 cars, 479 were tracked correctly. From this number 458 were classified (trucks or lorries not included). The resulting system can be used for traffic analysis.

In this diploma thesis, the reader is introduced to artificial neural networks and convolutional neural networks. Based on that, the design and implementation of a new library for convolutional neural networks is described. The library is then evaluated on widely used datasets and compared to other publicly available libraries. The added benefit of the library, that makes it unique, is its independence on data types. Each layer may contain up to three independent data types - for weights, for inference and for training. For the purpose of evaluating this feature, a data type with fixed point representation is also part of the library. The effects of this representation on trained net accuracy are put to a test.

The subject of this thesis is neural network acceleration with the goal of reducing the number of floating point multiplications. The theoretical part of the thesis surveys current trends and methods used in the field of neural network acceleration. However, the focus is on the binarization techniques which allow replacing multiplications with logical operators. The theoretical base is put into practice in two ways. First is the GPU implementation of crucial binary operators in the Tensorflow framework with a performance benchmark. Second is an application of these operators in simple image classifier. Results are certainly encouraging. Implemented operators achieve speed-up by a factor of 2.5 when compared to highly optimized cuBLAS operators. The last chapter compares accuracies achieved by binarized models and their full-precision counterparts on various architectures.

In this thesis I deal with the problem of navigation and autonomous driving using convolutional neural networks. I focus on the main approaches utilizing sensory inputs described in literature and the theory of neural networks, imitation and reinforcement learning. I also discuss the tools and methods applicable to driving systems. I created two deep learning models for autonomous driving in simulated environment. These models use the Dataset Aggregation and Deep Deterministic Policy Gradient algorithms. I tested the created models in the TORCS car racing simulator and compared the result with available sources.

This thesis deals with designing an automatic generator of deep neural networks for image classification. Theoretical part clarifies what a neural network and formal neuron are. Furthermore, the types of neural network architectures are presented. The focus of this thesis is convolutional neural networks, several pieces of research from this field are mentioned. The practical part of this thesis describes information with regards to the implementation of neural network generator, possible frameworks and programming languages for such implementation. Brief description of the implementation itself is presented as well as implemented layers. Generated neural networks are tested on Google-Landmarks dataset and results are commented upon.

Optical character recognition (OCR) has been a topic of interest for many years. It is defined as the process of digitizing a document image into a sequence of characters. Despite decades of intense research, OCR systems with capabilities to that of human still remains an open challenge. In this work there is presented a design and implementation of such system, which is capable of detecting texts in graphical user interfaces.

This thesis studies navigation and autonomous driving using convolutional neural networks. It presents main approaches to this problem used in literature. It describes theory of neural networks and imitation and reinforcement learning. It also describes tools and methods suitable for a driving system. There are two simulation driving models created using learning algorithms DAGGER and DDPG. The models are then tested in car racing simulator TORCS.

Optical character recognition (OCR) has been a topic of interest for many years. It is defined as the process of digitizing a document image into a sequence of characters. Despite decades of intense research, OCR systems with capabilities to that of human still remains an open challenge. In this work there is presented a design and implementation of such system, which is capable of detecting texts in graphical user interfaces.