In recent years, AI has developed significantly and become more popular than ever before. The applications of AI are expanding, making knowledge about its application and the systems it can be applied to more important. This project compares and evaluates deep learning models for object detection of traffic signs on the embedded systems Nvidia Jetson Nano and Raspberry Pi 3 Model B. The project compares and evaluates the models YOLOv5, SSD Mobilenet V1, FOMO, and Efficientdet-lite0. The project evaluates the performance of these models on the aforementioned embedded systems, measuring metrics such as CPU usage, FPS and RAM. Deep learning models are resource-intensive, and embedded systems have limited resources. Embedded systems often have different types of processor architectures than regular computers, which means that some frameworks and libraries may not be compatible. The results show that the tested systems are capable of object detection but with varying performance. Jetson Nano performs at a level we consider sufficiently high for use in production depending on the specific requirements. Raspberry Pi 3 performs at a level that may not be acceptable for real-time recognition of traffic signs. We see the greatest potential for Efficientdet-lite0 and YOLOv5 in recognizing traffic signs. The distance at which the models detect signs seems to be important for how many signs they find. For this reason, SSD MobileNet V1 is not recommended without further trai-ning despite its superior speed. YOLOv5 stood out as the model that detected signs at the longest distance and made the most detections overall. When considering all the results, we believe that Efficientdet-lite0 is the model that performs the best. / Under de senaste åren har AI utvecklats mycket och blivit mer populärt än någonsin. Tillämpningsområdena för AI ökar och därmed blir kunskap om hur det kan tillämpas och på vilka system viktigare. I det här projektet jämförs och utvärderas djupinlärningsmodeller för objektdetektering av trafikskyltar på de inbyggda systemen Nvidia Jetson Nano och Raspberry Pi 3 Model B. Modellerna som jämförs och utvärderas är YOLOv5, SSD Mobilenet V1, FOMO och Efficientdet-lite0. För varje modell mäts blandannat CPU-användning, FPS och RAM. Modeller för djupinlärning är resurskrävande och inbyggda system har begränsat med resurser. Inbyggda system har ofta andra typer av processorarkitekturer än en vanlig dator vilket gör att olika ramverk och andra bibliotek inte är kompatibla. Resultaten visar att de testade systemen klarar av objektdetektering med varierande prestation. Jetson Nano presterar på en nivå vi anser vara tillräckligt hög för användning i produktion beroende på hur hårda krav som ställs. Raspberry Pi 3 presterar på en nivå som möjligtvis inte är acceptabel för igenkänning av trafikskyltar i realtid. Vi ser störst potential för Efficientdet-lite0 och YOLOv5 för igenkänning av trafikskyltar. Hur långt avstånd modellerna upptäcker skyltar på verkar vara viktigt för hur många skyltar de hittar. Av den anledningen är SSD MobileNet V1 inte att rekommendera utan vidare träning trots sin överlägsna hastighet. YOLOv5 utmärkte sig som den som upptäckte skyltar på längst avstånd och som gjorde flest upptäckter totalt. När alla resultat vägs in anser vi dock att Efficientdet-lite0 är den modell som presterar bäst.

This diploma thesis deals with design of embedded computer vision system and transfer of existing computer vision application for 3D object detection from Windows OS to designed embedded system with Linux OS. Thesis focuses on design of communication interface for system control and camera video transfer through local network with video compression. Then, detection algorithm is enhanced by transferring computationally expensive functions to GPU using CUDA technology. Finally, a user application with graphical interface is designed for system control on Windows platform.

Hlavným cieľom práce bolo vytvorenie softvérového riešenia založeného na neurónových sieťach, pomocou ktorého bolo možné detegovať človeka a následne ho nasledovať. Tento výsledok bol dosiahnutý splnením jednotlivých bodov zadania tejto práce. V prvej časti práce je popísaný použitý hardvér, softvérové knižnice a rozhrania pre programovanie aplikácií (API), ako aj robotická platforma dodaná skupinou robotiky a umelej inteligencie ústavu automatizácie a meracej techniky Vysokého Učenia Technického v Brne, na ktorej bol výsledný robot postavený. Následne bola spracovaná rešerš viacerých typov neurónových sietí na detekciu osôb. Podrobne boli popísané štyri detektory. Niektoré z nich boli neskôr testované na klasickom počítači alebo na počítači NVIDIA Jetson Nano. V ďalšom kroku bolo vytvorené softvérové riešenie tvorené piatimi programmi, pomocou ktorého bolo dosiahnuté ciele ako rozpoznanie osoby pomocou neurónovej siete ped-100, určenie reálnej vzdialenosti vzhľadom k robotu pomocou monokulárnej kamery a riadenie roboty k úspešnému dosiahnutiu cieľa. Výstupom tejto práce je robotická platforma umožnujúca detekciu a nasledovanie osoby využiteľné v praxi.

Interconnected devices are becoming attractive solutions to integrate physical parameters and making them more accessible for further analysis. Edge devices, located at the end of the physical world, measure and transfer data to the remote server using either wired or wireless communication. The exploding number of sensors, being used in the Internet of Things (IoT), medical fields, or industry, are demanding huge bandwidth and computational capabilities in the cloud, to be processed by Artificial Neural Networks (ANNs) – especially, processing audio, video and images from hundreds of edge devices. Additionally, continuous transmission of information to the remote server not only hampers privacy but also increases latency and takes more power. Deep Neural Network (DNN) is proving to be very effective for cognitive tasks, such as speech recognition, object detection, etc., and attracting researchers to apply it in edge devices. Microcontrollers and single-board computers are the most commonly used types of edge devices. These have gone through significant advancements over the years and capable of performing more sophisticated computations, making it a reasonable choice to implement DNN. In this thesis, a DNN model is trained and implemented for Keyword Spotting (KWS) on two types of edge devices: a bare-metal embedded device (microcontroller) and a robot car. The unnecessary components and noise of audio samples are removed, and speech features are extracted using Mel-Frequency Cepstral Co-efficient (MFCC). In the bare-metal microcontroller platform, these features are efficiently extracted using Digital Signal Processing (DSP) library, which makes the calculation much faster. A Depth wise Separable Convolutional Neural Network (DSCNN) based model is proposed and trained with an accuracy of about 91% with only 721 thousand trainable parameters. After implementing the DNN on the microcontroller, the converted model takes only 11.52 Kbyte (2.16%) RAM and 169.63 Kbyte (8.48%) Flash of the test device. It needs to perform 287,673 Multiply-and-Accumulate (MACC) operations and takes about 7ms to execute the model. This trained model is also implemented on the robot car, Jetbot, and designed a voice-controlled robotic vehicle. This robot accepts few selected voice commands-such as “go”, “stop”, etc. and executes accordingly with reasonable accuracy. The Jetbot takes about 15ms to execute the KWS. Thus, this study demonstrates the implementation of Neural Network based KWS on two different types of edge devices: a bare-metal embedded device without any Operating System (OS) and a robot car running on embedded Linux OS. It also shows the feasibility of bare-metal offline KWS implementation for autonomous systems, particularly autonomous vehicles.<br>

The aim of this master's thesis thesis is to enable evaluation of quality, or the type of product in industrial applications using artificial neural networks, especially in applications where the classical approach of machine vision is too complicated. The system thus designed is implemented onto a specific hardware platform and becomes a subject to the final optimalisation for the hardware platform for the best performance of the system.

<p dir="ltr">A camera's image signal processor (ISP) is responsible for taking the mosaiced and noisy image signal from the image sensor and processing it such a way that an end-result image is produced that is informative and accurately captures the scene. Real-time video capture in photon-limited environments remains a challenge for many ISP's today. In these conditions, the image signal is dominated by the photon shot noise. Deep learning methods show promise in extracting the underlying image signal from the noise, but modern AI-based ISPs are too computationally complex to be realized as a fast and efficient hardware ISP. An ISP algorithm, BLADE2 has been designed, which leverages AI in a computationally conservative manner to demosaic and denoise low-light images. The original implementation of this algorihtm is in Python/PyTorch. This Thesis explores taking BLADE2 and implementing it on a general purpose GPU via a suite of Nvidia optimization toolkits, as well as a low-level implementation in C/C++, bringing the algorithm closer to FPGA realization. The GPU implementation demonstrated significant throughput gains and the C/C++ implementation demonstrated the feasibility of further hardware development.</p>

A agricultura de precisão tem agregado alto valor para os agricultores por causa das tecnologias que estão ligadas a ela. Sistemas que extraem informações de imagens digitais são extremamente utilizados para que o agricultor tome decisões a fim de aumentar sua produtividade. Uma das técnicas de realizar o monitoramento é a construção de um mosaico de imagens aéreas, onde são utilizadas aeronaves voando em baixa altitude. Esta técnica pode levar dezenas de horas para ser concluída, dependendo da configuração do computador que a executa. Com o intuito de reduzir o tempo nessa construção e tornar possível o embarque a essa aplicação, este trabalho apresenta uma maneira simplificada de construir o mosaico de imagens aéreas baseada na técnica de georreferenciamento direto, no qual utiliza a computação heterogênea para acelerar o desempenho. Essa abordagem é composta por apenas três técnicas que também compõem a abordagem clássica para a construção de mosaicos (warping, extração de características e combinação de características), além de inserir em seus cálculos os dados fornecidos pelos sensores GPS e IMU com a finalidade de direcionar e posicionar cada imagem pertencente ao conjunto que formará o mosaico. A plataforma de computação heterogênea utilizada neste trabalho é a NVIDIA Jetson TK1 escolhida pelo fato de disponibilizar de uma GPU que suporta a linguagem de programação CUDA. Utilizando esta abordagem, a falta de correção da perspectiva do conteúdo (geometria) da imagem gera um resultado inesperado, pois os dados fornecidos pela IMU, ao contrário do que se imagina, apenas servem para corrigir a posição das coordenadas do GPS registradas no momento de captura de cada imagem que compõem o mosaico. O tempo de execução da aplicação desenvolvida é satisfatório tornando possível a adoção desta abordagem. / Accuracy agriculture has added value to farmers thanks to the new technologies that are linked to it. Systems that extract information from digital images are very usefull to help farmers making decisions in order to increase their productivity. One of the techniques to perform this kind of monitoring is the construction of an aerial imagery mosaic where aircrafts flies in low altitude. This technique may take hours to be completed, depending on computer\'s configuration. With the purpose of reducing time in this construction, this thesis presents a simplified way to make aerial imagery mosaic based on direct georeferencing. This approach is composed by three techniques that also make up the classic approach to building mosaics (warping, extraction of characteristics and combination of characteristics), the difference is with this technique here presented is also possible to insert into the calculations the data provided by the GPS and IMU sensors with the purpose of directing and positioning each image to the belonging set to form the mosaic. The heterogeneous computing platform used in this work is the NVIDIA JetsonTK1, this platform was chosen because it offers a GPU that supports the language of CUDA programming. If the images\' geometry errors weren\'t rectfyed, using this approach, an unexpected result happens, because the data provided by IMU, contrary to what is imagined, only serve to correct the position of the GPS coordinates recorded at the moment of capture of each image that composes the mosaic. The developing time in this application is satisfactory making the adoption of this approch favorable.

For 500 years transient light phenomena (TLP) have been observed on the lunar surface by ground-based observers. The actual physical reason for most of these events is today still unknown. Current plans of NASA and SpaceX to send astronauts back to the Moon and already successful deep-space CubeSat mission will allow in the future research nanosatellite missions to the cislunar space. This thesis presents a new hardware and software concept for a future payload on such a nanosatellite. The main task was to develop and implement a high-performance image processing algorithm which task is to detect short brightening flashes on the lunar surface. Based on a review of historic reported phenomena, possible explanation theories for these phenomena and currently active and planed ground- or space-based observatories possible reference scenarios were analyzed. From the presented scenarios one, the detection of brightening events was chosen and requirements for this scenario stated. Afterwards, possible detectors, processing computers and image processing algorithms were researched and compared regarding the specified requirements. This analysis of available algorithm was used to develop a new high-performance detection algorithm to detect transient brightening events on the Moon. The implementation of this algorithm running on the processor and the internal GPU of a MacMini achieved a framerate of 55 FPS by processing images with a resolution of 4.2 megapixel. Its functionality and performance was verified on the remote telescope operated by the Chair of Space Technology of the University of Würzburg. Furthermore, the developed algorithm was also successfully ported on the Nvidia Jetson Nano and its performance compared with a FPGA based image processing algorithm. The results were used to chose a FPGA as the main processing computer of the payload. This concept uses two backside illuminated CMOS image sensor connected to a single FPGA. On the FPGA the developed image processing algorithm should be implemented. Further work is required to realize the proposed concept in building the actual hardware and porting the developed algorithm onto this platform.