Back to search

National and Kapodistrian University of Athens

Dissecting the Performance of AI Applications Using NVIDIA GPUs at the Edge

Abstract

dc:description

The vast amount of data stemming from an increasingly large number of end devices, such as User Equipment (UE) and Internet of Things (IoT), poses excessive bandwidth requirements and dramatically increased latency, when this data needs to be transferred to and processed by remote cloud sites. The advent of edge computing along with Artificial Intelligence, also abbreviated as edge intelligence (EI), attempts to solve this problem, allowing edge computing hardware closely located to the data sources to be combined with edge AI applications, such as AI inference. This thesis evaluates the performance of an object detection AI application on two NVIDIA Jetson edge computing devices. Specifically, both AI training and inference phases are evaluated against different settings that can impact application performance. The main contribution of this thesis is to provide experimental evidence on performance metrics (e.g., throughput) and power consumption, while assessing whether the NVIDIA Jetson devices can effectively conduct both AI training and inference. Specifically, the performance of the NVIDIA Jetson devices using different (i) processing engines, such as general-purpose Central Processing Units (CPUs) and (Accelerated) Graphics Processing Units (GPUs), (ii) power configurations, (iii) network architecture types, and (iv) values of training parameters is investigated. Our experimental results show that although the NVIDIA Jetson devices under test have enough capacity to perform both AI training and inference for the selected application, AI training is time-consuming on such low-end devices. On the other hand, AI inference on Jetson’s GPU-based processing engines achieves substantially higher performance than in the case of a CPU processing engine. Finally, apart from performance aspects, this thesis also addresses power consumption aspects of the tested processing engines (i.e., CPU and GPU).

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Srivallapanondh Sasipim

Subjects

dc:subject × 2

Rights

Language dc:language
English

Identifiers

dc:identifier.*
Identifier
uoadl:2962089

Chain of custody

source
Harvested from
National and Kapodistrian University of Athens
Base URL
pergamos.lib.uoa.gr/uoa/dl/frontend/oaipmh
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Srivallapanondh Sasipim. Dissecting the Performance of AI Applications Using NVIDIA GPUs at the Edge. 2021. https://pergamos.lib.uoa.gr/uoa/dl/object/2962089