Back to search

Michigan State University

Adaptive on-device deep learning systems

Abstract

dc:description

"Mobile systems such as smartphones, drones, and augmented-reality headsets are revolutionizing our lives. On-device deep learning is regarded as the key enabling technology for realizing their full potential. This is because communication with cloud adds additional latency or cost, or the applications must operate even with intermittent internet connectivity.The key to achieving the full promise of these mobile vision systems is effectively analyzing the streaming video frames. However, processing streaming video frames taken in mobile settings is challenging in two folds. First, the processing usually involves multiple computer vision tasks. This multi-tenant characteristic requires mobile vision systems to concurrently run multiple applications that target different vision tasks. Second, the context in mobile settings can be frequently changed. This requires mobile vision systems to be able to switch applications to execute new vision tasks encountered in the new context.In this article, we fill this critical gap by proposing NestDNN, a framework that enables resource-aware multi-tenant on-device deep learning for continuous mobile vision. NestDNN enables each deep learning model to offer flexible resource-accuracy trade-offs. At runtime,it dynamically selects the optimal resource-accuracy trade-off for each deep learning model to fit the model's resource demand to the system's available runtime resources. In doing so, NestDNN efficiently utilizes the limited resources in mobile vision systems to jointly maximize the performance of all the concurrently running applications.Although NestDNN is able to efficiently utilize the resource by being resource-aware, it essentially treats the content of each input image equally and hence does not realize the full potential of such pipelines. To realize its full potential, we further propose FlexDNN, a novel content-adaptive framework that enables computation-efficient DNN-based on-device video stream analytics based on early exit mechanism. Compared to state-of-the-art earlyexit-based solutions, FlexDNN addresses their key limitations and pushes the state-of-the-artforward through its innovative fine-grained design and automatic approach for generating the optimal network architecture."--Pages ii-iii.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Fang, Biyi
Contributors dc:contributor
  • Zhang, Mi
  • Tan, Xiaobo
  • Liu, Xiaoming
  • Ren, Jian

Subjects

dc:subject × 3

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
etd:48266
isbn:9781392384046
isbn:1392384044
oclc:1138556548
umi:27663010
local:Fang_grad.msu_0128D_17246

Chain of custody

source
Harvested from
Michigan State University
Base URL
d.lib.msu.edu/oai
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Fang, Biyi. Adaptive on-device deep learning systems. 2019. https://doi.org/doi:10.25335/6cbc-7224