Back to search

Michigan State University

Evolutionary multi-objective bi-level optimization for efficient deep neural network architecture design

Abstract

dc:description

Deep convolutional neural networks (CNNs) are the backbones of deep learning (DL) paradigms for numerous vision tasks, including object recognition, detection, segmentation, etc. Early advancements in CNN architectures are primarily driven by human expertise and elaborate design. Recently, neural architecture search (NAS) was proposed with the aim of automating the network design process and generating task-dependent architectures. While existing approaches have achieved competitive performance, they are still impractical to real-world deployment for three reasons: (1) the generated architectures are solely optimized for predictive performance, resulting in inefficiency in utilizing hardware resources---i.e. energy consumption, latency, memory size, etc.; (2) the search processes require vast computational resources in most approaches; (3) most existing approaches require one complete search for each deployment specification of hardware or requirement. In this dissertation, we propose an efficient evolutionary NAS algorithm to address the aforementioned limitations. In particular, we first introduce Pareto-optimization to NAS, leading to a diverse set of architectures, trading-off multiple objectives, being obtained simultaneously in one run. We then improve the algorithm's search efficiency through surrogate models. We finally integrate a transfer learning scheme to the algorithm that allows a new task to leverage previous search efforts that further improves both the performance of the obtained architectures and search efficiency. Therefore, the proposed algorithm enables an automated and streamlined process to efficiently generate task-specific custom neural network models that are competitive under multiple objectives.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Lu, Zhichao
Contributors dc:contributor
  • Deb, Kalyanmoy
  • Goodman, Erik
  • Banzhaf, Wolfgang
  • Boddeti, Vishnu

Subjects

dc:subject × 2

Rights

dc:rights
Statement dc:rights
  • In Copyright
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
etd:48679
isbn:9798662481879
oclc:1201190016
umi:28025436
local:Lu_grad.msu_0128D_17703

Chain of custody

source
Harvested from
Michigan State University
Base URL
d.lib.msu.edu/oai
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Lu, Zhichao. Evolutionary multi-objective bi-level optimization for efficient deep neural network architecture design. 2020. https://doi.org/doi:10.25335/a0ke-c965