Michigan State University
New perspectives in neural architecture search : architecture embeddings, efficient performance estimations, and their applications
Abstract
dc:descriptionNeural architecture search (NAS) has made significant strides in recent years, but challenges remain in terms of the stability of search performance and the high computational requirements of sampling-based NAS. Studying architecture representations offers a promising solution to these challenges, as it encourages neural architectures with similar structures or computations to cluster together. This helps to map neural architectures with similar performance to the same regions in the latent space and leads to smoother transitions in the latent space, benefiting downstream search. Additionally, learning curve extrapolation can accelerate the search process by estimating the final validation accuracy of a neural network from the learning curve of a partially trained network. Overall, understanding the neural architecture representations and their associated learning curves through theoretical analysis and empirical evaluations is crucial for achieving stable and scalable NAS.This dissertation presents our contributions to the field of neural architecture search (NAS), which push the limits of NAS and achieve state-of-the-art performance. Our contributions include efficient one-shot NAS via hierarchical masking, addressing the joint optimization problem of architecture representations and search through unsupervised pre-training, improving the generalization ability of architecture representations with computation-aware self-supervised training, and developing a method for facilitating multi-fidelity NAS research.
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Yan, Shen (Graduate of Michigan State University)
- Contributors dc:contributor
-
- Zhang, Mi
Subjects
dc:subject × 3Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial 4.0 International
- Language dc:language
- English
Identifiers
dc:identifier.*- Identifier
-
etd:50821
isbn:9798358499805
oclc:on1373601787
oclc:1373601787
umi:29993270
local:Yan_grad.msu_0128D_19493