Back to search

Michigan State University

New perspectives in neural architecture search : architecture embeddings, efficient performance estimations, and their applications

Abstract

dc:description

Neural architecture search (NAS) has made significant strides in recent years, but challenges remain in terms of the stability of search performance and the high computational requirements of sampling-based NAS. Studying architecture representations offers a promising solution to these challenges, as it encourages neural architectures with similar structures or computations to cluster together. This helps to map neural architectures with similar performance to the same regions in the latent space and leads to smoother transitions in the latent space, benefiting downstream search. Additionally, learning curve extrapolation can accelerate the search process by estimating the final validation accuracy of a neural network from the learning curve of a partially trained network. Overall, understanding the neural architecture representations and their associated learning curves through theoretical analysis and empirical evaluations is crucial for achieving stable and scalable NAS.This dissertation presents our contributions to the field of neural architecture search (NAS), which push the limits of NAS and achieve state-of-the-art performance. Our contributions include efficient one-shot NAS via hierarchical masking, addressing the joint optimization problem of architecture representations and search through unsupervised pre-training, improving the generalization ability of architecture representations with computation-aware self-supervised training, and developing a method for facilitating multi-fidelity NAS research.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Yan, Shen (Graduate of Michigan State University)
Contributors dc:contributor
  • Zhang, Mi

Subjects

dc:subject × 3

Rights

dc:rights
Statement dc:rights
  • Attribution-NonCommercial 4.0 International
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
etd:50821
isbn:9798358499805
oclc:on1373601787
oclc:1373601787
umi:29993270
local:Yan_grad.msu_0128D_19493

Chain of custody

source
Harvested from
Michigan State University
Base URL
d.lib.msu.edu/oai
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Yan, Shen (Graduate of Michigan State University). New perspectives in neural architecture search : architecture embeddings, efficient performance estimations, and their applications. 2022. https://doi.org/doi:10.25335/xyr7-mj54