Back to search

Cornell University

On Some Fundamental Aspects of Learning in Artificial Neural Networks and Universal Channel Codes

Abstract

dc:description.abstract

This thesis discusses a few interesting topics regarding fundamental aspects of learning in the following prevalent application scenarios: (1) training a neural network for image classification, and (2) using a channel code for universal communication at capacity. For the classification problem, we aim to develop a better understanding of what representations each layer of the network learns. In particular, we compare the higher-layer activations of two neural networks with identical architecture but different initializations via adaptive $k$NN graph approximation of the underlying manifold, and we show that there are vast similarities between the underlying manifolds of the two networks but with discrepancy in potentially highly-curved regions. We also investigate locality of the receptive field in the Convolutional Neural Networks by using semi-localized filters with random neuron connection, where we find out that the receptive field might be beyond local for feature extraction as is hard coded in traditional design. For the communication problem, we study universal channel coding under the high-dimensional statistical setting beyond Shannon's classical framework, and we prove a series of theorems that may surprisingly indicate a need to learn the entire channel in order to achieve its capacity.

Degree

thesis:*
Name thesis:degree_name
Ph.D., Electrical and Computer Engineering
Level thesis:degree_level
Doctor of Philosophy
Discipline thesis:degree_discipline
Electrical and Computer Engineering
Grantor
Cornell University
Year dc:date.issued
2019

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Gao, Yuguang
Chair dc:contributor.chair
  • Hopcroft, John E.
Committee members dc:contributor.committeemember
  • Tong, Lang
  • Kleinberg, Robert David

Subjects

dc:subject × 1

Rights

Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Dc Identifier Bibid
11050641
Dc Identifier Other
ProQuest Submission ID: 11340
ProQuest Publication ID: 13857602

Chain of custody

source
Harvested from
Cornell University
Base URL
ecommons.cornell.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Gao, Yuguang. On Some Fundamental Aspects of Learning in Artificial Neural Networks and Universal Channel Codes. Doctor of Philosophy thesis, Cornell University, 2019. https://hdl.handle.net/1813/67657