{"id":{"repo_id":"cornell","oai_identifier":"oai:ecommons.cornell.edu:1813/67657"},"canonical_url":"https://search.ndltd.org/etd/cornell/oai:ecommons.cornell.edu:1813/67657","repository":{"repo_id":"cornell","name":"Cornell University","base_url":"https://ecommons.cornell.edu/server/oai/request"},"display":{"title":"On Some Fundamental Aspects of Learning in Artificial Neural Networks and Universal Channel Codes","abstract":"This thesis discusses a few interesting topics regarding fundamental aspects of learning in the following prevalent application scenarios: (1) training a neural network for image classification, and (2) using a channel code for universal communication at capacity. For the classification problem, we aim to develop a better understanding of what representations each layer of the network learns. In particular, we compare the higher-layer activations of two neural networks with identical architecture but different initializations via adaptive $k$NN graph approximation of the underlying manifold, and we show that there are vast similarities between the underlying manifolds of the two networks but with discrepancy in potentially highly-curved regions. We also investigate locality of the receptive field in the Convolutional Neural Networks by using semi-localized filters with random neuron connection, where we find out that the receptive field might be beyond local for feature extraction as is hard coded in traditional design. For the communication problem, we study universal channel coding under the high-dimensional statistical setting beyond Shannon's classical framework, and we prove a series of theorems that may surprisingly indicate a need to learn the entire channel in order to achieve its capacity.","abstract_html":"This thesis discusses a few interesting topics regarding fundamental aspects of learning in the following prevalent application scenarios: (1) training a neural network for image classification, and (2) using a channel code for universal communication at capacity. For the classification problem, we aim to develop a better understanding of what representations each layer of the network learns. In particular, we compare the higher-layer activations of two neural networks with identical architecture but different initializations via adaptive $k$NN graph approximation of the underlying manifold, and we show that there are vast similarities between the underlying manifolds of the two networks but with discrepancy in potentially highly-curved regions. We also investigate locality of the receptive field in the Convolutional Neural Networks by using semi-localized filters with random neuron connection, where we find out that the receptive field might be beyond local for feature extraction as is hard coded in traditional design. For the communication problem, we study universal channel coding under the high-dimensional statistical setting beyond Shannon&#x27;s classical framework, and we prove a series of theorems that may surprisingly indicate a need to learn the entire channel in order to achieve its capacity.","abstract_has_math":true,"creators":["Gao, Yuguang"],"institution":"Cornell University","degree_name":"Ph.D., Electrical and Computer Engineering","degree_level":"Doctor of Philosophy","degree_discipline":"Electrical and Computer Engineering","degree_department":null,"school":null,"contributors":[],"advisors":[],"committee_chairs":["Hopcroft, John E."],"committee_members":["Tong, Lang","Kleinberg, Robert David"],"year":2019,"date_issued":"2019-08-30","date_published":"2019-08-30","updated_at":"2026-11-19T03:20:50Z","subjects":["Electrical engineering"],"languages":["en_US"],"rights":[],"rights_urls":[],"identifier_entries":[{"key":"dc:identifier.doi","label":"DOI","values":["https://doi.org/10.7298/6a79-d633"],"render_values":[{"text":"https://doi.org/10.7298/6a79-d633","href":"https://doi.org/10.7298/6a79-d633","code":true}]},{"key":"dc:identifier.bibid","label":"Dc Identifier Bibid","values":["11050641"],"render_values":[{"text":"11050641","href":null,"code":true}]},{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["ProQuest Submission ID: 11340","ProQuest Publication ID: 13857602"],"render_values":[{"text":"ProQuest Submission ID: 11340","href":null,"code":true},{"text":"ProQuest Publication ID: 13857602","href":null,"code":true}]}],"person_identifier_entries":[]},"links":{"outbound_url":"https://hdl.handle.net/1813/67657","outbound_label":"Handle","outbound_source":"dc:identifier.uri","record_url":"https://hdl.handle.net/1813/67657","record_label":"Repository record","record_kind":"handle","record_source":"dc:identifier.uri","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://ecommons.cornell.edu/server/oai/request?verb=GetRecord&metadataPrefix=dim&identifier=oai%3Aecommons.cornell.edu%3A1813%2F67657","prefix":"dim"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor.chair","label":"Chair","values":["Hopcroft, John E."]},{"key":"dc:contributor.committeemember","label":"Committee Member","values":["Tong, Lang","Kleinberg, Robert David"]},{"key":"dc:creator","label":"Author","values":["Gao, Yuguang"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.accessioned","label":"Dc Date Accessioned","values":["2019-10-15T16:49:21Z"]},{"key":"dc:date.available","label":"Dc Date Available","values":["2021-08-29T06:00:18Z"]},{"key":"dc:date.issued","label":"Date","values":["2019-08-30"]},{"key":"dc:type","label":"Dc Type","values":["dissertation or thesis"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical and Computer Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Doctor of Philosophy"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Ph.D., Electrical and Computer Engineering"]},{"key":"thesis:institution_name","label":"Thesis Institution Name","values":["Cornell University"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Electrical engineering"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:language.iso","label":"Language (ISO)","values":["en_US"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier.bibid","label":"Dc Identifier Bibid","values":["11050641"]},{"key":"dc:identifier.doi","label":"DOI","values":["https://doi.org/10.7298/6a79-d633"]},{"key":"dc:identifier.other","label":"Dc Identifier Other","values":["ProQuest Submission ID: 11340","ProQuest Publication ID: 13857602"]},{"key":"dc:identifier.uri","label":"Identifier URI","values":["https://hdl.handle.net/1813/67657"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"cerif:openaire","label":"Cerif Openaire","values":["<Publication xmlns=\"https://www.openaire.eu/cerif-profile/1.1/\" id=\"dc93d6a4-18c2-4c0e-b106-fe26f7d69877\"> <Type xmlns=\"https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types\">http://purl.org/coar/resource_type/c_1843</Type> <Language>en_US</Language> <Title>On Some Fundamental Aspects of Learning in Artificial Neural Networks and Universal Channel Codes</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2019-08-30</PublicationDate> <DOI>https://doi.org/10.7298/6a79-d633</DOI> <Authors> <Author> <DisplayName>Gao, Yuguang</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <OrgUnit /> </Publisher> </Publishers> <Keyword>Electrical engineering</Keyword> <Abstract>This thesis discusses a few interesting topics regarding fundamental aspects of learning in the following prevalent application scenarios: (1) training a neural network for image classification, and (2) using a channel code for universal communication at capacity. For the classification problem, we aim to develop a better understanding of what representations each layer of the network learns. In particular, we compare the higher-layer activations of two neural networks with identical architecture but different initializations via adaptive $k$NN graph approximation of the underlying manifold, and we show that there are vast similarities between the underlying manifolds of the two networks but with discrepancy in potentially highly-curved regions. We also investigate locality of the receptive field in the Convolutional Neural Networks by using semi-localized filters with random neuron connection, where we find out that the receptive field might be beyond local for feature extraction as is hard coded in traditional design. For the communication problem, we study universal channel coding under the high-dimensional statistical setting beyond Shannon's classical framework, and we prove a series of theorems that may surprisingly indicate a need to learn the entire channel in order to achieve its capacity.</Abstract> <Access xmlns=\"http://purl.org/coar/access_right\" > </Access> </Publication>"]},{"key":"cris:virtual.author","label":"Cris Virtual Author","values":["Gao, Yuguang"]},{"key":"cris:virtualsource.collection","label":"Cris Virtualsource Collection","values":["5893a6ea-7af3-41d7-abc6-04bcd26ab5df"]},{"key":"dc:description.abstract","label":"Abstract","values":["This thesis discusses a few interesting topics regarding fundamental aspects of learning in the following prevalent application scenarios: (1) training a neural network for image classification, and (2) using a channel code for universal communication at capacity. For the classification problem, we aim to develop a better understanding of what representations each layer of the network learns. In particular, we compare the higher-layer activations of two neural networks with identical architecture but different initializations via adaptive $k$NN graph approximation of the underlying manifold, and we show that there are vast similarities between the underlying manifolds of the two networks but with discrepancy in potentially highly-curved regions. We also investigate locality of the receptive field in the Convolutional Neural Networks by using semi-localized filters with random neuron connection, where we find out that the receptive field might be beyond local for feature extraction as is hard coded in traditional design. For the communication problem, we study universal channel coding under the high-dimensional statistical setting beyond Shannon's classical framework, and we prove a series of theorems that may surprisingly indicate a need to learn the entire channel in order to achieve its capacity."]},{"key":"dc:description.collection","label":"Dc Description Collection","values":["Cornell Theses and Dissertations"]},{"key":"dc:format.mimetype","label":"Dc Format Mimetype","values":["application/pdf"]},{"key":"dcterms:license","label":"Dcterms License","values":["https://hdl.handle.net/1813/59810"]},{"key":"dc:title","label":"Title","values":["On Some Fundamental Aspects of Learning in Artificial Neural Networks and Universal Channel Codes"]},{"key":"dspace:entity.type","label":"Dspace Entity Type","values":["Publication"]},{"key":"others:access_status","label":"Others Access Status","values":["open.access"]}]}],"canonical_facts":{"cerif:openaire":["<Publication xmlns=\"https://www.openaire.eu/cerif-profile/1.1/\" id=\"dc93d6a4-18c2-4c0e-b106-fe26f7d69877\"> <Type xmlns=\"https://www.openaire.eu/cerif-profile/vocab/COAR_Publication_Types\">http://purl.org/coar/resource_type/c_1843</Type> <Language>en_US</Language> <Title>On Some Fundamental Aspects of Learning in Artificial Neural Networks and Universal Channel Codes</Title> <PublishedIn> <Publication> </Publication> </PublishedIn> <PublicationDate>2019-08-30</PublicationDate> <DOI>https://doi.org/10.7298/6a79-d633</DOI> <Authors> <Author> <DisplayName>Gao, Yuguang</DisplayName> <Affiliation> <OrgUnit> </OrgUnit> </Affiliation> </Author> </Authors> <Editors> </Editors> <Publishers> <Publisher> <OrgUnit /> </Publisher> </Publishers> <Keyword>Electrical engineering</Keyword> <Abstract>This thesis discusses a few interesting topics regarding fundamental aspects of learning in the following prevalent application scenarios: (1) training a neural network for image classification, and (2) using a channel code for universal communication at capacity. For the classification problem, we aim to develop a better understanding of what representations each layer of the network learns. In particular, we compare the higher-layer activations of two neural networks with identical architecture but different initializations via adaptive $k$NN graph approximation of the underlying manifold, and we show that there are vast similarities between the underlying manifolds of the two networks but with discrepancy in potentially highly-curved regions. We also investigate locality of the receptive field in the Convolutional Neural Networks by using semi-localized filters with random neuron connection, where we find out that the receptive field might be beyond local for feature extraction as is hard coded in traditional design. For the communication problem, we study universal channel coding under the high-dimensional statistical setting beyond Shannon's classical framework, and we prove a series of theorems that may surprisingly indicate a need to learn the entire channel in order to achieve its capacity.</Abstract> <Access xmlns=\"http://purl.org/coar/access_right\" > </Access> </Publication>"],"cris:virtual.author":["Gao, Yuguang"],"cris:virtualsource.collection":["5893a6ea-7af3-41d7-abc6-04bcd26ab5df"],"dc:contributor.chair":["Hopcroft, John E."],"dc:contributor.committeemember":["Tong, Lang","Kleinberg, Robert David"],"dc:creator":["Gao, Yuguang"],"dc:date.accessioned":["2019-10-15T16:49:21Z"],"dc:date.available":["2021-08-29T06:00:18Z"],"dc:date.issued":["2019-08-30"],"dc:description.abstract":["This thesis discusses a few interesting topics regarding fundamental aspects of learning in the following prevalent application scenarios: (1) training a neural network for image classification, and (2) using a channel code for universal communication at capacity. For the classification problem, we aim to develop a better understanding of what representations each layer of the network learns. In particular, we compare the higher-layer activations of two neural networks with identical architecture but different initializations via adaptive $k$NN graph approximation of the underlying manifold, and we show that there are vast similarities between the underlying manifolds of the two networks but with discrepancy in potentially highly-curved regions. We also investigate locality of the receptive field in the Convolutional Neural Networks by using semi-localized filters with random neuron connection, where we find out that the receptive field might be beyond local for feature extraction as is hard coded in traditional design. For the communication problem, we study universal channel coding under the high-dimensional statistical setting beyond Shannon's classical framework, and we prove a series of theorems that may surprisingly indicate a need to learn the entire channel in order to achieve its capacity."],"dc:description.collection":["Cornell Theses and Dissertations"],"dc:format.mimetype":["application/pdf"],"dc:identifier.bibid":["11050641"],"dc:identifier.doi":["https://doi.org/10.7298/6a79-d633"],"dc:identifier.other":["ProQuest Submission ID: 11340","ProQuest Publication ID: 13857602"],"dc:identifier.uri":["https://hdl.handle.net/1813/67657"],"dc:language.iso":["en_US"],"dc:subject":["Electrical engineering"],"dcterms:license":["https://hdl.handle.net/1813/59810"],"dc:title":["On Some Fundamental Aspects of Learning in Artificial Neural Networks and Universal Channel Codes"],"dc:type":["dissertation or thesis"],"dspace:entity.type":["Publication"],"others:access_status":["open.access"],"thesis:degree_discipline":["Electrical and Computer Engineering"],"thesis:degree_level":["Doctor of Philosophy"],"thesis:degree_name":["Ph.D., Electrical and Computer Engineering"],"thesis:institution_name":["Cornell University"]},"updated_at":"2026-11-19T03:20:50Z"}