Back to search

University of Washington

Revealing structure in trained neural networks through dimensionality-based methods

Abstract

dc:description.abstract

Neural networks trained by machine learning optimization methods are currently being analyzed to shed light on brain function. While exciting progress is being made, the complicated nature of the network models typically considered has made ``opening the black box'' a significant challenge. In this thesis I approach the problem by starting with network models and tasks that can understood more easily, but that capture fundamental elements of more complex models. I reveal new aspects of the behavior of these models through the lens of effective dimensionality, which quantifies the number of axes needed to describe data. Through this investigation a new idea of "dimensionality balance'' emerges, where neural networks trained with stochastic gradient descent automatically strike a balance between increasing dimensionality (to more easily distinguish between different objects) and decreasing dimensionality (to build invariance to different examples of the same object). Mathematical analysis reveals the core mechanisms that may underlie the effects, and experiments with the image classification network VGG indicate that this balance is a general phenomenon. Finally, I demonstrate how and why dimensionality reduction methods can be used to extract information from network weights in a simple model, laying some guiding principles for ways of extracting insights from the recent explosion of brain connectomics data.

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Farrell, Matthew Stuart
Advisor dc:contributor.advisor
  • Shea-Brown, Eric

Subjects

dc:subject × 8

Rights

dc:rights
Statement dc:rights
  • CC BY
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/1773/46368

Chain of custody

source
Harvested from
University of Washington
Base URL
digital.lib.washington.edu/server/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Farrell, Matthew Stuart. Revealing structure in trained neural networks through dimensionality-based methods. 2020. http://hdl.handle.net/1773/46368