Back to search

University of Waterloo

Correlated Noise in Deep Convolutional Neural Networks

Abstract

dc:description.abstract

This thesis explores one of the differences between the visual cortex and deep convolutional neural networks, namely, correlated fluctuations of neuron response strength. First, we describe the similarities and differences between biological and artificial neural networks and provide motivation for bridging the gap between the two, with a focus on correlated variability. Next, we present a regularisation method for convolutional neural networks using correlated noise. The structure of the correlations are inspired from biological observations using two factors: spatial extent and tuning similarity of neurons. We provide empirical results on improved robust image classification in the setting of occluded and corrupted images. We then move on to studying the connection between popular dataset augmentation techniques and correlated variability. We compute a smooth estimate of the correlations of neural network activity as a function of distance and kernel similarity, and show the similarity to biological correlations. Finally, we introduce a structured form of Dropout for convolutional neural networks, taking into account spatial and kernel correlations. We show improvement in image classification for the VGG architecture on the CIFAR-10 dataset.

Degree

thesis:*
Grantor dc:publisher
University of Waterloo
Year dc:date.issued
2019

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Dutta, Shamak
Advisors dc:contributor.advisor
  • Tripp, Bryan
  • Taylor, Graham

Subjects

dc:subject × 4

Rights

Language dc:language.iso
en

Identifiers

dc:identifier.*
Handle dc:identifier.uri
http://hdl.handle.net/10012/14967

Chain of custody

source
Harvested from
University of Waterloo
Base URL
dspacemainprd01.lib.uwaterloo.ca/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Dutta, Shamak. Correlated Noise in Deep Convolutional Neural Networks. University of Waterloo, 2019. http://hdl.handle.net/10012/14967