University of British Columbia
Exploring internally dense but externally sparse deep convolutional neural networks
Abstract
dc:descriptionRecent years have witnessed two seemingly opposite developments of deep convolutional neural networks (CNNs). On the one hand, increasing the density of CNNs (e.g., by adding cross-layer connections) achieves better performance on basic computer vision tasks. On the other hand, creating sparsity structures (e.g., through pruning methods) achieves slimmer network structures. Inspired by modularity structures in the human brain, the researchers bridge these two trends by proposing a new network structure with internally dense yet externally sparse connections. Experimental results demonstrate that the proposed method could obtain competitive performance on benchmark tasks (CIFAR10, CIFAR100, and ImageNet) while keeping the network slim. Moreover, the researchers give out connection contributions by implementing a network damage experiment, which could indicate brief principles for hierarchical deep neural network design. The proposed method could achieve a 22% top-1 error on ImageNet classification at the computational budget of 3.38 GFLOPs. This means the proposed method could achieve 2% higher recognition accuracy than ResNet50 while only using 80% of the computation cost of it.
Degree
thesis:*- Name thesis:degree_name
- Master of Applied Science - MASc
- Level thesis:degree_level
- master's
- Discipline thesis:degree_discipline
- Electrical Engineering
- Grantor dc:publisher
- University of British Columbia
- Year dc:date
- 2019
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Duan, Yiqun
Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-NoDerivatives 4.0 International
- Language dc:language
- eng
Identifiers
dc:identifier.*- Handle dc:identifier
- http://hdl.handle.net/2429/72063