South Dakota State University
Refining the Inverse Lipschitz Constant for Injective ReLU Networks
Abstract
dc:description.abstractIn this thesis, we study the Inverse Lipschitz Constant (ILC) of injective ReLU layers. We study the tightness of the ILC lower bound established in Puthawala et al. Our approach has three components. First, we find that the conditions for injectivity on lines yield a weaker condition than the general condition given in Puthawala et al. Second, we perform numerical experiments to judge the tightness of the existing ILC lower bound and find that bound is overly conservative. Third, we identify the source of the potential slack in the proof of the existing ILC bound, and perform further numerical experiments to support this hypothesis. An accurate ILC is crucial for better understanding and potentially reducing the network’s sensitivity to input variations, which is essential for improving the performance and stability of neural networks in real-world applications.
Degree
thesis:*- Name thesis:degree_name
- Master of Science (MS)
- Level thesis:degree_level
- Thesis - Open Access
- Discipline thesis:degree_discipline
- Mathematics and Statistics
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Rausch, Cole
- Contributors dc:contributor
-
- Michael Puthawala
Subjects
dc:subject × 2Rights
dc:rightsIdentifiers
dc:identifier.*- Repository record dc:identifier
- https://openprairie.sdstate.edu/etd2/1175