Boston University
Neural network interpretations as means for model analysis and improvement
Abstract
dc:description.abstractWith the prevalent use of deep neural networks in a wide variety of applications, they mostly remain black-box tools. Replacing hand-crafted feature extraction algorithms with representations learned by deep neural networks increases the gap between the inventor’s intuition of the model’s algorithm and the actual algorithm learned by the model. While these models excel in performance, they lack interpretability, which is essential for transparency and accountability in AI systems, as mandated by regulations such as the GDPR in the EU and the Executive Order on AI in the US.Interpretable Machine Learning and Explainable Artificial Intelligence (XAI) are the areas of research that aim to reclaim some of that lost intuition back and provide the tools that enable the analysis of the neural model’s behavior. First, we focus on the task of calculating input feature importance in the context of Computer Vision models for Image Classification, Object Detection, and Image Captioning.We propose novel methods for feature attribution that advance the state of the art. We introduce new metrics for evaluating importance maps for images. The proposed metrics do not require human intervention or labels, which may result in human bias in evaluating explanations, which was common in previous works. The proposed methods can be used in a variety of applications extending the tools available for model analysis and improvement. Second, we explore the interpretability aspects of generative text-to-image models. We analyze concept grounding and manipulation --- another line of explainability research that is very relevant in light of the growing number of text-to-image applications. To prevent the models from generating undesired content, such as explicit, private, or copyrighted materials, methods for discovering and inhibiting such concepts within the models are being actively developed by the research community. We propose a theoretical framework that analyzes the limitations of concept inhibition methods and design attacks that can exploit these limitations. Our theoretical findings are confirmed with empirical evaluations on various inhibition methods, showing that the attacks are effective against the state-of-the-art methods. The developed theoretical framework points to the properties that the concept inhibition methods should satisfy to be effective and robust. Overall, this thesis contributes to the ongoing research efforts toward developing more intuitive and transparent AI systems, balancing the power of learned representations with the interpretability required for trust and accountability in AI applications.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy
- Level thesis:degree_level
- doctoral
- Discipline thesis:degree_discipline
- Computer Science
- Grantor
- Boston University
- Year dc:date.issued
- 2024
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Petsiuk, Vitali
- Advisor dc:contributor.advisor
-
- Saenko, Kate
Subjects
dc:subject × 1Rights
dc:rights- Statement dc:rights
-
- Attribution 4.0 International
- Licence dc:rights.uri
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/2144/53106