Back to search

Boston University

Neural network interpretations as means for model analysis and improvement

Abstract

dc:description.abstract

With the prevalent use of deep neural networks in a wide variety of applications, they mostly remain black-box tools. Replacing hand-crafted feature extraction algorithms with representations learned by deep neural networks increases the gap between the inventor’s intuition of the model’s algorithm and the actual algorithm learned by the model. While these models excel in performance, they lack interpretability, which is essential for transparency and accountability in AI systems, as mandated by regulations such as the GDPR in the EU and the Executive Order on AI in the US.Interpretable Machine Learning and Explainable Artificial Intelligence (XAI) are the areas of research that aim to reclaim some of that lost intuition back and provide the tools that enable the analysis of the neural model’s behavior. First, we focus on the task of calculating input feature importance in the context of Computer Vision models for Image Classification, Object Detection, and Image Captioning.We propose novel methods for feature attribution that advance the state of the art. We introduce new metrics for evaluating importance maps for images. The proposed metrics do not require human intervention or labels, which may result in human bias in evaluating explanations, which was common in previous works. The proposed methods can be used in a variety of applications extending the tools available for model analysis and improvement. Second, we explore the interpretability aspects of generative text-to-image models. We analyze concept grounding and manipulation --- another line of explainability research that is very relevant in light of the growing number of text-to-image applications. To prevent the models from generating undesired content, such as explicit, private, or copyrighted materials, methods for discovering and inhibiting such concepts within the models are being actively developed by the research community. We propose a theoretical framework that analyzes the limitations of concept inhibition methods and design attacks that can exploit these limitations. Our theoretical findings are confirmed with empirical evaluations on various inhibition methods, showing that the attacks are effective against the state-of-the-art methods. The developed theoretical framework points to the properties that the concept inhibition methods should satisfy to be effective and robust. Overall, this thesis contributes to the ongoing research efforts toward developing more intuitive and transparent AI systems, balancing the power of learned representations with the interpretability required for trust and accountability in AI applications.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy
Level thesis:degree_level
doctoral
Discipline thesis:degree_discipline
Computer Science
Grantor
Boston University
Year dc:date.issued
2024

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Petsiuk, Vitali
Advisor dc:contributor.advisor
  • Saenko, Kate

Subjects

dc:subject × 1

Rights

dc:rights
Statement dc:rights
  • Attribution 4.0 International
Language dc:language.iso
en_US

Identifiers

dc:identifier.*
Handle dc:identifier.uri
https://hdl.handle.net/2144/53106

Chain of custody

source
Harvested from
Boston University
Base URL
open.bu.edu/oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Petsiuk, Vitali. Neural network interpretations as means for model analysis and improvement. doctoral thesis, Boston University, 2024. https://hdl.handle.net/2144/53106