University of British Columbia
Detection, attribution, and localization of manipulated images
Abstract
dc:descriptionManipulation of images for malicious purposes like spreading misinformation and fraud poses a serious threat to society. Even worse, with the rapid advancement of deep learning technologies, these manipulated images now spread online quickly and widely. To counter the spread of such manipulated media, this thesis proposes effective solutions for three core tasks of image forensics: detection, attribution, and localization. Detection is the foundational task for image forensics, as it assesses the authenticity of an image. In this thesis, we explore various cues for manipulation detection. We first introduce a novel solution that utilizes the statistics of the Bayer pattern in images to determine their authenticity with high accuracy. We then utilize the power of deep learning techniques, and develop an effective one-class approach to detect synthesized images, training the network using only real images with Bayer pattern. In addition, we leverage information in frequency domain to train a deep learning-based approach that can identify synthesized images that are visually indistinguishable from real ones. Attribution aims at identifying the specific deep learning-based generative model (GM) responsible for creating an image, which is useful in addressing accountability and intellectual property concerns. In this thesis, we propose an effective and robust deep learning-based solution that learns input-adaptive masks to amplify GMs’ artifacts, enabling reliable attribution. This is motivated by our observations that GMs’ artifacts in frequency domain persist in sub-images of the original images and remain detectable even when the images are distorted. Localization pinpoints the exact regions in an image that are manipulated, thus improving the interpretability of image forensics. For this task, we explore the feasibility of using a deep learning network to localize manipulations in both shallow- and deep-fake images. To precisely differentiate between authentic and manipulated pixels, the Vision Mamba network is employed along with a novel Guided Graph Neural Network module to allocate the boundaries between tampered and untouched regions. Through comprehensive evaluations, the methods proposed in this thesis outperform state-of-the-art methods, demonstrating strong effectiveness in detecting, attributing, and localizing manipulated images, ultimately contributing to the fight against image-based misinformation and the enhancement of media integrity.
Degree
thesis:*- Name thesis:degree_name
- Doctor of Philosophy - PhD
- Level thesis:degree_level
- doctoral
- Discipline thesis:degree_discipline
- Electrical and Computer Engineering
- Grantor dc:publisher
- University of British Columbia
- Year dc:date
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Zhang, Junbin
- Contributors dc:contributor
-
- Nasiopoulos, Panos
Rights
dc:rights- Statement dc:rights
-
- Attribution-NonCommercial-NoDerivatives 4.0 International
- Language dc:language
- eng
Identifiers
dc:identifier.*- Handle dc:identifier
- http://hdl.handle.net/2429/92339