Back to search

University of North Texas

Sparse Autoencoder Enhanced Universal Image Restoration with DACLIP and Diffusion Models

Abstract

dc:description

Image restoration recovers high-quality images from degraded inputs affected by environmental conditions, sensor limitations, or content removal. While unified restoration frameworks like DA-CLIP achieve state-of-the-art performance through vision-language guidance and diffusion-based reconstruction, two limitations persist: cross-attended features may contain residual artifacts, and diffusion outputs often lack fine-grained spatial refinement under severe degradations. This thesis presents SAE-Restore, a dual-stage sparse autoencoder framework addressing these limitations through explicit feature denoising and image refinement. SAE-Restore integrates two sparse autoencoders at strategic positions. SAE1 performs feature-space denoising after cross-attention with 8-times expansion and 80 to 90 percent activation sparsity to remove artifacts before diffusion guidance. SAE2 applies image-space refinement after diffusion through residual learning to recover fine details. A learnable fusion module enables end-to-end training of all components while preserving frozen DA-CLIP parameters, reducing trainable parameters by 82 percent. Comprehensive evaluation on six degradation types reveals task-dependent effectiveness. SAE1 improves artifact removal tasks with strongest impact on snowy images at plus 2.29 decibels and low-light at plus 0.99 decibels, but degrades inpainting by 0.80 decibels, revealing that sparsity bottlenecks hinder content synthesis. SAE2 provides minimal benefit at negative 0.06 decibels average. Systematic ablation demonstrates that SAE1 only achieves best performance at 19.81 decibels, outperforming the full pipeline at 19.74 decibels. This research contributes the first integration of sparse autoencoders with vision-language guided diffusion, a multi-stage evaluation framework isolating component contributions, and empirical evidence that artifact removal and content synthesis require different architectural approaches. The findings validate task-dependent benefits and provide guidance for adaptive component routing in unified restoration frameworks.

Degree

thesis:*
Grantor dc:publisher
University of North Texas
Year dc:date
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Dodia, Panam Pareshkumar
Contributors dc:contributor
  • Cui, Xinrui
  • Shi, Weishi
  • Liu, Zhuqing

Subjects

dc:subject × 3

Rights

dc:rights
Statement dc:rights
  • Public
  • Dodia, Panam Pareshkumar
  • Copyright
  • Copyright is held by the author, unless otherwise noted. All rights Reserved.
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
local-cont-no: submission_4620
ark: ark:/67531/metadc2566489

Chain of custody

source
Harvested from
University of North Texas
Base URL
digital.library.unt.edu/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Dodia, Panam Pareshkumar. Sparse Autoencoder Enhanced Universal Image Restoration with DACLIP and Diffusion Models. University of North Texas, 2025. https://digital.library.unt.edu/ark:/67531/metadc2566489/