Back to search

University of Alabama Birmingham

Knowledge Distillation for Efficient Object Detection: Toward Scalable and Deployable Vision Models

Abstract

dc:description.abstract

Object detection is a critical component of autonomous driving, requiring real-time, robust perception to ensure safety. However, state-of-the-art deep neural network object detectors typically incur high computational cost and memory footprint, hindering their deployment in resource-constrained environments such as self-driving vehicles. This dissertation addresses the need for efficient yet accurate detectors by leveraging knowledge distillation (KD), a model compression technique that transfers knowledge from a high-capacity teacher model to a lightweight student model. While KD has seen success in image classification, its application to object detection poses unique challenges due to multiple instances per image and complex output structures. To overcome these challenges, this thesis presents five novel KD frameworks tailored for object detection. First, Adaptive Instance Distillation (AID) selectively weights the distillation of each object instance based on the teacher’s prediction confidence, enabling the student to focus on reliably learned knowledge. Second, Multi-Teacher AID (MAID) aggregates complementary knowledge from multiple teachers to provide richer supervision for the student. Third, Gradient-Guided KD (GKD) leverages teacher gradient information to prioritize critical features that most impact the detection loss, thereby guiding the student to imitate the most pertinent representations. Fourth, CLoCKDistill (Consistent Location-and-Context-aware KD) addresses transformer-based detectors (DETRs) by distilling global context from the teacher’s transformer encoder and aligning teacher–student attention on object locations for more effective knowledge transfer. Finally, ACAM-KD (Adaptive and Cooperative Attention Masking for KD) introduces an interactive distillation process, in which student–teacher feature maps are adaptively fused via cross-attention and dynamically masked to highlight important spatial and channel-wise information. Extensive experiments on benchmark datasets (e.g., KITTI and COCO) demonstrate the efficacy of these approaches. The proposed techniques consistently boost detection accuracy (achieving up to 6% mAP gains) while substantially reducing model complexity. The resulting student models – including one-stage, two-stage, and transformer-based detectors – attain performance comparable to much larger teachers at a fraction of the computation, contributing to the development of scalable and deployable vision systems.

Degree

thesis:*
Name thesis:degree_name
Doctor of Philosophy (PhD)
Level thesis:degree_level
Dissertation
Discipline thesis:degree_discipline
Computer and Information Sciences

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Lan, Qizhen
Contributors dc:contributor
  • <p>Qing Tian</p>
  • <p>Baocheng Geng<br>Chengcui Zhang<br>Rachel Smith<br>Tianyang Wang</p>

Subjects

dc:subject × 6

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
University of Alabama Birmingham
Base URL
digitalcommons.library.uab.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Lan, Qizhen. Knowledge Distillation for Efficient Object Detection: Toward Scalable and Deployable Vision Models. Dissertation thesis, https://digitalcommons.library.uab.edu/etd-collection/7235