Back to search

University of York

Deep learning with query sensitive attention mechanisms for content-based image retrieval

Abstract

dc:description.abstract

Content-based Image Retrieval (CBIR) is the task of searching for the most similar images to the query content from an extensive image database. Most existing feature extraction methods and attention mechanisms for CBIR tasks are query non-sensitive, ignoring the specifics of the query pattern, which may lead to focusing on irrelevant regions to the query content. In this thesis, we explore query sensitive attention mechanisms for CBIR task, which involves query feature information in the feature extraction procedure of the candidate image. Firstly, we propose the Conditional Attention Network (CANet). CANet takes the query image and a candidate image as input, resulting in a co-attention map of the candidate image under the condition of the query content. The generated co-attention map could correctly highlight the target object and improve image retrieval performance when embedded into a convolution neural network (CNN) based feature extraction pipeline. Secondly, another more efficient co-attention method is proposed based on local feature selection and clustering over candidate local features. Using local feature selection and clustering dramatically reduces the computation costs caused by the query sensitivity but still leads to accurate co-attention maps even under challenging situations. The proposed clustering-based co-attention method leads to new state-of-the-art performance on several benchmark datasets. Lastly, we explore using clustered expressive local features to perform many-to-many local feature matching for CBIR. We show that the proposed local feature matching method implicitly generates co-attention-like local matching maps. In addition, a trainable binary encoding layer is applied for network fine-tuning, enabling the model to generate compact binary codes with slight performance degradation and greatly reducing computation costs. In summary, we demonstrate that the query information could play an important role in feature extraction for the CBIR task. With a simple design, co-attention could be practical and effective even for large-scale image retrieval tasks.

Degree

thesis:*
Name dc:type.qualificationname
Ph.D
Level dc:type.qualificationlevel
doctoral
Grantor dc:publisher.institution
University of York
Year dc:date.issued
2022

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Hu, Zechao
Advisor dc:contributor.advisor
  • Bors, Adrian

Rights

Language dc:language.dcterms_iso639_2
eng

Identifiers

dc:identifier.*
Identifier
uk.bl.ethos.875104

Chain of custody

source
Harvested from
White Rose University Consortium
Base URL
etheses.whiterose.ac.uk/cgi/oai2
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
related terms
citation

Hu, Zechao. Deep learning with query sensitive attention mechanisms for content-based image retrieval. doctoral thesis, University of York, 2022.