Back to search

Institutional Repository of Vilnius University

Vision transformer fine-tuning for image similarity using metric learning /

Abstract

dc:description

Image similarity detection is a fundamental computer vision task applied in content moderation, copyright enforcement, and misinformation tracking. This thesis investigates how different metric learning loss functions affect the fine­tuning of a DINOv2 Vision Transformer for image similarity tasks. Experiments are conducted on the DISC21 dataset, a large­scale benchmark for image copy detection. Five loss functions—contrastive, triplet, ArcFace, Focal, and CLIP/InfoNCE—are evaluated across two DINOv2 backbone variants: ViT­S/14 (21M parameters) and ViT­B/14 (86M parameters), yielding 12 experimental configurations. Models are assessed using Precision@K, mean Average Precision (mAP), and micro Average Precision (μAP) metrics. Embedding space structure is further examined through UMAP projections and cosine similarity distribution histograms. Results show that CLIP/InfoNCE achieves the best performance on both backbones: ViTS P@1 = 60.49%, ViT­B P@1 = 61.63%. Triplet loss provides no learning signal for either backbone, as DINOv2’s pre­trained embeddings already satisfy the margin constraint. The ViT­B backbone does not uniformly outperform ViT­S: contrastive loss benefits substantially from the larger backbone (+4.91 percentage points in P@1), while the advantage is negligible for triplet loss.

Degree

thesis:*
Grantor dc:publisher
Institutional Repository of Vilnius University
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Mielkus, Vytautas,

Subjects

dc:subject × 1

Rights

dc:rights
Statement dc:rights
  • info:eu-repo/semantics/openAccess
Language dc:language
eng

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Vilnius University
Base URL
epublications.vu.lt/oai
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Mielkus, Vytautas,. Vision transformer fine-tuning for image similarity using metric learning /. Institutional Repository of Vilnius University, 2026. https://repository.vu.lt/VU:ELABAETD308338014&prefLang=en_US