Institutional Repository of Vilnius University
Vision transformer fine-tuning for image similarity using metric learning /
Abstract
dc:descriptionImage similarity detection is a fundamental computer vision task applied in content moderation, copyright enforcement, and misinformation tracking. This thesis investigates how different metric learning loss functions affect the finetuning of a DINOv2 Vision Transformer for image similarity tasks. Experiments are conducted on the DISC21 dataset, a largescale benchmark for image copy detection. Five loss functions—contrastive, triplet, ArcFace, Focal, and CLIP/InfoNCE—are evaluated across two DINOv2 backbone variants: ViTS/14 (21M parameters) and ViTB/14 (86M parameters), yielding 12 experimental configurations. Models are assessed using Precision@K, mean Average Precision (mAP), and micro Average Precision (μAP) metrics. Embedding space structure is further examined through UMAP projections and cosine similarity distribution histograms. Results show that CLIP/InfoNCE achieves the best performance on both backbones: ViTS P@1 = 60.49%, ViTB P@1 = 61.63%. Triplet loss provides no learning signal for either backbone, as DINOv2’s pretrained embeddings already satisfy the margin constraint. The ViTB backbone does not uniformly outperform ViTS: contrastive loss benefits substantially from the larger backbone (+4.91 percentage points in P@1), while the advantage is negligible for triplet loss.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Mielkus, Vytautas,
Subjects
dc:subject × 1Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- eng
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD308338014&prefLang=en_US