Institutional Repository of Vilnius University
Analysis of algorithms to reduce hallucinations in vision language models /
Abstract
dc:descriptionVision-Language Models (VLMs) exhibit impressive multimodal capabilities but remain susceptible to hallucinations — fluent yet visually ungrounded or factually incorrect outputs. This thesis analyzes a white-box hallucination detection framework, Multi-Token Reliability Estimation (MTRE), applied to the LLaVA-v1.5-7B architecture, which exploits internal logit dynamics to detect unreliability during generation. Experiments were conducted on the MAD-Bench and POPE benchmarks, investigating the impact of hyperparameters: generation temperature and the token logit sequence length. Results show that in-domain performance peaks at an ultra-narrow window of 2 logits (AUROC of 97.67%), improves substantially as the sequence is expanded to 20 logits. Simultaneous cross-dataset and cross-prompt transfer causes complete classifier collapse. High decoding temperatures introduce arithmetic overflow that is recoverable via sanitization, though boundary calibration remains limited under small training sets. The findings suggest that effective hallucination detection benefits from wider token horizons and is highly sensitive to both prompt format and dataset shifts.
Degree
thesis:*- Grantor dc:publisher
- Institutional Repository of Vilnius University
- Year dc:date
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Bruškytė, Brigita,
Subjects
dc:subject × 1Rights
dc:rights- Statement dc:rights
-
- info:eu-repo/semantics/openAccess
- Language dc:language
- eng
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://repository.vu.lt/VU:ELABAETD308332492&prefLang=en_US