Back to search

University of Nevada - Reno

Enhancing Infrastructure Monitoring with Calibrated Vision Language Model Ensembles: A Graffiti Detection Case Study

Abstract

dc:description.abstract

Urban transit authorities face significant challenges in efficiently monitoring distributed infrastructure assets. This dissertation presents a novel three-stage pipeline for automated infrastructure condition assessment: (1) GPS-based geofence creation that automatically defines inspection boundaries, (2) YOLOv11-powered detection optimized for transit infrastructure, and (3) damage assessment using a specialized Vision Language Model (VLM) ensemble. Our ensemble approach employs probability calibration and weight optimization, outperforming individual models with an 84.4% F1 score. To address extreme data scarcity, we developed a synthetic graffiti generation methodology, expanding from just 3 real examples to a comprehensive evaluation dataset with balanced color representation. Experiments utilized data collected along 227.58 miles of bus routes covering 778 unique bus stops across the Reno metro area. The system operates efficiently on standard hardware with geofence-triggered processing, making advanced VLM technology accessible for practical transit authority deployment. Key contributions include: (1) a pipeline architecture that reduces computational requirements by 95% compared to continuous processing; (2) ensemble techniques using isotonic regression calibration and differential evolution optimization that significantly improve detection accuracy; (3) a parametric graffiti generation methodology creating realistic synthetic data with controllable characteristics; and (4) a color-specific analysis framework revealing critical insights about model performance across different visual characteristics. Ablation studies demonstrate that while traditional CNN models catastrophically fail on unseen variations, VLMs maintain more consistent behavior across varied visual characteristics, highlighting their superior generalization capabilities. The system transforms infrastructure monitoring from a manual, subjective process that burdens drivers into a data-driven approach supporting proactive maintenance and strategic planning.

Degree

thesis:*
Level thesis:degree_level
Doctorate Degree
Department thesis:degree_department
Computer Science
Year dc:date.issued
2025

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Evangelista, Gaetano Roberti
Advisors dc:contributor.advisor
  • Hand, Emily
  • Kelley, Richard
Committee members dc:contributor.committeemember
  • Papachristos, Christos
  • Nicolescu, Monica
  • Xu, Hao

Subjects

dc:subject × 4

Rights

Language dc:language.iso
en_US, English

Identifiers

dc:identifier.*
Repository record dc:identifier.uri
https://scholarwolf.unr.edu/handle/11714/11733

Chain of custody

source
Harvested from
University of Nevada - Reno
Base URL
scholarwolf.unr.edu/server/oai/request
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Evangelista, Gaetano Roberti. Enhancing Infrastructure Monitoring with Calibrated Vision Language Model Ensembles: A Graffiti Detection Case Study. Doctorate Degree thesis, 2025. https://scholarwolf.unr.edu/handle/11714/11733