Back to search

South Dakota State University

Video Comprehension Score (VCS): A Metric for Long-Form Video Description Evaluation

Abstract

dc:description.abstract

Existing video description evaluation metrics fail to capture the long-range chronology and semantic alignment essential for long-form descriptions. An effective evaluation metric for long-form descriptions must (i) assess global thematic alignment, (ii) measure local semantic alignment, and (iii) evaluate chronological alignment while detecting corrupted content. We introduce Video Comprehension Score (VCS), a reference-based metric, which directly addresses these evaluation requirements through three components: Global Alignment Score for thematic alignment, Local Alignment Score for local semantic alignment, and Narrative Alignment Score for chronological alignment with adjustable tolerance. We evaluate VCS on two large-scale synthetic datasets designed to test corruption detection and cross-author consistency. VCS consistently outperforms traditional metrics on corruption detection tasks, being the only metric capable of distinguishing valid variations from invalid corruptions. On cross-author consistency tasks, VCS is the only metric that consistently produces scores >80% regardless of which authorial reference is used for evaluation. VCSshort, our implementation for short-form descriptions, attains state-of-the-art human correlation on VATEX-EVAL in the 9-ref setting (Kendall’s τ = 41.5, Spearman’s ρ = 52.8) and competitive results in the 1-ref setting (Kendall’s τ = 30.0, Spearman’s ρ = 38.1). These results demonstrate VCS effectiveness for evaluating both long-form and short-form video descriptions.

Degree

thesis:*
Name thesis:degree_name
Master of Science (MS)
Level thesis:degree_level
Thesis - Open Access
Discipline thesis:degree_discipline
Electrical Engineering and Computer Science

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Dubey, Harsh
Contributors dc:contributor
  • Chulwoo Pack

Subjects

dc:subject × 2

Rights

dc:rights

Identifiers

dc:identifier.*
Repository record dc:identifier
https://openprairie.sdstate.edu/etd2/1722

Chain of custody

source
Harvested from
South Dakota State University
Base URL
openprairie.sdstate.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Dubey, Harsh. Video Comprehension Score (VCS): A Metric for Long-Form Video Description Evaluation. Thesis - Open Access thesis, https://openprairie.sdstate.edu/etd2/1722