{"id":{"repo_id":"sdstate","oai_identifier":"oai:openprairie.sdstate.edu:etd2-2745"},"canonical_url":"https://search.ndltd.org/etd/sdstate/oai:openprairie.sdstate.edu:etd2-2745","repository":{"repo_id":"sdstate","name":"South Dakota State University","base_url":"https://openprairie.sdstate.edu/do/oai/"},"display":{"title":"Video Comprehension Score (VCS): A Metric for Long-Form Video Description Evaluation","abstract":"Existing video description evaluation metrics fail to capture the long-range chronology and semantic alignment essential for long-form descriptions. An effective evaluation metric for long-form descriptions must (i) assess global thematic alignment, (ii) measure local semantic alignment, and (iii) evaluate chronological alignment while detecting corrupted content. We introduce Video Comprehension Score (VCS), a reference-based metric, which directly addresses these evaluation requirements through three components: Global Alignment Score for thematic alignment, Local Alignment Score for local semantic alignment, and Narrative Alignment Score for chronological alignment with adjustable tolerance. We evaluate VCS on two large-scale synthetic datasets designed to test corruption detection and cross-author consistency. VCS consistently outperforms traditional metrics on corruption detection tasks, being the only metric capable of distinguishing valid variations from invalid corruptions. On cross-author consistency tasks, VCS is the only metric that consistently produces scores >80% regardless of which authorial reference is used for evaluation. VCSshort, our implementation for short-form descriptions, attains state-of-the-art human correlation on VATEX-EVAL in the 9-ref setting (Kendall’s τ = 41.5, Spearman’s ρ = 52.8) and competitive results in the 1-ref setting (Kendall’s τ = 30.0, Spearman’s ρ = 38.1). These results demonstrate VCS effectiveness for evaluating both long-form and short-form video descriptions.","abstract_html":"Existing video description evaluation metrics fail to capture the long-range chronology and semantic alignment essential for long-form descriptions. An effective evaluation metric for long-form descriptions must (i) assess global thematic alignment, (ii) measure local semantic alignment, and (iii) evaluate chronological alignment while detecting corrupted content. We introduce Video Comprehension Score (VCS), a reference-based metric, which directly addresses these evaluation requirements through three components: Global Alignment Score for thematic alignment, Local Alignment Score for local semantic alignment, and Narrative Alignment Score for chronological alignment with adjustable tolerance. We evaluate VCS on two large-scale synthetic datasets designed to test corruption detection and cross-author consistency. VCS consistently outperforms traditional metrics on corruption detection tasks, being the only metric capable of distinguishing valid variations from invalid corruptions. On cross-author consistency tasks, VCS is the only metric that consistently produces scores &gt;80% regardless of which authorial reference is used for evaluation. VCSshort, our implementation for short-form descriptions, attains state-of-the-art human correlation on VATEX-EVAL in the 9-ref setting (Kendall’s τ = 41.5, Spearman’s ρ = 52.8) and competitive results in the 1-ref setting (Kendall’s τ = 30.0, Spearman’s ρ = 38.1). These results demonstrate VCS effectiveness for evaluating both long-form and short-form video descriptions.","abstract_has_math":false,"creators":["Dubey, Harsh"],"institution":null,"degree_name":"Master of Science (MS)","degree_level":"Thesis - Open Access","degree_discipline":"Electrical Engineering and Computer Science","degree_department":null,"school":null,"contributors":["Chulwoo Pack"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T09:05:44Z","subjects":["Computer Sciences","Electrical and Computer Engineering"],"languages":[],"rights":[],"rights_urls":["http://rightsstatements.org/vocab/InC/1.0/"],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://openprairie.sdstate.edu/etd2/1722","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://openprairie.sdstate.edu/etd2/1722","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://openprairie.sdstate.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Aopenprairie.sdstate.edu%3Aetd2-2745","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Chulwoo Pack"]},{"key":"dc:creator","label":"Author","values":["Dubey, Harsh"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2025-08-20T07:00:00Z"]},{"key":"dc:date.created","label":"Dc Date Created","values":["2025-01-01T08:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical Engineering and Computer Science"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis - Open Access"]},{"key":"thesis:degree_name","label":"Degree Name","values":["Master of Science (MS)"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Computer Sciences","Electrical and Computer Engineering"]}]},{"id":"language_rights","label":"Language and Rights","entries":[{"key":"dc:rights","label":"Dc Rights","values":["http://rightsstatements.org/vocab/InC/1.0/"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://openprairie.sdstate.edu/etd2/1722"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Existing video description evaluation metrics fail to capture the long-range chronology and semantic alignment essential for long-form descriptions. An effective evaluation metric for long-form descriptions must (i) assess global thematic alignment, (ii) measure local semantic alignment, and (iii) evaluate chronological alignment while detecting corrupted content. We introduce Video Comprehension Score (VCS), a reference-based metric, which directly addresses these evaluation requirements through three components: Global Alignment Score for thematic alignment, Local Alignment Score for local semantic alignment, and Narrative Alignment Score for chronological alignment with adjustable tolerance. We evaluate VCS on two large-scale synthetic datasets designed to test corruption detection and cross-author consistency. VCS consistently outperforms traditional metrics on corruption detection tasks, being the only metric capable of distinguishing valid variations from invalid corruptions. On cross-author consistency tasks, VCS is the only metric that consistently produces scores >80% regardless of which authorial reference is used for evaluation. VCSshort, our implementation for short-form descriptions, attains state-of-the-art human correlation on VATEX-EVAL in the 9-ref setting (Kendall’s τ = 41.5, Spearman’s ρ = 52.8) and competitive results in the 1-ref setting (Kendall’s τ = 30.0, Spearman’s ρ = 38.1). These results demonstrate VCS effectiveness for evaluating both long-form and short-form video descriptions."]},{"key":"dc:title","label":"Title","values":["Video Comprehension Score (VCS): A Metric for Long-Form Video Description Evaluation"]}]}],"canonical_facts":{"dc:contributor":["Chulwoo Pack"],"dc:creator":["Dubey, Harsh"],"dc:date.available":["2025-08-20T07:00:00Z"],"dc:date.created":["2025-01-01T08:00:00Z"],"dc:description.abstract":["Existing video description evaluation metrics fail to capture the long-range chronology and semantic alignment essential for long-form descriptions. An effective evaluation metric for long-form descriptions must (i) assess global thematic alignment, (ii) measure local semantic alignment, and (iii) evaluate chronological alignment while detecting corrupted content. We introduce Video Comprehension Score (VCS), a reference-based metric, which directly addresses these evaluation requirements through three components: Global Alignment Score for thematic alignment, Local Alignment Score for local semantic alignment, and Narrative Alignment Score for chronological alignment with adjustable tolerance. We evaluate VCS on two large-scale synthetic datasets designed to test corruption detection and cross-author consistency. VCS consistently outperforms traditional metrics on corruption detection tasks, being the only metric capable of distinguishing valid variations from invalid corruptions. On cross-author consistency tasks, VCS is the only metric that consistently produces scores >80% regardless of which authorial reference is used for evaluation. VCSshort, our implementation for short-form descriptions, attains state-of-the-art human correlation on VATEX-EVAL in the 9-ref setting (Kendall’s τ = 41.5, Spearman’s ρ = 52.8) and competitive results in the 1-ref setting (Kendall’s τ = 30.0, Spearman’s ρ = 38.1). These results demonstrate VCS effectiveness for evaluating both long-form and short-form video descriptions."],"dc:identifier":["https://openprairie.sdstate.edu/etd2/1722"],"dc:rights":["http://rightsstatements.org/vocab/InC/1.0/"],"dc:subject":["Computer Sciences","Electrical and Computer Engineering"],"dc:title":["Video Comprehension Score (VCS): A Metric for Long-Form Video Description Evaluation"],"thesis:degree_discipline":["Electrical Engineering and Computer Science"],"thesis:degree_level":["Thesis - Open Access"],"thesis:degree_name":["Master of Science (MS)"]},"updated_at":"2026-11-19T09:05:44Z"}