Back to search

University of North Texas

Probabilistic Temporal Correspondence Learning for Self-Supervised Video Alignment

Abstract

dc:description

Video alignment, the task of establishing temporal correspondences between frames of videos that depict the same activity, is a fundamental building block for applications ranging from action recognition to rehabilitation monitoring and robotic imitation learning. Self-supervised methods based on temporal cycle-consistency make fine-grained alignment possible without frame-level annotations, but they rest on assumptions that break down in realistic settings: they presume a single one-to-one correspondence for every frame, they assume that every frame is alignable, and they rely on local features that cannot disambiguate visually identical phases of repetitive actions. This dissertation introduces Probabilistic Temporal Cycle Consistency (PTCC), a self-supervised framework that addresses these limitations. First, PTCC replaces the unimodal softmax of standard cycle-consistency with a Gaussian mixture model, representing frame correspondences as a multi-modal distribution so that one frame can match every valid repetition of an action. Second, a parameter-free bidirectional verification mechanism identifies and down-weights non-alignable outlier frames, such as background or idle segments, without additional supervision. Third, a Transformer-based temporal encoder gives each frame a global temporal receptive field, allowing the model to distinguish repetitions that are separated in time yet visually similar. We evaluate PTCC on the Penn Action and Multiview Pouring datasets using six complementary metrics. PTCC establishes a new state of the art on both, achieving 82.65% phase alignment accuracy and a Kendall's Tau of 0.99 on Penn Action and reducing the cycle consistency error by over 97% relative to standard temporal cycle-consistency. Ablation and qualitative studies confirm the contribution of each component.

Degree

thesis:*
Grantor dc:publisher
University of North Texas
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Liu, Siyuan 1984-
Contributors dc:contributor
  • Yang, Qing
  • Guo, Xuan
  • Yuan, Xiaohui
  • Morozov, Kirill

Subjects

dc:subject × 3

Rights

dc:rights
Statement dc:rights
  • Public
  • Liu, Siyuan 1984-
  • Copyright
  • Copyright is held by the author, unless otherwise noted. All rights Reserved.
Language dc:language
English

Identifiers

dc:identifier.*
Identifier
local-cont-no: submission_5065
ark: ark:/67531/metadc2652554

Chain of custody

source
Harvested from
University of North Texas
Base URL
digital.library.unt.edu/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Liu, Siyuan 1984-. Probabilistic Temporal Correspondence Learning for Self-Supervised Video Alignment. University of North Texas, 2026. https://digital.library.unt.edu/ark:/67531/metadc2652554/