Back to search

University of Patras

Motion and geometry-aware methods for automatic sign language processing

Abstract

dc:description

Automatic Sign Language Processing aims to interpret signed communication from visual input, enabling accessible interaction between Deaf and hearing communities. Despite substantial progress driven by deep learning and large-scale datasets, robust sign language processing remains challenging due to the fine-grained, highly structured, and multimodal nature of signed articulation. In particular, many contemporary approaches rely predominantly on data-driven temporal modeling of pose sequences, often overlooking the underlying geometric and kinematic structure of human motion that governs sign production. This dissertation investigates geometry and motion aware methods for automatic sign language processing, with a focus on pose-based representations derived from skeletal keypoints. The proposed approaches make use of geometric information extracted from hand and wrist motion and hand skeletal pose for sign language video summarization and recognition. First, the thesis introduces training-free motion-based summarization techniques for continuous sign language videos. Using wrist motion trajectories, a keyframe selection method based on time-parameterized curvature is proposed to extract lexically salient frames while significantly reducing temporal redundancy. This approach is extended to three dimensions, incorporating both curvature and torsion to capture bending and twisting motion. Extensive quantitative and human-based evaluations demonstrate that the proposed summarization methods preserve semantic content more effectively than existing baselines, while remaining lightweight and interpretable. Second, the dissertation addresses geometry-aware modeling of handshape and skeletal motion for sign recognition. Complementary pose-level distance measures are utilized, including point cloud-based handshape distances and an articulated pose distance that respects hierarchical kinematic structure. Building on these formulations, a self-supervised distance-preserving embedding framework is introduced to transfer explicit geometric similarity into compact latent representations. These embeddings are integrated into Transformer-based recognition models, where they act as strong inductive biases and yield consistent improvements in recognition accuracy on large-scale benchmarks. Overall, this work demonstrates that explicitly incorporating geometric and kinematic priors into sign language processing pipelines enhances both motion summarization and recognition performance. The findings highlight the value of geometry-informed representations as a principled bridge between human articulatory structure and modern learning-based models, contributing toward more efficient, robust, and interpretable sign language technologies.

Degree

thesis:*
Grantor dc:publisher
University of Patras
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Sartinas, Evangelos
  • Σαρτίνας, Ευάγγελος

Subjects

dc:subject × 18

Rights

Language dc:language
eng

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Sartinas, Evangelos; Σαρτίνας, Ευάγγελος. Motion and geometry-aware methods for automatic sign language processing. University of Patras, 2026. http://hdl.handle.net/10442/hedi/61154