Back to search

University of Thessaly (UTH)

Automatic sign language recognition algorithms and their incorporation into a greek sign language educational platform

Abstract

dc:description

This Ph.D. Thesis focuses on the problem of automatic Sign Language Recognition (SLR)from video data, with a use-case of learning Greek Sign Language (GSL) through a properly designed platform with integrated functionality of automatic GSL recognition. The SLR problem is complex and challenging, due to the multitude of articulators, the complexity of its formation, the variation of the articulation process among signers, the difficulty of robust visual detection and tracking in a variety of non-ideal environments with often low-quality camera sensors, as well as the lack of large-volume SLR databases. At the same time, the process of learning sign language as a “2nd language” requires a very significant effort and student-teacher interaction both during learning as well as objective assessment of students by their teachers, both lacking appropriate supporting tools. The main innovation and contribution of the Thesis lies in the combination of these two problems, i.e., both in the development of innovative and effective algorithms for SLR and their successful integration into a GSL learning platform, demonstrating success both through the evaluation of recognition on various SLR databases, as well as the evaluation of the learning platform objectively and subjectively by a large number of learner users. Specifically, in this Thesis: • The largest GSL database to date has been collected (at least 3 times longer in duration and number of interpreters than the largest database previously), allowing the training and evaluation of GSL models and their integration into the educational platform, while also made available to the scientific community to further advance the field. • Numerous algorithms have been investigated for visual detection and tracking of signing articulators and then for extracting visual features from them. Further, it was determined which ones are most suitable for SLR, through experiments on isolated sign recognition. Many of these algorithms have not been used before for this particular problem. • Innovative algorithms were developed for SLR, with a primary emphasis on sequential learning architectures. These were used to recognize isolated signs, continuous signing, continuous fingerspelling, and finally “cued speech”. These techniques were evaluated on multiple databases, including the aforementioned GSL one. In many cases the algorithms advanced the state-of-the-art. • Finally, computationally efficient models for GSL recognition (specifically for isolated signing, continuous signing, and continuous fingerspelling) were developed, which were integrated into the GSL educational platform, further aided by appropriate GSL language models. The system was evaluated by a large number of users (GSL students at University of Thessaly and GSL experts), achieving excellent subjective and objective results, the latter being in perfect agreement with the off-line SLR results.

Degree

thesis:*
Grantor dc:publisher
University of Thessaly (UTH)
Year dc:date
2023

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Papadimitriou, Aikaterini
  • Παπαδημητρίου, Αικατερίνη

Subjects

dc:subject × 22

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/56096

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-18
Source record
OAI-PMH GetRecord
citation

Papadimitriou, Aikaterini; Παπαδημητρίου, Αικατερίνη. Automatic sign language recognition algorithms and their incorporation into a greek sign language educational platform. University of Thessaly (UTH), 2023. http://hdl.handle.net/10442/hedi/56096