University of North Florida. School of Computing
Architectures for Real-Time Automatic Sign Language Recognition on Resource-Constrained Device
Abstract
dc:description.abstractPowerful, handheld computing devices have proliferated among consumers in recent years. Combined with new cameras and sensors capable of detecting objects in three-dimensional space, new gesture-based paradigms of human computer interaction are becoming available. One possible application of these developments is an automated sign language recognition system. This thesis reviews the existing body of work regarding computer recognition of sign language gestures as well as the design of systems for speech recognition, a similar problem. Little work has been done to apply the well-known architectural patterns of speech recognition systems to the domain of sign language recognition. This work creates a functional prototype of such a system, applying three architectures seen in speech recognition systems, using a Hidden Markov classifier with 75-90% accuracy. A thorough search of the literature indicates that no cloud-based system has yet been created for sign language recognition and this is the first implementation of its kind. Accordingly, there have been no empirical performance analyses regarding a cloud-based Automatic Sign Language Recognition (ASLR) system, which this research provides. The performance impact of each architecture, as well as the data interchange format, is then measured based on response time, CPU, memory, and network usage across an increasing vocabulary of sign language gestures. The results discussed herein suggest that a partially-offloaded client-server architecture, where feature extraction occurs on the client device and classification occurs in the cloud, is the ideal selection for all but the smallest vocabularies. Additionally, the results indicate that for the potentially large data sets transmitted for 3D gesture classification, a fast binary interchange protocol such as Protobuf has vastly superior performance to a text-based protocol such as JSON.
Degree
thesis:*- Name thesis:degree_name
- Master of Science in Computer and Information Sciences (MS)
- Discipline thesis:degree_discipline
- Computing
- Grantor
- University of North Florida. School of Computing
- Date dc:date.issued
- Fall
Author and committee
dc:creator, dc:contributor.*- Authors dc:creator
-
- Blair, James M
- Blair, James, 1988-
- Advisors dc:contributor.advisor
-
- Chuan, Ching-Hua
- Eggen, Roger
- Ahuja, Sanjay
- Contributors dc:contributor
-
- Dr. Ching-Hua Chuan
- Dr. Roger Eggen
- Dr. Sanjay Ahuja
Subjects
dc:subject × 7- Thesis; University of North Florida; UNF; Dissertations
- Academic -- UNF -- Master of Science in Computer and Information Sciences; Dissertations
- Academic -- UNF -- Computing; sign language; real time; recognition; architecture; mobile
- Computer and Systems Architecture
- Digital Communications and Networking
- Systems and Communications
- Sign language -- Software -- Testing; Pattern recognition systems -- Software -- Testing; Human activity recognition -- Software -- Testing; Automatic speech recognition -- Software -- Testing; Client/server computing -- Software -- Testing; Mobile computing -- Software -- Testing; Cloud computing -- Software -- Testing; Sign language -- Technological innovations -- Testing
Rights
dc:rightsIdentifiers
dc:identifier.*- Repository record dc:identifier
- https://digitalcommons.unf.edu/etd/851