Back to search

Cal Poly

Enabling EMG-based Silent Speech Transcription Through Speech-To-Text Transfer Learning

Abstract

dc:description.abstract

In recent years, advances in deep learning have allowed various forms of electrographic signals, such as electroencephalography (EEG) and electromyography (EMG), to be used as a viable form of input in artificial intelligence applications, particularly for applications in the medical field. One such topic that EMG inputs have been used is in silent speech interfaces, or devices capable of processing speech without an audio-based input. The goal of this thesis is to explore a novel method of training a machine learning model to be used for silent speech interface development: using transfer learning to leverage a pre-trained speech recognition model for classifying EMG-based silent speech inputs. To accomplish this, we pass labeled EMG data through a custom transformation process, turning the data into musical notes that represent changes in an EMG sensor as silent speech data is captured. This transformed data was used as input into a pre-trained speech recognition model, and the model's classification layers were retrained to better fit the incoming data. The custom transformation process and model demonstrated progress towards effective classification with a small, closed-vocabulary dataset but showed no signs of effective training with a larger, open-vocabulary dataset. The effectiveness on the small closed-vocabulary dataset demonstrate that training a model to recognize EMG data using transfer learning on a pre-trained speech to text model is a viable approach.

Degree

thesis:*
Name thesis:degree_name
MS in Computer Science
Discipline thesis:degree_discipline
Computer Science

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Garcia, Alexander T
Contributors dc:contributor
  • Franz Kurfess
  • Computer Science
  • College of Engineering

Subjects

dc:subject × 7

Identifiers

dc:identifier.*
Repository record dc:identifier
https://digitalcommons.calpoly.edu/theses/2886

Chain of custody

source
Harvested from
Cal Poly
Base URL
digitalcommons.calpoly.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Garcia, Alexander T. Enabling EMG-based Silent Speech Transcription Through Speech-To-Text Transfer Learning. https://digitalcommons.calpoly.edu/theses/2886