{"id":{"repo_id":"calpoly","oai_identifier":"oai:digitalcommons.calpoly.edu:theses-4562"},"canonical_url":"https://search.ndltd.org/etd/calpoly/oai:digitalcommons.calpoly.edu:theses-4562","repository":{"repo_id":"calpoly","name":"Cal Poly","base_url":"https://digitalcommons.calpoly.edu/do/oai/"},"display":{"title":"Enabling EMG-based Silent Speech Transcription Through Speech-To-Text Transfer Learning","abstract":"In recent years, advances in deep learning have allowed various forms of electrographic signals, such as electroencephalography (EEG) and electromyography (EMG), to be used as a viable form of input in artificial intelligence applications, particularly for applications in the medical field. One such topic that EMG inputs have been used is in silent speech interfaces, or devices capable of processing speech without an audio-based input. The goal of this thesis is to explore a novel method of training a machine learning model to be used for silent speech interface development: using transfer learning to leverage a pre-trained speech recognition model for classifying EMG-based silent speech inputs. To accomplish this, we pass labeled EMG data through a custom transformation process, turning the data into musical notes that represent changes in an EMG sensor as silent speech data is captured. This transformed data was used as input into a pre-trained speech recognition model, and the model's classification layers were retrained to better fit the incoming data. The custom transformation process and model demonstrated progress towards effective classification with a small, closed-vocabulary dataset but showed no signs of effective training with a larger, open-vocabulary dataset. The effectiveness on the small closed-vocabulary dataset demonstrate that training a model to recognize EMG data using transfer learning on a pre-trained speech to text model is a viable approach.","abstract_html":"In recent years, advances in deep learning have allowed various forms of electrographic signals, such as electroencephalography (EEG) and electromyography (EMG), to be used as a viable form of input in artificial intelligence applications, particularly for applications in the medical field. One such topic that EMG inputs have been used is in silent speech interfaces, or devices capable of processing speech without an audio-based input. The goal of this thesis is to explore a novel method of training a machine learning model to be used for silent speech interface development: using transfer learning to leverage a pre-trained speech recognition model for classifying EMG-based silent speech inputs. To accomplish this, we pass labeled EMG data through a custom transformation process, turning the data into musical notes that represent changes in an EMG sensor as silent speech data is captured. This transformed data was used as input into a pre-trained speech recognition model, and the model&#x27;s classification layers were retrained to better fit the incoming data. The custom transformation process and model demonstrated progress towards effective classification with a small, closed-vocabulary dataset but showed no signs of effective training with a larger, open-vocabulary dataset. The effectiveness on the small closed-vocabulary dataset demonstrate that training a model to recognize EMG data using transfer learning on a pre-trained speech to text model is a viable approach.","abstract_has_math":false,"creators":["Garcia, Alexander T"],"institution":null,"degree_name":"MS in Computer Science","degree_level":null,"degree_discipline":"Computer Science","degree_department":null,"school":null,"contributors":["Franz Kurfess","Computer Science","College of Engineering"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T07:12:23Z","subjects":["Silent Speech","EMG","Neural Network","Transfer Learning","Data Transformation","Sonification","Artificial Intelligence and Robotics"],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://digitalcommons.calpoly.edu/theses/2886","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://digitalcommons.calpoly.edu/theses/2886","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://digitalcommons.calpoly.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Adigitalcommons.calpoly.edu%3Atheses-4562","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Franz Kurfess","Computer Science","College of Engineering"]},{"key":"dc:creator","label":"Author","values":["Garcia, Alexander T"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2024-08-20T07:00:00Z"]},{"key":"dc:date.created","label":"Dc Date Created","values":["2024-09-01T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Computer Science"]},{"key":"thesis:degree_name","label":"Degree Name","values":["MS in Computer Science"]}]},{"id":"subjects_keywords","label":"Subjects and Keywords","entries":[{"key":"dc:subject","label":"Dc Subject","values":["Silent Speech","EMG","Neural Network","Transfer Learning","Data Transformation","Sonification","Artificial Intelligence and Robotics"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://digitalcommons.calpoly.edu/theses/2886"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["In recent years, advances in deep learning have allowed various forms of electrographic signals, such as electroencephalography (EEG) and electromyography (EMG), to be used as a viable form of input in artificial intelligence applications, particularly for applications in the medical field. One such topic that EMG inputs have been used is in silent speech interfaces, or devices capable of processing speech without an audio-based input. The goal of this thesis is to explore a novel method of training a machine learning model to be used for silent speech interface development: using transfer learning to leverage a pre-trained speech recognition model for classifying EMG-based silent speech inputs. To accomplish this, we pass labeled EMG data through a custom transformation process, turning the data into musical notes that represent changes in an EMG sensor as silent speech data is captured. This transformed data was used as input into a pre-trained speech recognition model, and the model's classification layers were retrained to better fit the incoming data. The custom transformation process and model demonstrated progress towards effective classification with a small, closed-vocabulary dataset but showed no signs of effective training with a larger, open-vocabulary dataset. The effectiveness on the small closed-vocabulary dataset demonstrate that training a model to recognize EMG data using transfer learning on a pre-trained speech to text model is a viable approach."]},{"key":"dc:title","label":"Title","values":["Enabling EMG-based Silent Speech Transcription Through Speech-To-Text Transfer Learning"]}]}],"canonical_facts":{"dc:contributor":["Franz Kurfess","Computer Science","College of Engineering"],"dc:creator":["Garcia, Alexander T"],"dc:date.available":["2024-08-20T07:00:00Z"],"dc:date.created":["2024-09-01T07:00:00Z"],"dc:description.abstract":["In recent years, advances in deep learning have allowed various forms of electrographic signals, such as electroencephalography (EEG) and electromyography (EMG), to be used as a viable form of input in artificial intelligence applications, particularly for applications in the medical field. One such topic that EMG inputs have been used is in silent speech interfaces, or devices capable of processing speech without an audio-based input. The goal of this thesis is to explore a novel method of training a machine learning model to be used for silent speech interface development: using transfer learning to leverage a pre-trained speech recognition model for classifying EMG-based silent speech inputs. To accomplish this, we pass labeled EMG data through a custom transformation process, turning the data into musical notes that represent changes in an EMG sensor as silent speech data is captured. This transformed data was used as input into a pre-trained speech recognition model, and the model's classification layers were retrained to better fit the incoming data. The custom transformation process and model demonstrated progress towards effective classification with a small, closed-vocabulary dataset but showed no signs of effective training with a larger, open-vocabulary dataset. The effectiveness on the small closed-vocabulary dataset demonstrate that training a model to recognize EMG data using transfer learning on a pre-trained speech to text model is a viable approach."],"dc:identifier":["https://digitalcommons.calpoly.edu/theses/2886"],"dc:subject":["Silent Speech","EMG","Neural Network","Transfer Learning","Data Transformation","Sonification","Artificial Intelligence and Robotics"],"dc:title":["Enabling EMG-based Silent Speech Transcription Through Speech-To-Text Transfer Learning"],"thesis:degree_discipline":["Computer Science"],"thesis:degree_name":["MS in Computer Science"]},"updated_at":"2026-11-19T07:12:23Z"}