Back to search

Cal Poly

Video Based Automatic Speech Recognition Using Neural Networks

Abstract

dc:description.abstract

Neural network approaches have become popular in the field of automatic speech recognition (ASR). Most ASR methods use audio data to classify words. Lip reading ASR techniques utilize only video data, which compensates for noisy environments where audio may be compromised. A comprehensive approach, including the vetting of datasets and development of a preprocessing chain, to video-based ASR is developed. This approach will be based on neural networks, namely 3D convolutional neural networks (3D-CNN) and Long short-term memory (LSTM). These types of neural networks are designed to take in temporal data such as videos. Various combinations of different neural network architecture and preprocessing techniques are explored. The best performing neural network architecture, a CNN with bidirectional LSTM, compares favorably against recent works on video-based ASR.

Degree

thesis:*
Name thesis:degree_name
MS in Electrical Engineering
Discipline thesis:degree_discipline
Electrical Engineering

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Lin, Alvin
Contributors dc:contributor
  • Jane Zhang
  • Electrical Engineering
  • College of Engineering

Subjects

dc:subject × 6

Identifiers

dc:identifier.*

Chain of custody

source
Harvested from
Cal Poly
Base URL
digitalcommons.calpoly.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
citation

Lin, Alvin. Video Based Automatic Speech Recognition Using Neural Networks. https://digitalcommons.calpoly.edu/theses/2343