Back to search

Rowan University

When Words Fail: Why AI-Driven Context Beats Word Error Rate for Safety-Critical Air Traffic Control

Abstract

dc:description.abstract

Accurate speech-to-text transcription is critical for Air Traffic Control (ATC) to en- sure safety and operational efficiency. However, existing Automatic Speech Recognition (ASR) systems struggle with domain-specific phraseology, high noise levels, and real-time constraints. This work explores domain adaptation strategies of OpenAI’s Whisper ASR model, applying Transfer Learning and Low-Rank Adaptation (LoRA) on ATC benchmark and a Federal Aviation Administration (FAA) corpus. Experimental results show that fine- tuning significantly reduces Word Error Rate (WER) from 55.2% to 6.8%, yet WER alone fails to capture safety-critical transcription errors. To address WER’s limitations, we introduce the Contextual Error Rate (CER), a novel evaluation metric powered by a Large Language Model (LLM) that prioritizes errors with operational impact rather than surface-level differences. Our experiments showed that while Transfer Learning improved domain-specific accuracy, LoRA preserved better gen- eralization to conversational English. This research demonstrates the importance of combining domain adapted ASR models with context-aware evaluation metrics. By integrating LLM-assisted CER scoring into the evaluation process, we offer a more reliable and context aware method for evaluating tran- scription quality of domain adapted ASR models in aviation, paving the way for scalable, automated speech recognition in air traffic management. The approach proposed is extensible to other context-critical communication systems where transcription fidelity is paramount. These contributions advance the intersection of speech recognition, machine learning, and aviation safety.

Degree

thesis:*
Name thesis:degree_name
M.S. Electrical and Computer Engineering
Level thesis:degree_level
Thesis
Discipline thesis:degree_discipline
Electrical and Computer Engineering

Author and committee

dc:creator, dc:contributor.*
Author dc:creator
  • Maaz, Muhammad
Contributors dc:contributor
  • Nidhal Bouaynaya, Ph.D.
  • Ying Tang, Ph.D.
  • Tom Tessitore

Identifiers

dc:identifier.*
Repository record dc:identifier
https://rdw.rowan.edu/etd/3414

Chain of custody

source
Harvested from
Rowan University
Base URL
rdw.rowan.edu/do/oai/
Last updated
2026-11-19
Source record
OAI-PMH GetRecord
related terms
citation

Maaz, Muhammad. When Words Fail: Why AI-Driven Context Beats Word Error Rate for Safety-Critical Air Traffic Control. Thesis thesis, https://rdw.rowan.edu/etd/3414