Rowan University
When Words Fail: Why AI-Driven Context Beats Word Error Rate for Safety-Critical Air Traffic Control
Abstract
dc:description.abstractAccurate speech-to-text transcription is critical for Air Traffic Control (ATC) to en- sure safety and operational efficiency. However, existing Automatic Speech Recognition (ASR) systems struggle with domain-specific phraseology, high noise levels, and real-time constraints. This work explores domain adaptation strategies of OpenAI’s Whisper ASR model, applying Transfer Learning and Low-Rank Adaptation (LoRA) on ATC benchmark and a Federal Aviation Administration (FAA) corpus. Experimental results show that fine- tuning significantly reduces Word Error Rate (WER) from 55.2% to 6.8%, yet WER alone fails to capture safety-critical transcription errors. To address WER’s limitations, we introduce the Contextual Error Rate (CER), a novel evaluation metric powered by a Large Language Model (LLM) that prioritizes errors with operational impact rather than surface-level differences. Our experiments showed that while Transfer Learning improved domain-specific accuracy, LoRA preserved better gen- eralization to conversational English. This research demonstrates the importance of combining domain adapted ASR models with context-aware evaluation metrics. By integrating LLM-assisted CER scoring into the evaluation process, we offer a more reliable and context aware method for evaluating tran- scription quality of domain adapted ASR models in aviation, paving the way for scalable, automated speech recognition in air traffic management. The approach proposed is extensible to other context-critical communication systems where transcription fidelity is paramount. These contributions advance the intersection of speech recognition, machine learning, and aviation safety.
Degree
thesis:*- Name thesis:degree_name
- M.S. Electrical and Computer Engineering
- Level thesis:degree_level
- Thesis
- Discipline thesis:degree_discipline
- Electrical and Computer Engineering
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Maaz, Muhammad
- Contributors dc:contributor
-
- Nidhal Bouaynaya, Ph.D.
- Ying Tang, Ph.D.
- Tom Tessitore
Identifiers
dc:identifier.*- Repository record dc:identifier
- https://rdw.rowan.edu/etd/3414