{"id":{"repo_id":"rowan","oai_identifier":"oai:rdw.rowan.edu:etd-4420"},"canonical_url":"https://search.ndltd.org/etd/rowan/oai:rdw.rowan.edu:etd-4420","repository":{"repo_id":"rowan","name":"Rowan University","base_url":"https://rdw.rowan.edu/do/oai/"},"display":{"title":"When Words Fail: Why AI-Driven Context Beats Word Error Rate for Safety-Critical Air Traffic Control","abstract":"Accurate speech-to-text transcription is critical for Air Traffic Control (ATC) to en- sure safety and operational efficiency. However, existing Automatic Speech Recognition (ASR) systems struggle with domain-specific phraseology, high noise levels, and real-time constraints. This work explores domain adaptation strategies of OpenAI’s Whisper ASR model, applying Transfer Learning and Low-Rank Adaptation (LoRA) on ATC benchmark and a Federal Aviation Administration (FAA) corpus. Experimental results show that fine- tuning significantly reduces Word Error Rate (WER) from 55.2% to 6.8%, yet WER alone fails to capture safety-critical transcription errors. To address WER’s limitations, we introduce the Contextual Error Rate (CER), a novel evaluation metric powered by a Large Language Model (LLM) that prioritizes errors with operational impact rather than surface-level differences. Our experiments showed that while Transfer Learning improved domain-specific accuracy, LoRA preserved better gen- eralization to conversational English. This research demonstrates the importance of combining domain adapted ASR models with context-aware evaluation metrics. By integrating LLM-assisted CER scoring into the evaluation process, we offer a more reliable and context aware method for evaluating tran- scription quality of domain adapted ASR models in aviation, paving the way for scalable, automated speech recognition in air traffic management. The approach proposed is extensible to other context-critical communication systems where transcription fidelity is paramount. These contributions advance the intersection of speech recognition, machine learning, and aviation safety.","abstract_html":"Accurate speech-to-text transcription is critical for Air Traffic Control (ATC) to en- sure safety and operational efficiency. However, existing Automatic Speech Recognition (ASR) systems struggle with domain-specific phraseology, high noise levels, and real-time constraints. This work explores domain adaptation strategies of OpenAI’s Whisper ASR model, applying Transfer Learning and Low-Rank Adaptation (LoRA) on ATC benchmark and a Federal Aviation Administration (FAA) corpus. Experimental results show that fine- tuning significantly reduces Word Error Rate (WER) from 55.2% to 6.8%, yet WER alone fails to capture safety-critical transcription errors. To address WER’s limitations, we introduce the Contextual Error Rate (CER), a novel evaluation metric powered by a Large Language Model (LLM) that prioritizes errors with operational impact rather than surface-level differences. Our experiments showed that while Transfer Learning improved domain-specific accuracy, LoRA preserved better gen- eralization to conversational English. This research demonstrates the importance of combining domain adapted ASR models with context-aware evaluation metrics. By integrating LLM-assisted CER scoring into the evaluation process, we offer a more reliable and context aware method for evaluating tran- scription quality of domain adapted ASR models in aviation, paving the way for scalable, automated speech recognition in air traffic management. The approach proposed is extensible to other context-critical communication systems where transcription fidelity is paramount. These contributions advance the intersection of speech recognition, machine learning, and aviation safety.","abstract_has_math":false,"creators":["Maaz, Muhammad"],"institution":null,"degree_name":"M.S. Electrical and Computer Engineering","degree_level":"Thesis","degree_discipline":"Electrical and Computer Engineering","degree_department":null,"school":null,"contributors":["Nidhal Bouaynaya, Ph.D.","Ying Tang, Ph.D.","Tom Tessitore"],"advisors":[],"committee_chairs":[],"committee_members":[],"year":null,"date_issued":"","date_published":null,"updated_at":"2026-11-19T08:59:59Z","subjects":[],"languages":[],"rights":[],"rights_urls":[],"identifier_entries":[],"person_identifier_entries":[]},"links":{"outbound_url":"https://rdw.rowan.edu/etd/3414","outbound_label":"Repository record","outbound_source":"dc:identifier","record_url":"https://rdw.rowan.edu/etd/3414","record_label":"Repository record","record_kind":"record","record_source":"dc:identifier","download_url":null,"download_label":null,"download_source":null,"access_status":null,"embargo_until":null,"access_note":null},"source_record":{"url":"https://rdw.rowan.edu/do/oai/?verb=GetRecord&metadataPrefix=oai_etdms&identifier=oai%3Ardw.rowan.edu%3Aetd-4420","prefix":"oai_etdms"},"metadata_groups":[{"id":"people","label":"People","entries":[{"key":"dc:contributor","label":"Contributor","values":["Nidhal Bouaynaya, Ph.D.","Ying Tang, Ph.D.","Tom Tessitore"]},{"key":"dc:creator","label":"Author","values":["Maaz, Muhammad"]}]},{"id":"academic_context","label":"Academic Context","entries":[{"key":"dc:date.available","label":"Dc Date Available","values":["2025-06-30T07:00:00Z"]},{"key":"dc:date.created","label":"Dc Date Created","values":["2025-06-30T07:00:00Z"]},{"key":"thesis:degree_discipline","label":"Discipline","values":["Electrical and Computer Engineering"]},{"key":"thesis:degree_level","label":"Degree Level","values":["Thesis"]},{"key":"thesis:degree_name","label":"Degree Name","values":["M.S. Electrical and Computer Engineering"]}]},{"id":"identifiers","label":"Identifiers","entries":[{"key":"dc:identifier","label":"Identifier","values":["https://rdw.rowan.edu/etd/3414"]}]},{"id":"additional","label":"Additional Metadata","entries":[{"key":"dc:description.abstract","label":"Abstract","values":["Accurate speech-to-text transcription is critical for Air Traffic Control (ATC) to en- sure safety and operational efficiency. However, existing Automatic Speech Recognition (ASR) systems struggle with domain-specific phraseology, high noise levels, and real-time constraints. This work explores domain adaptation strategies of OpenAI’s Whisper ASR model, applying Transfer Learning and Low-Rank Adaptation (LoRA) on ATC benchmark and a Federal Aviation Administration (FAA) corpus. Experimental results show that fine- tuning significantly reduces Word Error Rate (WER) from 55.2% to 6.8%, yet WER alone fails to capture safety-critical transcription errors. To address WER’s limitations, we introduce the Contextual Error Rate (CER), a novel evaluation metric powered by a Large Language Model (LLM) that prioritizes errors with operational impact rather than surface-level differences. Our experiments showed that while Transfer Learning improved domain-specific accuracy, LoRA preserved better gen- eralization to conversational English. This research demonstrates the importance of combining domain adapted ASR models with context-aware evaluation metrics. By integrating LLM-assisted CER scoring into the evaluation process, we offer a more reliable and context aware method for evaluating tran- scription quality of domain adapted ASR models in aviation, paving the way for scalable, automated speech recognition in air traffic management. The approach proposed is extensible to other context-critical communication systems where transcription fidelity is paramount. These contributions advance the intersection of speech recognition, machine learning, and aviation safety."]},{"key":"dc:title","label":"Title","values":["When Words Fail: Why AI-Driven Context Beats Word Error Rate for Safety-Critical Air Traffic Control"]}]}],"canonical_facts":{"dc:contributor":["Nidhal Bouaynaya, Ph.D.","Ying Tang, Ph.D.","Tom Tessitore"],"dc:creator":["Maaz, Muhammad"],"dc:date.available":["2025-06-30T07:00:00Z"],"dc:date.created":["2025-06-30T07:00:00Z"],"dc:description.abstract":["Accurate speech-to-text transcription is critical for Air Traffic Control (ATC) to en- sure safety and operational efficiency. However, existing Automatic Speech Recognition (ASR) systems struggle with domain-specific phraseology, high noise levels, and real-time constraints. This work explores domain adaptation strategies of OpenAI’s Whisper ASR model, applying Transfer Learning and Low-Rank Adaptation (LoRA) on ATC benchmark and a Federal Aviation Administration (FAA) corpus. Experimental results show that fine- tuning significantly reduces Word Error Rate (WER) from 55.2% to 6.8%, yet WER alone fails to capture safety-critical transcription errors. To address WER’s limitations, we introduce the Contextual Error Rate (CER), a novel evaluation metric powered by a Large Language Model (LLM) that prioritizes errors with operational impact rather than surface-level differences. Our experiments showed that while Transfer Learning improved domain-specific accuracy, LoRA preserved better gen- eralization to conversational English. This research demonstrates the importance of combining domain adapted ASR models with context-aware evaluation metrics. By integrating LLM-assisted CER scoring into the evaluation process, we offer a more reliable and context aware method for evaluating tran- scription quality of domain adapted ASR models in aviation, paving the way for scalable, automated speech recognition in air traffic management. The approach proposed is extensible to other context-critical communication systems where transcription fidelity is paramount. These contributions advance the intersection of speech recognition, machine learning, and aviation safety."],"dc:identifier":["https://rdw.rowan.edu/etd/3414"],"dc:title":["When Words Fail: Why AI-Driven Context Beats Word Error Rate for Safety-Critical Air Traffic Control"],"thesis:degree_discipline":["Electrical and Computer Engineering"],"thesis:degree_level":["Thesis"],"thesis:degree_name":["M.S. Electrical and Computer Engineering"]},"updated_at":"2026-11-19T08:59:59Z"}