Georgia Institute of Technology
Applying Language Understanding to Motion Prediction and Planning for Autonomous Systems
Abstract
dc:description.abstractThis thesis investigates how natural language understanding can be integrated with classical robotics algorithms to improve motion prediction and planning. While traditional methods rely on geometric information and recent approaches favor end-to-end learning, we propose a hybrid framework that uses language to extract structured representations that augment existing planning and prediction methods. For motion prediction, we consider aircraft in non-towered airspace, where pilots communicate intent through radio calls. We develop a language-conditioned goal prediction model that combines speech transcription, intent extraction using large language models, and multimodal trajectory prediction. Experiments on real-world aviation data show that incorporating language reduces goal prediction error compared to trajectory-only methods, particularly for longer prediction horizons. For motion planning, we present a hybrid system for long-horizon mobile manipulation that integrates classical navigation planning with a vision-language-action model for manipulation. An LLM-based module decomposes tasks given in natural language and coordinates execution across planning components. Evaluation on the BEHAVIOR-1K benchmark demonstrates improved task success over end-to-end baselines, especially in tasks requiring long-horizon reasoning. These results suggest that language understanding and reasoning can be effectively incorporated into robotics systems by serving as a source of structured semantic priors for spatial prediction and planning, enabling improved performance.
Degree
thesis:*- Name thesis:degree_name
- Aerospace Engineering, MS
- Department thesis:degree_department
- Aerospace Engineering
- Grantor
- Georgia Institute of Technology
- Year dc:date.issued
- 2026
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Vinodh Sangeetha, Sundhararajan
- Advisors dc:contributor.advisor
-
- Kousik, Shreyas
- Li, Sarah
- Committee members dc:contributor.committeemember
-
- Chou, Glen
- Chiu, Chih-Yuan
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/81599