Abstract
dc:description.abstractThis thesis investigates how finetuning multimodal large language models (MLLMs) with large-scale embodied experience unlocks their ability to act as generalist embodied agents. While MLLMs provide reasoning abilities and broad world knowledge, finetuning aligns these capabilities with the demands of embodied agents, unlocking their potential to act autonomously across diverse tasks and domains. The first part of the thesis introduces Habitat 2.0, a platform for generating large-scale embodied experience in simulated 3D, interactive environments. The next part presents a method to adapt MLLMs as generalizable embodied policies with a study on how best to ground these MLLM policies in embodied action spaces. Last, this thesis proposes the Generalist Embodied Agent (GEA), a single model that generalizes to unseen tasks across diverse domains and highlights the importance of online RL for developing a capable generalist agent. Overall, this research aims to utilize the strengths of MLLMs to build generalist embodied agents with strong generalization capabilities.
Degree
thesis:*- Level thesis:degree_level
- Doctoral
- Department dc:contributor.department
- Interactive Computing
- Grantor dc:publisher
- Georgia Institute of Technology
- Year dc:date.issued
- 2025
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Szot, Andrew
- Advisors dc:contributor.advisor
-
- Kira, Zsolt
- Batra, Dhruv
- Committee members dc:contributor.committeemember
-
- Ha, Sehoon
- Heck, Larry
- Toshev, Alexander
Subjects
dc:subject × 3Rights
- Language dc:language.iso
- en_US
Identifiers
dc:identifier.*- Handle dc:identifier.uri
- https://hdl.handle.net/1853/78686