University of Minnesota
Simulation Driven Multimodal Imitation Learning Based Navigation for Unstructured Corn Crop Field Environment
Abstract
dc:description.abstractAutonomous navigation in dense agricultural environments remains a challenging problemdue to heavy vegetation occlusions, perceptual ambiguity, irregular obstacle layouts, and significant environmental variability. Traditional navigation systems often rely on handcrafted planning and perception pipelines that struggle to maintain robust performance in highly unstructured crop-field environments. At the same time, collecting large-scale real-world navigation demonstrations for learning-based approaches is costly, time-consuming, and difficult to scale across diverse agricultural conditions. This thesis presents a simulation-driven multimodal imitation learning framework for autonomous navigation in unstructured corn crop field environments. The proposed framework combines RGB imagery, LiDAR measurements, and goal-conditioned representations within a unified policy architecture that directly predicts navigation commands from multimodal observations. To improve robustness and reduce the effects of covariate shift commonly associated with behavior cloning, an intervention-based data aggregation strategy inspired by Dataset Aggregation (DAgger) is incorporated to iteratively refine the policy using corrective demonstrations collected from failure states. A high-fidelity agricultural simulation environment is developed in NVIDIA Isaac Sim to generate expert demonstrations under varying crop-row geometries, vegetation structures, lighting conditions, terrain appearances, and obstacle configurations. Expert demonstrations are collected through teleoperation and processed into synchronized multimodal state-action pairs suitable for supervised policy learning. The resulting dataset consists of 83 expert trajectories and 61,611 synchronized training samples containing RGB observations, LiDAR measurements, goal information, and corresponding expert control commands. The proposed framework is evaluated in previously unseen simulated agricultural environments and compared against the ROS2 Navigation Stack, an Image + Goal imitation learning policy, and a LiDAR + Goal imitation learning policy. The evaluation includes structured environments, obstacle-rich environments, and curved-lane environments designed to assess navigation robustness, multimodal sensor fusion, and generalization capability. Experimental results show that the proposed multimodal policy achieves reliable navigation performance across diverse scenarios and consistently outperforms both classical navigation and single-modality learning baselines in challenging unstructured environments. Additional ablation experiments further quantify the contribution of individual sensing modalities and demonstrate the importance of multimodal perception for robust autonomous navigation. The presented framework provides a scalable simulation-driven methodology for developing and evaluating autonomous navigation policies in agricultural environments and establishes a foundation for future real-world deployment and sim-to-real transfer studies.
Author and committee
dc:creator, dc:contributor.*- Author dc:creator
-
- Dixit, Manas
Subjects
dc:subject × 4Rights
- Language dc:language.iso
- en
Identifiers
dc:identifier.*- Dc Identifier Citation
- Dixit, Manas. (2026). Simulation Driven Multimodal Imitation Learning Based Navigation for Unstructured Corn Crop Field Environment. Retrieved from the University Digital Conservancy, https://hdl.handle.net/11299/282245.