Back to search

University of Ioannina

Advanced reinforcement learning methods for autonomous navigation of unmanned surface vessels

Abstract

dc:description

The increasing development of autonomous maritime systems has created a need for intelligent navigation methods capable of operating reliably in complex, dynamic, and uncertain environments. Unmanned Surface Vehicles (USVs) are particularly challenging autonomous platforms because their navigation is affected by environmental disturbances, moving vessels, limited perception, and the requirement to maintain safe separation from surrounding obstacles. While deep reinforcement learning (DRL) provides a promising framework for learning navigation policies through interaction with an environment, its application to autonomous maritime navigation remains constrained by issues related to safety, the availability of expert knowledge, generalization, and coordination among multiple autonomous vessels. This dissertation investigates advanced reinforcement learning methods for autonomous USV navigation, with particular emphasis on developing learning-based navigation policies that are effective, safe, and applicable to increasingly complex maritime scenarios.To establish a practical foundation for the research, a small-scale autonomous USV platform is designed and constructed to support the development, validation, and eventual transfer of learning-based navigation methods to physical hardware. In parallel, a comprehensive simulation framework is developed to enable efficient training and systematic evaluation without exposing the physical platform to the risks associated with trial-and-error learning. Within this framework, DRL-based navigation policies are investigated for autonomous USV navigation in environments containing environmental disturbances and moving maritime traffic. The resulting navigation framework demonstrates the ability of learning-based agents to integrate goal-directed navigation and collision avoidance within a continuous control setting. Building upon this foundation, imitation learning is investigated to address the difficulty of acquiring effective navigation policies solely through trial-and-error interaction. In particular, Generative Adversarial Imitation Learning (GAIL) is employed to derive navigation policies from expert demonstrations, enabling the agent to acquire desirable navigation behavior without explicitly reproducing a manually designed reward function. This provides an alternative mechanism for incorporating prior navigation knowledge into the learning process and demonstrates the applicability of adversarial imitation learning to autonomous USV navigation.The dissertation subsequently addresses safety as an explicit requirement of the learning process. Autonomous navigation is formulated as a Constrained Markov Decision Process (CMDP), in which task performance and safety are represented through separate reward and cost objectives. A Lagrangian relaxation framework is developed to incorporate constraints associated with static and moving obstacles into policy optimization. The approach enables the navigation policy to balance task performance with collision avoidance while explicitly accounting for different classes of maritime hazards. An extension of this framework further investigates trainable Lagrangian parameters and Actor-Critic-based optimization to provide adaptive safety enforcement during policy learning and the underlying interpretability of the safety constraints.Finally, the dissertation extends autonomous navigation from a single-USV setting to multi-USV environments. A Cross-Agent Multi-agent Soft Actor-Critic (CROSS-MASAC) framework is proposed in which decentralized actors are trained through cross-agent policy updates while leveraging shared learning information. The approach aims to improve coordination and policy consistency among independently acting USVs without requiring centralized execution. This establishes a transition from single-agent navigation toward scalable multi-agent maritime autonomy, where multiple vessels must simultaneously navigate and respond to one another within a shared environment.

Degree

thesis:*
Grantor dc:publisher
University of Ioannina
Year dc:date
2026

Author and committee

dc:creator, dc:contributor.*
Authors dc:creator
  • Chaysri, Piyabhum
  • Σάισι, Πιγάπουμ

Subjects

dc:subject × 35

Rights

Language dc:language
eng

Identifiers

dc:identifier.*
Identifier
10.12681/eadd/63028

Chain of custody

source
Harvested from
Greek National Archive of PhD Theses
Base URL
phdtheses.ekt.gr/eadd_oai/request
Last updated
2026-11-20
Source record
OAI-PMH GetRecord
citation

Chaysri, Piyabhum; Σάισι, Πιγάπουμ. Advanced reinforcement learning methods for autonomous navigation of unmanned surface vessels. University of Ioannina, 2026. http://hdl.handle.net/10442/hedi/63028