Session: 06-11-02 Ocean Engineering Technology - II

Submission Number: 180638

A Reinforcement Learning-Based Navigation Framework for Autonomous Surface Vessels 

In the last decades, autonomous surface vessel applications have been evolving at an unprecedented rate. When in the ocean, these platforms may face harsh environmental conditions, which can be a significant challenge for the control system. Under these scenarios, the system must be able to handle nonlinear hydrodynamics, environmental disturbances, and complex maneuvering tasks. Thus, with the recent developments in ReinforcementLearning (RL), we are proposing an RL-based control framework for ships equipped with vectorized propulsion systems. The presented approach utilizes a Lyapunov-based RL to ensure closed-loop stability, which is critical to real-world applications. The RL controller  may learn optimal control policies within a simulated environment, which may incorporate different sources of disturbances such as wind, waves, and current. Then, the nonlinearities that are inherent in this kind of system can be represented and used in the training phase. The propulsion system is then controlled using variables that represent a priori known parameters, as the thrust magnitude. This approach offers a good maneuvering capability for precise trajectory tracking and station keeping. Results from simulations indicate that the RL-based control was able to converge under the disturbance scenarios, showing improved stability and good control in different circumstances. Thus, the proposed approach provides a promising solution for robust and adaptive vessel navigation in complex marine environments.

Presenting Author: Emerson Andrade UFRJ

Presenting Author Biography: Received a B.Sc. in Naval Architecture, Marine and Ocean Engineering from the Polytechnic School of the Federal University of Rio de Janeiro, Rio de Janeiro, RJ, Brazil, in 2020, and the M.Sc. degree from The Alberto Luiz Coimbra Institute for Graduate Studies and Research in Engineering (COPPE-UFRJ), Rio de Janeiro, RJ, Brazil, in 2022, where he is currently working toward the D.Sc. degree with the Department of Ocean Engineering. He is a Researcher at the Laboratory of Waves and Current (LOC), from the Department of Ocean Engineering, COPPE-UFRJ. His research interests include hydrodynamics, numerical methods, experimental tests, and autonomous vehicles.

Authors:

Emerson Andrade UFRJ
Antonio Fernandes UFRJ
Joel Sales Junior UFRJ