Session: 06-17-01 AI Technology for Ocean Engineering

Submission Number: 156420

Optimization of Path Following for Autonomous Surface Vehicles Using Reinforcement and Imitation Learning 

Autonomous Surface Vehicles (ASVs) require precise and adaptable path-following capabilities to navigate safely through complex marine environments. However, constructing a model that captures all dynamic environmental conditions remains challenging, particularly without access to hydrodynamic coefficients necessary for an accurate nonlinear MMG model. This motivates the use of an alternative approach to train reinforcement learning (RL) policies effectively under simplified conditions. The objective of this research is to develop a robust RL-based control policy for ASV path following, starting with a linearized NOMOTO model to determine initial policy weights through Proximal Policy Optimization (PPO). Given the limitations of the NOMOTO model in capturing nonlinear dynamics, we employ Generative Adversarial Imitation Learning (GAIL) to enhance policy performance. In this framework, the NOMOTO-based policy acts as a generator, and expert data is provided by a PD controller operating on the MMG model of the KVLCC2 ship in a 3-DOF simulation under wave disturbances. Simulation results show that the GAIL-enhanced policy reduces both heading error (HE) and cross-track error (CTE) compared to baseline methods, demonstrating improved path-following performance in dynamic conditions. This approach highlights the potential of using imitation learning to bridge the gap between simplified dynamics models and complex real-world requirements. In conclusion, this study presents a promising pathway for achieving reliable ASV path following through RL and expert imitation. Future work will focus on refining reward functions, incorporating rudder dynamics, and validating the model on physical ASVs for real-world application.

Presenting Author: Anurag Akula Indian Institute of Technology Madras

Presenting Author Biography: Anurag Akula is an M.S. Research Scholar under the supervision of Dr. Suresh Rajendran at Ocean Engineering department, IIT Madras. He specializes in Marine Autonomous Vehicles and Deep Reinforcement Learning, focusing on advanced control systems. Anurag has got the Best Paper Award at World Ocean Science Congress (2024) for his contribution to health monitoring of coral reefs by combining Generative AI and Computer Vision techniques. His current research work explores constrained navigation of ships, particularly to avoid collisions with other marine vehicles.