Session: 06-17-01 AI Technology for Ocean Engineering

Submission Number: 156146

Auto-Docking for Leisure Boats Using Deep Reinforcement Learning With Action Masking 

The demand for auto-docking technology is increasing to alleviate the burden on inexperienced operators. Previous studies using guidance law-based approaches have mostly focused on ensuring the robustness of auto-docking in designated scenarios and marina topographies. Meanwhile, reinforcement learning has become an actively researched area within control systems, and recent studies have explored its applications in the maritime sector, including collision avoidance and path planning for autonomous ships. This study aims to examine how docking performance can be improved using deep reinforcement learning in various scenarios, such as different initial docking positions and heading angles. A training and test environment was developed specifically for applying deep reinforcement learning to the auto-docking problem of leisure boats. By comparing the learning performance of various deep reinforcement learning algorithms, including Proximal Policy Optimization (PPO), Soft Actor-Critic (SAC), and Deep Q-Network (DQN), it was observed that PPO was the most suitable algorithm for solving the present problem. Additionally, action masking was introduced to PPO to prevent invalid actions that could cause the leisure boat to move away from the target position or collide with obstacles. Comparative trials demonstrated that PPO with action masking offered higher adaptability to changing initial docking positions or heading angles compared to polynomial function-based guidance laws. The findings suggest that the incorporation of deep reinforcement learning with action masking in auto-docking systems could improve operational safety and efficiency, potentially leading to more reliable autonomous docking solutions in varying maritime conditions.

Presenting Author: Ji Young Seo Seoul National University

Presenting Author Biography: Ms. Seo is currently a PhD student in Naval Architecture and Ocean Engineering at Seoul National University, Korea, since 2022. She earned bachelor degree in the same field from Seoul National University in 2022. Her research focuses on ship manoeuvring and control problems.