Session: 06-17-05 AI Technology for Ocean Engineering V

Paper Number: 126218

126218 - Autonomous Berthing Motion Planning for Under-Actuated Asv Based on Imagined Sub-Goals and Soft Actor-Critic 

For the autonomous berthing motion planning problem of under-actuated autonomous surface vehicle (ASV), this paper proposes an approach based on hierarchical reinforcement learning algorithm that incorporates imagined sub-goals and Soft Actor-Critic (HRL-ISSAC). Three layers of policies named the top-level policy, sub-goal arrival policy and auto-berthing policy are implemented in the algorithm. Throughout the training process, the top-level policy is designed to continuously generate appropriate sub-goals that are less difficult to reach than the ultimate berthing target status for ASV. Then the agent would train the sub-goal arrival policy by effectively addressing sub-goals. The Kullback-Leibler Divergence is utilized to reduce the distance between the distributions of sub-goal arrival policy and auto-berthing policy, thereby narrowing down the scope of action exploration of auto-berthing policy and accelerating its training speed. The hindsight experience replay mechanism is also employed in our algorithm to deal with sparse reward problem and optimize the sample efficiency. In the testing process the auto-berthing policy is solely applied, thus simplifying the structure of HRL. The simulation results show that HRL-ISSAC successfully achieved autonomous berthing guidance for the agent and the berthing trajectories generated by our algorithm take both the kinetics and dynamic constraints of ASV into consideration, which validate the effectiveness of our algorithm.

Presenting Author: Zhiyao Li Shanghai Jiao Tong University

Presenting Author Biography: Zhiyao Li received the B.S. degree in Naval Architecture and Ocean Engineering from Wuhan University of Technology, Wuhan, China in 2022. She is currently working toward the M.S. degree in Design and Manufacture of Ship and Ocean Structure with the School of Naval Architecture, Ocean & Civil Engineering, Shanghai Jiao Tong University, Shanghai, China. Her research interests include the application of deep reinforcement learning in autonomous ship navigation, obstacle avoidance and motion control.

Authors:

Zhiyao Li Shanghai Jiao Tong University
Yiting Wang Shanghai Jiao Tong University
Lei Wang Shanghai Jiao Tong University