Session: 08-04-02 Machine Learning and Optimization for Marine Design and Operation

Paper Number: 125392

125392 - Artificial Intelligence for Cooperative Collision Avoidance of Ships Developed by Multi Agent Deep Reinforcement Learning 

Collision accidents of ships have been reported even now. Most collision accidents are caused by human error, making them unpreventable as long as humans operate a ship. Therefore, autonomous navigation technology is expected to contribute for preventing collision accidents by human errors. In recent years, artificial intelligence with deep reinforcement learning for autonomous collision avoidance has been studied. These studies demonstrated the capability of AIs for the autonomous navigation task. However, almost the autonomous collision avoidance technology, not limited to the AI, have been developed and validated in ideal environment, such that other ships keeping their courses and speeds. In actual navigation in a congested water, safe collision avoidance is achieved by cooperative maneuvers for collision avoidance between ships considering COLREGs. To realize cooperative navigation as humans do, AIs should consider how other ships will act when taking an action. In this regard, a multi agent system would have the advantage for training AI for collision avoidance.

This study aims at developing artificial intelligence for autonomous collision avoidance of ships with multi agent deep reinforcement learning. The AI was developed with multi agent deep deterministic policy gradient that is an algorithm in which each agent concurrently learns a Q-function and an action policy considering other agent’s policies. The developed AI was numerically evaluated with several encounter scenarios consisted of multiple ships. The encounter scenarios are determined according to Imazu problems that are scenarios set of important encounters proposed by analyzing AIS data. It is validated that the AIs realize safe collision avoidance and heading to a given waypoint with actions complying COLREGs. The results indicated that the multi agent deep reinforcement learning could improve the capability of collision avoidance AIs. Furthermore, the contribution of multi agent system was discussed by comparing with AI developed using an ordinary single agent system. According to the results of comparison in same scenarios, the AIs developed by multi agent deep reinforcement learning are realize safer and more efficient autonomous navigation.

Presenting Author: Hitoshi Yoshioka Osaka Metropolitan University

Presenting Author Biography: I am a PhD student at Osaka Metropolitan University.

Authors:

Hitoshi Yoshioka Osaka Metropolitan University
Hirotada Hashimoto Osaka Metropolitan University
Akihiko Matsuda Japan Fisheries Reserch and Education Agency