Session: 08-06-01 non-presentations

Paper Number: 121726

121726 - A Vision-Based Maritime Monitoring and Situational Awareness Framework Using Deep Learning Techniques 

Maritime awareness holds immense significance towards national security by monitoring borders and countering threats like piracy while safeguarding marine environments from pollution and ecological damage. Maritime awareness promotes global safety, prosperity, and environmental stewardship by facilitating efficient trade, aiding search and rescue, supporting scientific research, and fostering international cooperation to address shared maritime challenges.

This research focuses on enhancing maritime safety through comprehensive environmental awareness leveraging advanced Artificial Intelligence and Computer Vision techniques. To this end, ship-mounted cameras are intended to capture images of neighboring vessels and other objects in the maritime scenario. Further, the proposed maritime surveillance involves three key processes: ship detection, precise localization, and reliable tracking.

For the former step of ship detection, advanced YOLO (You Only Look Once) v8 deep learning models are employed on the acquired camera images. Since the publicly available datasets for maritime monitoring are limited, a new virtual image dataset, namely VirtualMaritime dataset is curated in-house, which is one of its first kind. The new synthetic dataset generated in the simulator graphics engine platform (Unity) encapsulates a wide array of maritime scenarios, such as diverse ship vessels of various size/pose, etc. Further, comprehensive detection analysis on both real-world and synthetic datasets is carried out.  For the real-world dataset, a collection of ship images sourced from the internet are meticulously curated. Its diversity and robustness are improved by applying several data augmentation methods, such as grayscaling and adding noise. Subsequently, YOLOv8 models with varying complexities, namely nano, medium, and large, are trained using this real-world dataset and synthetic (VirtualMaritime) dataset. By combining synthesized data with training, this approach enabled us to harness the strengths of both datasets, resulting in a more robust and effective object detection framework for ship-related imagery.

Following the detection phase, our study advances to the localization phase, a pivotal stage in pinpointing the exact location of the target ship within the given image. To achieve this goal, a novel custom algorithm using the OpenCV framework is proposed that combines advanced image processing methods to precisely pinpoint the location of the target ship and further improves its accuracy by utilizing distinctive ship characteristics like shape and unique distinctive features. By incorporating this algorithm, we achieve a comprehensive understanding of the target ship's spatial attributes, including its precise geographical coordinates, course over ground (COG), and speed over ground (SOG). This robust localization framework not only enhances accuracy but also contributes to a more comprehensive analysis of ship behavior and movement patterns in maritime scenarios.

 In the third tracking phase, our study leverages an extended version to dynamically track evolving ship attributes over time by building upon the conventional Kalman filtering concept. Our approach uniquely incorporates a deep learning model to forecast the ship's direction within the image, which supplements the initial estimation made by the Kalman filter with precise heading data. By blending Kalman filtering and deep learning, we create an adaptable tracking system suited for dynamic maritime situations, resulting in improved accuracy for tracking ship attributes, including position updates, course corrections, and speed variations, ultimately leading to a more dependable and precise comprehension of ship behavior and changes over time.

 

Presenting Author: Athira Muraleedharan Nambiar SRM Institute of Science and Technology

Presenting Author Biography: Dr. Athira Nambiar is a faculty in the Dept. of Computation Intelligence at SRM Institute of Science and Technology. She works in the field of Deep Learning and computer vision, Biometrics, Image pro- cessing, Vision for surveillance etc.

Authors:

Suresh Rajendran Indian Institute of Technology Madras
Athira Muraleedharan Nambiar SRM Institute of Science and Technology
Nilay Kumar Bhatnaga SRM Institute of Science and Technology
Aryan Raj SRM Institute of Science and Technology
Arnav Sharma SRM Institute of Science and Technology