Target tracking control for Unmanned Surface Vehicles: An end-to-end deep reinforcement learning approach

Target tracking serves as a fundamental motion control function for Unmanned Surface Vehicles (USVs), requiring the USV to rapidly approach a moving target without prior knowledge of its behavior. However, system time lag, underactuation, and environmental disturbances often lead to response delays, degrading tracking efficiency. To address this, we propose a deep reinforcement learning-based end-to-end control method aimed at enhancing the USV’s tracking efficiency and responsiveness. Unlike conventional approaches, this method directly learns the mapping from sensor observations to control commands, optimizing decision-making and control actions within a unified framework. A specific deep reinforcement learning algorithm for target tracking is designed based on soft actor–critic framework and integrating predictive target information into the observation space to learn an anticipatory control policy. This paradigm enables the USV to comprehensively account for target movement uncertainty and its own maneuverability under environmental disturbances. Comparative studies are conducted using a high-fidelity simulator that considers the USV’s nonlinear dynamics and external influences. The results demonstrate that the proposed method outperforms conventional pure pursuit-based strategy, exhibiting a more efficient and adaptive tracking behavior, akin to human driving habits.

Recommended citation: Wang, Z., Hu, Q., Wang, C., Liu, Y., & Xie, W. (2025). Target tracking control for Unmanned Surface Vehicles: An end-to-end deep reinforcement learning approach. Ocean Engineering, 317, 120059.
Download Paper