Sim-to-Real RL for ASVs using SysID

2026-10-08

AI Summary

This study addresses the sim-to-real transfer challenges in reinforcement learning for autonomous surface vehicles (ASVs), which arise from the absence of parallel simulation environments and the difficulty of acquiring hydrodynamic parameters. To overcome these limitations, this work proposes a system identification-based simulation training pipeline. By leveraging CAD models alongside limited empirical trajectory data, the method approximates hydrodynamic and thruster parameters without requiring prior dynamic knowledge, thereby enabling the construction of high-fidelity simulation environments that support parallel training. Real-world experiments conducted on a BlueBoat platform demonstrate that the proposed approach successfully achieves zero-shot policy transfer for both path-following and station-keeping tasks. Ultimately, this research presents an efficient and cost-effective deployment framework for applying reinforcement learning to ASV control.

Abstract

Autonomous Surface Vehicles (ASVs) operating in dynamic marine environments require robust control policies for tasks such as path following and station keeping, making reinforcement learning (RL) a promising alternative to classical controllers. However, existing ASV simulators rarely support parallel environments for RL training. Such existing simulators require accurate hydrodynamic modeling from computational fluid dynamics solvers or towing tank tests for setting hydrodynamic parameters to address the sim-to-real gap. To address these challenges, we present an ASV simulator and accompanying pipeline that enables training policies starting from unknown vehicle dynamics. Our framework uses only a CAD model and brief set of open-water field trajectories for approximating and refining both hydrodynamic and thruster parameters. Real-world deployments on a BlueBoat ASV demonstrate successful zero-shot sim-to-real transfer in path following and station-keeping tasks without prior hydrodynamic and propeller information.

Problem

• Autonomous Surface Vehicles
• Sim-to-Real Transfer
• Reinforcement Learning
• System Identification
• Hydrodynamic Modeling

Innovation

• Sim-to-Real Transfer
• Reinforcement Learning
• System Identification
• Autonomous Surface Vehicles
• Zero-Shot