EndoWAM: A Grounded World-Action Model for Generalizable Endoscopic Navigation

2026-08-02

AI Summary

This work addresses the challenge of insufficient control robustness in autonomous endoscopic navigation caused by tissue deformation, transient occlusions, and abrupt viewpoint changes. To this end, we propose EndoWAM—the first world-action model (WAM) tailored for endoscopic navigation. EndoWAM integrates task-relevant target region prediction into intermediate features of a video world model via a “future grounding” mechanism, jointly coupling future visual prediction and action generation within a single denoising step. Leveraging a lightweight diffusion Transformer and a shared-representation-based discrete action expert module, EndoWAM achieves state-of-the-art performance on our newly curated EndoMotion dataset, which encompasses diverse anatomical procedures. The method demonstrates markedly enhanced robustness to visual degradation and viewpoint shifts, while exhibiting strong zero-shot generalization capabilities, thereby enabling accurate, stable, and long-horizon autonomous endoscopic navigation.

Abstract

Autonomous endoscopic navigation can reduce clinicians' operational burden, yet robust control remains challenging due to tissue deformation, transient occlusions, and rapidly changing viewpoints. Existing learning-based policies typically predict actions from current observations without explicitly modeling future dynamics, limiting their robustness and reliability in safety-critical settings. World Action Models (WAMs) offer a promising alternative by coupling predictive visual dynamics with action generation, but extending them to robotic endoscopy remains challenging due to limited training data, restricted viewpoint diversity, deformable anatomy, and high inference latency. We present EndoWAM, which is, to our knowledge, the first WAM for generalizable robotic endoscopic navigation. EndoWAM introduces future grounding, which predicts task-relevant target regions in future observations from intermediate denoising features of a video world model. Specifically, EndoWAM couples a lightweight diffusion transformer for future target-region prediction with a discrete action expert through a shared predictive representation. This design injects target-aware supervision into predictive dynamics modeling, improving robustness to visual degradation and viewpoint changes while enabling real-time control in a single denoising pass. We further introduce EndoMotion, a robotic endoscopic motion dataset spanning three anatomically distinct procedures: ureteroscopy, esophagoscopy, and endoscopic retrograde cholangiopancreatography (ERCP). EndoWAM consistently outperforms all baselines and alternative grounding strategies, while demonstrating strong zero-shot generalization to unseen viewpoints, environments, and targets. These results establish EndoWAM as a predictive, target-grounded framework for accurate, generalizable, and long-horizon navigation in visually constrained endoscopic environments.

Problem

• endoscopic navigation
• world action models
• visual dynamics
• generalization
• autonomous control

Innovation

• World-Action Model
• future grounding
• diffusion transformer
• endoscopic navigation
• zero-shot generalization