EndoControlMag: Robust Endoscopic Vascular Motion Magnification with Periodic Reference Resetting and Hierarchical Tissue-aware Dual-Mask Contro

2025-07-21

AI Summary

In endoscopic surgery, visualizing subtle vascular motion is critical for intraoperative precision and decision-making, yet existing methods lack robustness due to scene complexity and dynamics. This paper proposes a training-free Lagrangian motion magnification framework. Its core innovations are: (1) a periodic reference reset mechanism to suppress long-term error accumulation; and (2) a hierarchical tissue-aware dual-mode mask that adaptively modulates amplification intensity per region based on both motion magnitude and anatomical distance, ensuring temporal coherence and biomechanical plausibility. The method integrates pretrained visual tracking, overlapping clip partitioning, dynamic reference frame updating, and optical-flow-guided softening. Evaluated on the multi-surgical-type, highly challenging EndoVMM24 dataset, it significantly improves magnification accuracy, visual quality, and robustness—validated by consistent endorsement from clinical experts.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Manipulation
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Visualizing subtle vascular motions in endoscopic surgery is crucial for surgical precision and decision-making, yet remains challenging due to the complex and dynamic nature of surgical scenes. To address this, we introduce EndoControlMag, a training-free, Lagrangian-based framework with mask-conditioned vascular motion magnification tailored to endoscopic environments. Our approach features two key modules: a Periodic Reference Resetting (PRR) scheme that divides videos into short overlapping clips with dynamically updated reference frames to prevent error accumulation while maintaining temporal coherence, and a Hierarchical Tissue-aware Magnification (HTM) framework with dual-mode mask dilation. HTM first tracks vessel cores using a pretrained visual tracking model to maintain accurate localization despite occlusions and view changes. It then applies one of two adaptive softening strategies to surrounding tissues: motion-based softening that modulates magnification strength proportional to observed tissue displacement, or distance-based exponential decay that simulates biomechanical force attenuation. This dual-mode approach accommodates diverse surgical scenarios-motion-based softening excels with complex tissue deformations while distance-based softening provides stability during unreliable optical flow conditions. We evaluate EndoControlMag on our EndoVMM24 dataset spanning four different surgery types and various challenging scenarios, including occlusions, instrument disturbance, view changes, and vessel deformations. Quantitative metrics, visual assessments, and expert surgeon evaluations demonstrate that EndoControlMag significantly outperforms existing methods in both magnification accuracy and visual quality while maintaining robustness across challenging surgical conditions. The code, dataset, and video results are available at https://szupc.github.io/EndoControlMag/.

Problem

• Visualizing subtle vascular motions in endoscopic surgery
• Preventing error accumulation in dynamic surgical scenes
• Adapting magnification to diverse surgical scenarios

Innovation

• Periodic Reference Resetting for error control
• Hierarchical Tissue-aware Dual-Mask framework
• Adaptive softening strategies for tissue handling