RawVLA: Embodied Neural Image Signal Processor For Robotic Manipulation

2026-09-29

AI Summary

This study addresses the limitation of conventional Vision-Language-Action (VLA) models that rely on fixed Image Signal Processors (ISPs), which decouple the imaging pipeline from the learning loop and constrain robotic manipulation robustness. We propose a streaming neural ISP that, for the first time, integrates image processing into the VLA closed loop. By adaptively rendering RAW observations to optimize freezing strategies, our approach bridges perception and control. Furthermore, we establish an embodied-behavior-oriented evaluation benchmark in the RAW domain. Experimental results demonstrate that the proposed method maintains performance under standard conditions while significantly improving manipulation success rates and robustness in adverse imaging environments.

Abstract

Vision-language-action (VLA) models typically operate on RGB images produced by a fixed camera image signal processor (ISP), leaving the imaging pipeline outside the learning and evaluation loop. We systematically examine the consequences of this overlooked design choice across five fundamental ISP dimensions: gain, sensor noise, chromatic response, tonal response, and bit depth. Our analysis reveals that RAW-to-RGB processing materially shapes both action prediction and manipulation success, with different ISP dimensions exerting substantially different effects. Guided by these findings, we introduce RawVLA, a streaming neural ISP that adaptively renders RAW observations for frozen VLA policies while concentrating its capacity on the imaging factors relevant to embodied behavior. We further present RawVLA-Bench, a RAW-domain manipulation benchmark to expose image processing as an explicit evaluation variable across clean and adverse acquisition conditions. Experiments on RawVLA-Bench show that RawVLA preserves performance under standard conditions while substantially improving robustness under degraded imaging, establishing adaptive RAW processing as an effective interface between physical cameras and embodied policies.

Problem

• Vision-Language-Action models
• Image Signal Processor
• Robotic manipulation
• RAW image processing
• Embodied AI

Innovation

• Vision-Language-Action (VLA)
• Neural Image Signal Processor
• RAW-domain Processing
• Robotic Manipulation
• Embodied AI