Brain-Conditioned Action Policies for Neural Motor Decoding

2026-09-28

AI Summary

This study addresses the scarcity of paired data in neural motor decoding, which hinders the efficient mapping of motor intentions. To overcome this limitation, we propose BrainVLA, a framework that leverages language as an intermediary interface to align neural activity with pretrained vision-language-action models such as OpenVLA. By incorporating robotic priors, BrainVLA circumvents the heavy reliance on massive annotated datasets inherent in traditional task-specific mappings. Methodologically, the framework employs a LoRA-based fine-tuning strategy to train a neural encoder for neural-language alignment and constructs a multimodal-compatible dataset. Experimental results demonstrate that BrainVLA outperforms baseline models in cross-session decoding across two datasets, achieving superior R² scores and task success rates while significantly improving training data efficiency.

Abstract

Motor brain-computer interfaces (BCIs) aim to decode motor intention, enabling people with paralysis to control external devices. Neural motor decoding typically learns task-specific mappings from neural activity to kinematics, yet remains constrained by scarce paired neural-action data. We propose BrainVLA, a framework that enables neural motor decoding by drawing on a pretrained vision-language-action (VLA) model through language-mediated alignment. BrainVLA mitigates reliance on scarce paired neural-action data by leveraging VLA policies. We first construct VLA-compatible datasets including paired neural activity, action signals, language instructions, and rendered visual observations. Then, we adapt the OpenVLA-OFT policy to the target action spaces through LoRA fine-tuning. To establish an effective interface through which neural activity can convey motor intention to adapted VLA policies and guide action generation, we train a neural encoder via neural-language alignment, using language representations as semantic targets to capture latent motor intent from neural activity. The resulting neural representations serve as an endogenous intention signal to guide VLA policies to generate executable actions, while visual observations provide complementary information about the evolving task state. BrainVLA is evaluated on two neural motor datasets with different action dimensionalities using causal rollout decoding. It outperforms the evaluated baselines in cross-session decoding $R^2$ and task success rate, while demonstrating high training data efficiency. These results establish a route for neural motor decoding to draw on large-scale robotic priors through brain-conditioned VLA policies.

Problem

• Brain-computer interfaces
• Neural motor decoding
• Paired neural-action data scarcity
• Motor intention

Innovation

• Brain-Computer Interface
• Vision-Language-Action Model
• Neural Motor Decoding
• Neural-Language Alignment
• LoRA Fine-tuning