Mechanistic Interpretability Reveals Shared Causal Subspaces in Brain-to-Speech Decoders

2026-09-25

AI Summary

This study addresses the challenges of silent speech decoding in brain-computer interfaces (BCIs) and the unclear mechanisms underlying cross-modal transfer. By integrating neural decoding models with mechanistic interpretability techniques such as activation patching, we dissect the causal structure of neurons within the decoder. Our research reveals, for the first time, that distinct speech modalities share neuron populations and causal subspaces during later processing stages. These findings elucidate the intrinsic mechanism by which overt speech data enhances silent speech decoding efficiency. Consequently, this work provides both a theoretical foundation and technical guidance for developing highly efficient BCI decoders with reduced data dependency.

Abstract

Decoding covert speech, such as mimed or imagined, from brain activity is harder than decoding vocalized speech. Cross-modal transfer, where information from one speech form helps decode another, is a promising remedy; yet how a decoder internally represents and processes brain activity from different speech forms remains unclear. In this work, we ask: which internal neurons of a decoder carry cross-modal information, and are these neurons shared across different speech forms? To answer these questions, we leverage mechanistic interpretability, using recordings of the same sentences in vocalized, mimed, and imagined input pairs for activation patching. We insert the decoder's internal activity for a sentence in one condition into its processing of the same sentence in another and measure the change in decoding accuracy. We find that no single neuron drives this benefit; instead, it arises from small groups of neurons, with vocalized speech as the most useful source. These groups are largely condition-specific in the early stage of the decoder but overlap in the later stage. These findings point toward more data-efficient covert speech decoders through training objectives that encourage shared later-stage representations learned mainly from vocalized data.

Problem

• covert speech decoding
• cross-modal transfer
• mechanistic interpretability
• brain-to-speech
• shared representations

Innovation

• Mechanistic Interpretability
• Brain-to-Speech Decoding
• Cross-modal Transfer
• Activation Patching
• Covert Speech