gradient-based attribution

Existing approaches struggle to uncover the causal influence of hidden neurons on neural network outputs, as activation patterns alone are insufficient to decipher internal computational mechanisms. This work proposes CODEC, a novel method that—unlike prior activation-based analyses—decouples network behavior into interpretable, sparse contribution modes by integrating contribution decomposition with sparse autoencoders. Applying this framework, the study reveals cross-layer causal computation pathways and demonstrates its efficacy in both image classification and retinal neural activity modeling. The approach enables precise intervention and visualization of intermediate layers, uncovers a progressive decoupling of positive and negative contributions in deeper layers, and elucidates how compositional interactions among intermediate neurons give rise to dynamic receptive fields.

This work addresses the opacity of neural network training, which poses significant challenges for beginners. To enhance interpretability, the authors developed an interactive web-based visualization tool that, for the first time, synchronously displays the dynamic correspondence between weight updates and activation signals within a single training iteration, while also rendering neuron-specific update equations in real time. The system enables fine-grained tracking and dynamic visualization of the training process, allowing users to interactively configure and observe forward and backward propagation, activation values, and loss evolution. In a user study with 31 participants, the tool achieved a System Usability Scale (SUS) score of 80.97—indicating excellent usability—with mean rankings of 2.47 for clarity and 2.23 for practicality; over 70% of users reported that it substantially improved their understanding of neural network training mechanisms.

This work addresses the lack of unified, cross-type evaluation for input feature attribution methods. We propose the first automated, horizontal evaluation framework covering token-level, token-interaction-level, and span-interaction-level explanation methods. Grounded in four diagnostic properties—faithfulness, stability, interpretability, and robustness—the framework integrates diverse techniques including Shapley values, Integrated Gradients, Bivariate Shapley, attention mechanisms, and Louvain-based Span Interactions. It is systematically validated on two benchmark datasets (SST-2 and BoolQ) and two foundational models (BERT and RoBERTa). Results demonstrate that span-interaction explanations significantly outperform conventional approaches across most metrics, revealing their previously underappreciated potential; moreover, the three explanation types exhibit complementary strengths. This study establishes a reproducible, principled benchmark for scientifically selecting and improving attribution methods.

The causal origins of interpretable units—such as induction heads—in large language models remain poorly understood. This work proposes a scalable mechanistic data attribution framework that integrates influence functions with causal interventions to establish, for the first time, direct causal links between specific training examples and the emergence of such interpretable components. The study reveals that structured repetitive data plays a catalytic role in circuit formation and demonstrates a direct functional relationship between induction heads and in-context learning capabilities. By selectively intervening on a small set of high-influence training samples, the emergence of attention heads can be significantly modulated. Furthermore, the proposed data augmentation strategy consistently accelerates circuit convergence across different model scales.

This study addresses the lack of systematic evaluation of tokenization strategies for magnetoencephalography (MEG) signals in current large-scale neuroimaging foundation models, which impacts both model performance and biological plausibility. The work presents the first comprehensive assessment of sample-level tokenization approaches in MEG foundation models, comparing learnable (a novel autoencoder-based tokenizer) and non-learnable strategies across multiple criteria: signal reconstruction, token prediction accuracy, biological plausibility, preservation of individual-specific information, and downstream task performance. Experiments on three public MEG datasets demonstrate that both approaches achieve comparable performance across most metrics and attain high reconstruction fidelity, suggesting that fixed tokenization schemes are sufficient to support the development of efficient and biologically plausible neural foundation models.

Although trained solely on category labels, video classification models implicitly encode task-irrelevant yet semantically rich information about action outcomes—such as success or failure—posing challenges for trustworthy AI. This work employs mechanistic interpretability to perform causal reverse engineering on a pretrained VideoViT, revealing for the first time a distributed and redundant internal circuit dedicated to computing action outcomes. Specifically, attention heads aggregate low-level evidence while MLP blocks compose high-level concepts to construct success signals. The study identifies a progressively amplified semantic pathway spanning layers 5 to 11, confirming the complementary roles of attention and MLP components. Through activation patching and ablation experiments, the model demonstrates robust reliance on this implicit knowledge, highlighting its resilience and structured internal representation of outcome-related semantics.

This study addresses the limited interpretability of existing digital biomarkers for Parkinson’s disease, which predominantly rely on statistical features without modeling the underlying motivation of typing behavior. The authors propose a novel approach by introducing inverse reinforcement learning to keystroke dynamics analysis, treating each keystroke as a speed-selection action and recovering an interpretable three-parameter reward function that characterizes individual temporal typing patterns. Through rigorous validation—including reward function decomposition, correlation analysis, sensitivity testing, and data leakage controls—they demonstrate that the recovered speed-preference weight exhibits a significant negative correlation with UPDRS-III scores (r = –0.607, p < 0.001). This association remains robust across multiple sub-cohorts and configurations and provides incremental explanatory power beyond raw typing speed (ΔR² = 0.144, p = 0.006).

This work addresses a critical limitation of existing explainable AI methods, which predominantly offer passive attribution and thus fail to support practitioners in effectively intervening on model behavior. To bridge this gap, the authors propose an interactive analysis workflow that integrates sparse autoencoder (SAE)-based attribution with activation intervention, introducing activation steering into human-in-the-loop debugging for the first time and enabling a paradigm shift from observation to active intervention. Through semi-structured interviews with eight domain experts, the study reveals that users commonly engage in intervention-based hypothesis testing, primarily employing component suppression strategies and grounding their trust in model responses rather than the plausibility of explanations. The research also uncovers key risks—including ripple effects and limited instance-level generalizability of corrections—thereby charting a new path toward trustworthy AI debugging.

This work addresses the challenge that explanation methods in explainable artificial intelligence (XAI) often introduce signal noise in medical applications, thereby obscuring the true decision-making basis of models. To tackle this issue, the study proposes spectral entropy as a novel, objective metric to quantify the noise induced by XAI techniques on electrocardiogram (ECG) signals. Leveraging post-hoc explanation methods such as Grad-CAM and Integrated Gradients, the approach is evaluated within the context of arrhythmia classification. Experimental results demonstrate that spectral entropy effectively identifies and measures the interference introduced by XAI, offering a new paradigm for enhancing the fidelity and trustworthiness of explanations generated by deep learning models in clinical decision-making.