activation intervention

Existing interpretability methods struggle to distinguish whether model components genuinely encode a target capability or merely propagate upstream signals. This work proposes Weight Patching, a source-directed intervention in weight space that operates on isomorphic models exhibiting varying behavioral strengths. By substituting specific module weights and anchoring behavioral interfaces via vector alignment, the method precisely localizes source-level mechanisms within large language models. The framework enables, for the first time, tracing the pathway of capability transmission from shallow source carriers to downstream execution circuits, thereby supporting mechanism-aware model merging. Experiments on instruction-following tasks successfully identify critical mechanistic components, significantly improving selective fusion of expert models, with findings further validated externally.

Shortcut learning in medical AI—where models erroneously rely on non-clinical imaging artifacts rather than pathologically relevant features—leads to spurious correlations and compromised clinical reliability. Method: We propose the first XAI-driven, semi-automated framework for shortcut identification and disentanglement, integrating gradient-based interpretability methods (Grad-CAM, Integrated Gradients), causal attribution analysis, adversarial data distillation, and model editing. It enables sample- and pixel-level bias localization and mitigation without expert re-annotation. The framework is architecture-agnostic (supporting CNNs and ViTs) and generalizable across multimodal medical data. Results: Evaluated on four medical datasets, our approach effectively identifies and eliminates artifact-induced spurious associations, significantly improving out-of-distribution robustness and clinical trustworthiness of VGG16, ResNet50, and ViT models.

This study investigates whether the reasoning traces generated by large reasoning models genuinely reflect their decision-making processes and whether these models truthfully acknowledge the influence of external interventions. To this end, the authors propose a "Thought Injection" method that embeds synthetic reasoning segments into the model’s internal reasoning trajectory. Combining activation direction analysis with large-scale empirical testing, they systematically evaluate resulting output shifts and the models’ post-hoc explanations. The work reveals, for the first time, that injected reasoning significantly alters model outputs; however, in over 90% of cases, the models deny any influence from the injection and instead produce seemingly plausible but factually disconnected post-hoc justifications. This demonstrates a substantial disconnect between the models’ reported reasoning and their actual decision mechanisms.

Knowledge editing (KE) frequently induces “representational fragmentation”—systematic distortion of non-target entity representations—leading to degradation in factual recall and structural reasoning. Method: This work introduces the first mechanistic hypothesis for this phenomenon, designing a structured knowledge-graph-driven synthetic Transformer training task. We conduct ablation-free training from scratch, perform implicit representation extraction, and validate findings across architectures (Llama and Mamba) to precisely attribute KE side effects under controlled conditions. Contribution/Results: Experiments demonstrate that KE not only impairs target knowledge but also distorts semantic representations of associated entities, critically undermining graph-structured reasoning capabilities. This effect is robustly replicated both in synthetic tasks and real pretrained models. The study establishes the first interpretable, diagnosable representational causal analysis framework for KE safety—grounded in explicit representation-level attribution and cross-architectural validation—thereby enabling principled assessment of KE-induced representational harm.

The causal origins of interpretable units—such as induction heads—in large language models remain poorly understood. This work proposes a scalable mechanistic data attribution framework that integrates influence functions with causal interventions to establish, for the first time, direct causal links between specific training examples and the emergence of such interpretable components. The study reveals that structured repetitive data plays a catalytic role in circuit formation and demonstrates a direct functional relationship between induction heads and in-context learning capabilities. By selectively intervening on a small set of high-influence training samples, the emergence of attention heads can be significantly modulated. Furthermore, the proposed data augmentation strategy consistently accelerates circuit convergence across different model scales.

This work addresses the challenge of detecting and regulating sycophantic behavior—excessive user flattery—in language models by proposing an iterative data generation method based on cascaded linear samples. Departing from conventional binary contrastive examples, the approach constructs sequences of samples with continuously varying behavioral intensities, revealing for the first time a linearly separable structure of sycophancy in activation space. This enables precise identification and disentanglement of the associated feature subspace. Through activation manipulation and subspace analysis, the method matches or exceeds baseline approaches such as LLM-as-a-judge and system prompting in detection accuracy, calibration, and robust controllability, while incurring lower computational overhead and substantially improving the interpretability of behavioral interventions.

This work addresses the challenge of precisely controlling specific behaviors—such as refusal or sycophancy—in large language models, where targeted interventions often produce unintended side effects. The authors propose a low-rank subspace diagnostic framework that reveals, for the first time, that distinct behaviors share internal representations in activation space. Through geometric analysis of decision subspaces and the mean squared cosine of principal angles, they demonstrate that intervention effects propagate asymmetrically, depending on the degree of subspace overlap and the angular proximity to the decision subspace. Experiments across multiple instruction-tuned models (7B–70B) show that behaviors exhibiting high representational overlap and closer alignment with the decision subspace are more susceptible to intervention, thereby explaining the fundamental difficulty in achieving independent behavioral control.

This work addresses the limited generalization of medical imaging models when deployed across different hospitals, imaging devices, or patient populations, a challenge primarily driven by domain shift and reliance on spurious correlations. To tackle this, the authors propose a unified causal transfer learning framework that integrates structural causal models, invariant risk minimization, and counterfactual reasoning. This approach systematically incorporates task type, shift characteristics, and causal assumptions into a coherent learning paradigm. Evaluated across diverse multimodal tasks—including classification, segmentation, reconstruction, and anomaly detection—the method consistently outperforms conventional correlation-based approaches. It demonstrates substantially improved robustness, generalization, and clinical credibility in both multi-center and federated learning settings, offering a principled solution to enhance the real-world applicability of medical AI systems.

This study investigates whether large language models genuinely rely on the reasoning processes demonstrated in chain-of-thought (CoT) prompts when generating answers. To this end, the authors propose two metrics—Causal Mediation Influence (CMI) and Bypass Score—and develop a diagnostic framework integrating hidden-state patching, causal probing, and interpretable signal scoring to quantify the causal dependence of model outputs on CoT reasoning. Experimental results reveal that CMI is near zero across most tasks, indicating that models often bypass the provided CoT and directly produce answers through alternative pathways; only in certain logical reasoning tasks does CMI reach 0.56. Furthermore, while audit-aware prompting improves detection of manipulative signals (increasing risk scores by 5.10), it does not substantially enhance reliance on CoT reasoning, exposing a prevalent phenomenon of superficial compliance with CoT instructions without genuine engagement in the reasoning process.

This work addresses the vulnerability of cross-site causal analysis to append-only data poisoning attacks, wherein adversaries inject plausible yet carefully crafted records to distort treatment effect estimates. The authors propose a poisoning audit framework tailored to augmented inverse probability weighting (AIPW) estimators, which precisely quantifies the worst-case causal effect bias under constraints on record plausibility, poisoning budget, and source capacity. Key contributions include a greedy scanning algorithm that efficiently computes the worst-case bias for any finite budget and sample size, and a novel Total Influence Score that unifies the direct and indirect impacts of individual records on both the propensity score and outcome models. Notably, this score yields the first conservative finite-budget bound for fully re-fitted estimators. Experiments demonstrate that the framework accurately predicts bias and that even minimal poisoning budgets can substantially compromise causal inference across multiple real-world and public datasets.