activation based attribution

This work addresses the limitation of existing output-confidence–based fault detection methods, which often fail to capture internal errors in neural networks. The authors propose Self-Detecting Neural Networks (SDNN), a novel framework that introduces the concept of “spectral drift” to reveal that erroneous predictions manifest as pronounced multi-scale spectral instabilities in internal activations. Spectral features are extracted via short-time Fourier transform, wavelet decomposition, and statistical moments, and a lightweight detector is trained using curriculum learning to establish an end-to-end learnable internal monitoring mechanism. Evaluated on CIFAR-10, SDNN achieves an AUROC of 79.0 ± 25.3%, outperforming baseline methods such as MaxSoftmax and Energy Score by 25–30 percentage points.

This work addresses a critical limitation in existing neuron-level concept explanation methods, which often assume that all neurons possess clear functional roles, thereby overlooking redundant or misleading neurons that can distort interpretations of model decision-making. To overcome this, the authors propose the Select-Hypothesize-Verify (SHV) framework: it first selects the most representative samples based on activation distributions, then generates natural language concept hypotheses, and finally validates these hypotheses through a neuron activation verification mechanism. SHV introduces, for the first time, a systematic pipeline for concept validation, effectively identifying and focusing on neurons with genuine semantic meaning. Experimental results demonstrate that concepts produced by SHV activate target neurons at 1.5 times the rate of state-of-the-art methods, substantially improving the accuracy and reliability of model interpretations.

This paper addresses the challenge of identifying hierarchical, distributed activation patterns in neural networks. To overcome limitations of neuron-level or hand-crafted interpretable feature analyses, we propose Neural Activation Pattern (NAP) modeling based on full-layer activation distributions. Methodologically, we introduce normalized preprocessing, kernel density estimation for distribution modeling, SNR-adaptive distance metrics, and hierarchical clustering. For the first time, our approach uncovers a continuous activation manifold in neural communication receivers that is dominantly governed by signal-to-noise ratio (SNR), demonstrating its physical interpretability. Experiments show that NAP significantly improves separation between in-distribution and out-of-distribution samples, empirically confirming SNR as a critical implicit factor learned by the model. The method enhances generalization, interpretability, and reliability diagnostics—enabling robust, physics-informed analysis of deep neural representations.

This work proposes a “patterning” approach that reframes interpretability as the active shaping of a neural network’s internal structure and generalization behavior through deliberate training data design. Grounded in linear response theory, the method employs susceptibilities to quantify model sensitivity to data perturbations and inversely solves for optimal data intervention strategies—such as reweighting and localized learning coefficient optimization—to directionally control the formation of inductive circuits. Experiments on small language models demonstrate that this technique can effectively accelerate or delay the emergence of specific circuits and, in the context of a bracket balancing task, guide the model to learn a prescribed algorithm. This represents the first demonstration of actively writing and regulating internal model structures through targeted data interventions.

This work investigates the topological structure of activation patterns in ReLU neural networks and its intrinsic relationship with model behavior. For binary classification, we propose characterizing the geometry of decision boundaries via Fiedler partitioning on the dual graph of the piecewise-linear partition induced by the network. For regression, we introduce algebraic topological tools—specifically, homology group computation—to quantify the complexity of the polyhedral cell decomposition; we observe a strong dynamic correlation between training loss and the number of cells. Experiments reveal that the cell count monotonically decreases in a predictable manner during training, and the Fiedler partition closely approximates the true decision boundary. This study establishes, for the first time, a systematic bridge among the piecewise-linear structure of ReLU networks, graph-theoretic partitioning, and algebraic topological invariants—providing a novel geometric framework for understanding generalization in deep networks.

This study investigates whether large language models can perceive the degradation of their computational substrate caused by quantization. Inspired by agnosia, we explore the self-monitoring capabilities of models using linear probes and LoRA fine-tuning to extract and analyze “quantization fingerprints” from both internal representations and generated text. We demonstrate that internal representations offer greater interpretability than external outputs and propose a quantization-aware pathway based on these internal fingerprints. Our findings confirm that while models struggle to identify their quantization state from isolated outputs, quantization fingerprints can be effectively decoded from internal representations. Furthermore, this work reveals limitations in the generalizability of such perception mechanisms across different quantization methods, offering novel insights into the underlying self-awareness of large language models.

This study addresses the inefficiency and limited scalability of manual artifact component identification in traditional electroencephalography (EEG) research following independent component analysis (ICA). To overcome this bottleneck, the work introduces computer vision techniques into the automatic labeling of ICA components for the first time, developing an end-to-end automated system compatible with both EEGLAB and ICLabel. The proposed method enables efficient detection and removal of non-neural components, substantially reducing reliance on expert annotation. It achieves a classification accuracy of 89.45% while accelerating processing speed by a factor of 7,200 compared to manual approaches, thereby facilitating large-scale and near real-time EEG analysis.

本文提出了一种基于神经元激活的符号框架，使用SMT求解器等逻辑引擎有效计算深度神经网络行为的解释，解决了现有技术无法处理深层架构的问题。

This work addresses the challenge of efficiently selecting the optimal model checkpoint and enabling early stopping in the absence of a labeled validation set. It proposes a lightweight, label-free proxy metric that leverages the Frobenius norm of the classification head’s weight gradients—computed from a single forward-backward pass—as a performance prediction signal. To the best of our knowledge, this is the first approach to utilize per-batch gradient norms as a universal, unsupervised estimator of model performance across diverse tasks and architectures, including image classification, object detection, segmentation, and diffusion models. The method adapts to different network structures through feature or head-scale normalization. Experiments demonstrate near-oracle checkpoint selection on ImageNet-1k (average gap of only 1.12%) and effective prediction of mAP and FID on COCO and CIFAR-10 diffusion models, with computational overhead below 0.1% of a single training epoch.

This work addresses the challenge of reliably identifying and attributing content generated by large language models without compromising textual quality. The authors propose a lossless fingerprinting method that embeds detectable, semantically non-intrusive identification signals by injecting random sparse vectors into the residual stream, thereby encoding fingerprints within the model’s activation space. This approach demonstrates, for the first time, that large language models can achieve high-precision self-identification and support multi-model attribution. Experimental results show attribution accuracy exceeding 98% across diverse detection settings, while preserving the fluency and coherence of generated text with no statistically significant degradation in quality.