residual analysis

To address the fundamental limitation in Transformers—where residual flow capacity is tightly constrained by computational cost and model parameters, hindering efficient scaling—this paper proposes the Residual Matrix Transformer (RMT). RMT replaces conventional scalar/vector residual connections with independently scalable outer-product memory matrices, thereby redefining information storage and retrieval. It further introduces variance-aware propagation rules and a theory-guided training dynamic optimization strategy. Crucially, RMT achieves the first decoupling of residual capacity from both FLOPs and parameter count. Experiments demonstrate that, at equivalent loss, RMT reduces FLOPs by 58%, parameters by 25%, and training tokens by 41%. Across diverse downstream tasks, RMT consistently outperforms standard Transformers, delivering substantial gains in both training efficiency and generalization performance.

Prior research on regression for dynamic data streams suffers from insufficient methodological investigation and inconsistent evaluation practices. Method: We propose the first systematic evaluation framework tailored to streaming regression, unifying support for both point prediction and prediction interval tasks. The framework introduces a novel synthetic data generation strategy capable of precisely modeling complex concept drift types—including incremental drift—and establishes a multidimensional evaluation metric suite encompassing error measures, prediction interval coverage probability, and interval width. Contribution/Results: Extensive experiments across multiple state-of-the-art streaming regression methods demonstrate that our framework significantly enhances fairness, reproducibility, and robustness in model comparison. It provides a standardized benchmark and an extensible evaluation paradigm for streaming regression research.

To address the inefficiency and heavy reliance on manual effort in classifying erroneous waveforms and localizing faults during RTL-level simulation failures, this paper proposes an automated root-cause localization method based on Value Change Dump (VCD) waveforms. We innovatively design a signal selection and statistical compression strategy that preserves discriminative temporal features while achieving over 120× waveform data reduction. A lightweight machine learning classifier is then built to enable cross-design and cross-platform deployment. Evaluated on large-scale industrial RTL designs, the method achieves 94.2% accuracy in ranking the top three suspicious modules. It is the first approach to deliver high-accuracy, transferable, and low-overhead automatic attribution of RTL simulation failures, significantly reducing manual waveform analysis time. This work provides a key enabling technology for closing the functional verification loop.

Existing residual correction methods for multivariate time series forecasting lack safety guarantees, often suffering from local performance degradation or failure due to greedy optimization. Method: We propose a plug-and-play non-degradation guarantee framework featuring a causally inspired, direction-aware encoder that decouples intra- and inter-variable dynamics and models mixed residuals; it incorporates a quadruple safety gating mechanism—confidence thresholding, error sign consistency enforcement, local sensitivity suppression, and counterfactual stability verification—to rigorously constrain correction behavior. Contribution/Results: The framework ensures prediction performance never degrades across arbitrary scenarios. Evaluated on multiple benchmarks and backbone architectures (e.g., Transformer, GNN), it achieves a 99.3% non-degradation rate—significantly outperforming prior approaches—and establishes, for the first time, a unified guarantee of both safety and efficacy in residual correction.

In data stream regression, simultaneous occurrence and indistinguishability of outliers and concept drift—particularly under continuous output spaces—pose significant challenges. To address this, we propose a dual-channel joint detection framework: a fast channel employs residual analysis coupled with Exponentially Weighted Moving Absolute Deviation (EWMAD) for real-time point outlier filtering; a deep channel integrates dynamic threshold adaptation with an EWMAD-based Drift Type Decision Tree (EWMAD-DT) to distinguish abrupt from gradual concept drift online. This is the first approach enabling synchronous, fine-grained identification of both outliers and drift types, achieving both low latency and high accuracy. Extensive evaluation on multiple synthetic and real-world datasets demonstrates substantial improvements over state-of-the-art baselines, validating the method’s effectiveness and practical applicability.

This work addresses the limitations of generic pretrained models in industrial-scale video and live-stream content moderation, where platform-specific data distributions, policy objectives, and safety constraints are poorly aligned with off-the-shelf solutions, and systematic failure diagnosis and remediation mechanisms are lacking. The paper introduces a diagnostic methodology for audio-visual language models (AVLMs) that pioneers the characterization of model failures through observable feature signatures and establishes a principled mapping between failure categories and targeted intervention strategies, replacing heuristic trial-and-error approaches. Built upon multimodal foundation model architectures and validated on real-world platform traffic, this framework enables precise interventions throughout the model lifecycle. The resulting AVLM system has been deployed across more than 100 regions globally, significantly improving moderation accuracy and traceability for high-noise, semantically ambiguous, and highly diverse content.

This study addresses the lack of empirical analysis on how quality defects in requirements documents affect the performance of automated trace link recovery (TLR). For the first time, it systematically annotates 28 types of requirements quality defects across 189 use cases from two datasets, evaluates five state-of-the-art TLR methods, and analyzes their performance impacts using both statistical significance and effect size measures. The findings reveal that specific defect types differentially influence TLR effectiveness: while certain defects significantly degrade performance, others unexpectedly enhance it. Beyond identifying key factors that either hinder or facilitate TLR accuracy, this work demonstrates that the choice of TLR method should be strategically tailored to the quality characteristics of the underlying requirements documentation.

This work addresses the limitation of conventional residual connections, which sum sublayer updates with fixed coefficients and cannot dynamically assess the reliability of proposed updates. To overcome this, the authors propose Review Residuals—a novel mechanism that explicitly incorporates conditional dependence on the proposed update within the residual gating function. By employing a learnable sigmoid gate conditioned on two inputs via RMSNorm, the method dynamically scales the residual term while preserving the identity additive structure, thereby balancing training stability and representational capacity. The approach integrates seamlessly into standard Transformers and demonstrates statistically significant improvements (p<0.05) over both standard residual connections and Highway gating in models of 590M parameters and larger, with performance gains increasing with model scale. It also enables stable training of extremely deep networks.

Existing long-horizon benchmarks merely show that agent performance degrades as task length increases, yet they cannot distinguish whether this decline stems from the intrinsic difficulty of extended tasks or from error accumulation across stages. This work introduces the "horizon residual" metric, which quantifies the additional difficulty beyond what is attributable to compounding errors by comparing the actual success rate on full-length tasks against a baseline predicted from short-segment performance. We formally define this concept for the first time and establish a comparable short-task baseline framework incorporating trajectory-induced degradation analysis, context decay modeling, and log-ratio metrics. Our approach emphasizes the necessity of predefined stage segmentation and resource allocation to control confounding variables, providing an attribution tool for long-horizon evaluation and demonstrating that declining aggregate success rates alone are insufficient evidence of length-specific challenges.

This study addresses the limitations of traditional residual plot diagnostics—namely, their reliance on subjective human interpretation, low efficiency, and poor scalability—by introducing computer vision techniques for the first time to automate the assessment of residual plots in linear models. The authors develop the R package autovi and an accompanying Shiny-based interactive web application, autovi.web. Their approach leverages deep visual models to quantify the strength of structural signals in residual plots and produces interpretable diagnostic metrics. This methodology significantly enhances the consistency and efficiency of model fit evaluation, offering statisticians and data analysts a robust, objective, and scalable tool for automated diagnostic assessment in statistical modeling.