scattering capacity analysis

This study addresses the problem of quantifying the feature separation capacity of convolutional neural networks in classification tasks. Building upon Cover’s function counting theory and integrating tools from combinatorics with an analysis of scattering network architectures, we extend the classical theoretical framework to derive, for the first time, a practical expression for the separation capacity specific to scattering networks. Our analysis elucidates the critical influence of architectural components—such as the number of filters and network depth—on separation capability. Furthermore, this work establishes the first theoretically grounded design principles for scattering networks, offering both significant theoretical insight and practical utility for network architecture development.

This work investigates the design of pooling-free scattering networks employing fixed monomial nonlinearities to maximize separability for data with low intrinsic dimensionality. By integrating frame theory, geometric measure theory, and moment analysis, the study provides the first geometric characterization of a scattering network’s separation capacity, establishing theoretical bounds for feature extractors operating on low-dimensional rectifiable data. The core contribution consists of two practical design principles: the network’s filters must span a sufficiently broad frequency range, and the frame formed by these filters—when coupled with the data’s geometric structure through a coupling matrix—must exhibit a well-conditioned condition number. These criteria jointly ensure significantly enhanced separation performance, offering concrete guidance for the construction of effective scattering architectures tailored to geometrically structured low-dimensional data.

本文提出神经谱容量(NSC)方法，从网络架构规格计算能力分配，无需模型实例化、数据或梯度，以优化Transformer和CNN架构设计。

This work addresses non-Euclidean geometric data—including directed graphs, signed graphs, and manifolds with boundary—by proposing a unified geometric scattering transform framework on metric measure spaces. Methodologically, it integrates diffusion maps, multi-layer wavelet-type scattering, and stochastic sampling approximation to yield a computationally tractable model that simultaneously satisfies group invariance and Lipschitz stability, while establishing a data-driven graph construction scheme and a quantitative convergence theory. Key contributions include: (i) the first extension of geometric scattering to manifolds with boundary and to directed/signed graphs; (ii) a universal invariance criterion applicable across diverse geometric domains; and (iii) a rigorous proof of asymptotic convergence of scattering transforms on sampled graphs to those on continuous manifolds. Experiments demonstrate superior effectiveness and robustness on spherical images, directed networks, and high-dimensional single-cell data, offering a theoretically grounded and practically viable tool for geometric deep learning.

This study addresses the loss of algorithmic interpretability in neural network parameters following training by proposing a controlled fine-tuning framework that preserves algorithmic semantics. The method initializes from network architectures that explicitly implement statistical algorithms, such as the scattering transform, and constrains the fine-tuning radius to retain parameter interpretability. Furthermore, an analytical bound relating the fine-tuning radius to generalization error is derived based on PAC-Bayes theory. Evaluated on texture classification tasks, the proposed approach successfully reconciles high interpretability with strong predictive performance. This work establishes a new paradigm for constructing deep models that are both grounded in statistical theory and inherently transparent.

This study addresses the limitation of conventional data scaling research that overlooks disparities in sample size, source diversity, and spatial coverage within spatially structured data by reformulating data scaling as a resource allocation problem. Methodologically, it establishes these three dimensions as independent scaling axes for the first time, employing a spatial proximity-supervised contrastive learning model to process 11.6 million whole-brain tissue sections. Results demonstrate that representation performance improves with increased sample size, spatial coverage, computational resources, and model capacity. Furthermore, the findings reveal that under fixed budgets, merely expanding data sources yields no benefit, and cross-subject generalization is constrained by biological variation rather than driven by the number of sources.

Traditional metrics struggle to characterize the geometric organization of task-relevant information in the high-dimensional state space of reservoir computing. This work proposes an analysis framework based on spectral decomposition, establishing—for the first time—a quantitative link between state-space modes and information processing capacity. By integrating spectral decomposition, information capacity analysis, and nonlinear dynamical system modeling, the study quantifies the representational energy of distinct modes and reveals that low-energy modes, despite their susceptibility to noise, may nonetheless encode critical computational functions. These findings challenge the conventional design paradigm that relies solely on high-dimensional expansion and offer a new theoretical foundation for optimizing physical reservoirs.

This work investigates rank collapse in deep Transformers at initialization, where nonlinearities and matrix multiplications degrade representational capacity and training stability. The authors systematically analyze how components within feedforward blocks influence rank preservation across depth, unifying skip connections and normalization mechanisms under a common framework as gradient-based rank-preserving strategies. They reveal a fundamental distinction between Pre-Norm and Post-Norm architectures in terms of rank dynamics and demonstrate that the two-matrix structure and width expansion are critical for maintaining full-rank Jacobians. Through spectral analysis, Jacobian rank tracking, Marchenko–Pastur law modeling, and CIFAR-10 experiments, they establish that the rank of the input–output Jacobian at initialization strongly predicts training success, offering a new principle for deep architecture design grounded in rank evolution.

本文提出Neurogenesis Network(NGN)，通过可学习边界选择模型结构组件数量，优化神经网络大小，提高性能。

This study addresses the fragmentation and limited diagnostic utility of existing multi-reference image generation benchmarks by proposing a capability-oriented evaluation perspective. We formalize multi-reference generation as compositions of four atomic operators and construct Trace-Bench, a dataset comprising 1,600 cases. Furthermore, we introduce a novel compositional evaluation framework based on operator alignment alongside a recursive fault diagnosis tree mechanism. Experimental results precisely localize the root causes of model failures, revealing that attribute disentanglement and binding remain primary bottlenecks, with the state-of-the-art model achieving an attribute fidelity score of only 0.74. Ultimately, this work establishes an interpretable diagnostic paradigm for advancing multi-reference generation research.