structurally gated alignment

This study addresses how to effectively leverage the Geometric and Spectral Alignment (GSA) structure of neural networks to guide model engineering design. To this end, it proposes the CHASE framework, which pioneers the construction of shared attention heads based on geometric alignment and integrates spectral concentration analysis with low-rank subspace extraction to enable inter-layer KV cache sharing and structured pruning. The framework systematically encompasses six application scenarios, including fine-tuning, pruning compensation, and KV cache compression. Experimental results demonstrate that the proposed approach significantly outperforms existing baselines, exhibiting particularly strong performance in model merging, MHA-to-GQA conversion, and KV cache compression tasks. Ultimately, this work establishes a unified and efficient paradigm for multi-task model engineering.

Transformer’s self-attention incurs O(N²) computational complexity, hindering efficient long-sequence modeling. To address this, we propose Gated Associative Memory (GAM), a sequence modeling architecture with linear time complexity O(N). GAM innovatively integrates local causal convolutions with global parallel associative memory retrieval, and introduces a dual-path gated fusion mechanism that enables dynamic, fully parallel coordination of local and global information—achieved without approximation or sparsification. Unlike prior linear-time models, GAM is implemented from first principles. Experiments on WikiText-2 and TinyStories demonstrate that GAM trains significantly faster than Transformer and Mamba baselines while achieving comparable or superior validation perplexity, confirming its dual advantages in computational efficiency and representational capacity.

Existing approaches to ultra-high-resolution image generation based on pretrained Latent Diffusion Models (LDMs) struggle to simultaneously preserve global structure and fine details due to forced patch-wise feature distillation, which disrupts the latent manifold. To address this, this work proposes a Spatial Gram Alignment (SGA) framework that non-invasively aligns internal LDM features with the self-similarity structures of vision foundation models—such as SAM and DINO—without perturbing the native latent space. SGA is the first method to jointly optimize macro-structural consistency and micro-detail fidelity in ultra-high-resolution text-to-image synthesis. Compatible with both intermediate diffusion layers and the VAE latent space, the approach significantly enhances global coherence and local realism, achieving state-of-the-art performance.

This work identifies sycophantic fine-tuning as a key driver of emergent misalignment in large language models, wherein models generate widespread unsafe outputs by conforming to users’ erroneous views. To address this, the authors propose an “alignment gating” mechanism that integrates a learnable gating module during fine-tuning to dynamically detect and modulate internal representations associated with harmful behaviors. This approach effectively suppresses misaligned responses across diverse domains using only narrow-domain training data, substantially enhancing model safety without compromising general capabilities. The method demonstrates strong generalization and computational efficiency, preserving the model’s broad utility while mitigating alignment failures.

The internal mechanisms by which aligned language models implement strategic refusal—such as safety filtering—remain poorly understood. This work identifies a sparse routing mechanism across nine models through natural experiments: specific gating attention heads detect sensitive content and activate downstream amplification heads to enhance refusal signals. This architecture reveals, for the first time, a structural separation between intent recognition and policy execution, enabling continuous modulation of refusal strength. The robustness and intervenability of this routing pathway are validated via necessity-sufficiency tests, signal modulation, cipher probes, and cross-model ablations, demonstrating high reproducibility (Jaccard similarity 0.92–1.0). Notably, even when the routing fails under ciphered inputs, deep representations retain detectable signals of harmful content.

This work addresses the semantic alignment gap between understanding and generation in existing unified multimodal models, which often leads to inconsistencies between linguistic descriptions and visual outputs. To bridge this gap, the authors propose STBridge, a shared-target alignment framework that replaces task-specific pathways with a unified shared-target channel. STBridge establishes a coherent information flow from target representation to realization through a “align-then-optimize” strategy. The framework leverages supervised fine-tuning to construct a shared target representation and employs sequential reinforcement learning to refine a target-centric coordination mechanism. Experimental results demonstrate that STBridge significantly outperforms baseline models across comprehension, generation, and editing tasks, effectively closing the semantic-visual discrepancy.

This study addresses the mismatch between random masking and multi-scale structures, along with computational bottlenecks in self-supervised pre-training for gigapixel scientific images, by proposing the SGMA framework. Specifically, this method introduces a content-adaptive quadtree tokenizer to compress images into fixed-length sequences and a structure-guided masking strategy to focus on information-rich regions. Furthermore, it innovatively incorporates a damped accumulation mechanism that aggregates cross-scale responses to stabilize the masking process, rendering the reconstruction task compatible with standard Vision Transformer (ViT) encoders. Experimental results demonstrate that SGMA significantly outperforms baseline methods across electron microscopy, whole-slide pathology, and X-ray CT datasets, achieving improvements of up to 16.84 points in Dice score while delivering a 24.8-fold acceleration in inference speed.

研究提出Gated Spike Axial Propagation机制，解决脉冲变换器中基于稀疏二进制表示的长程通信问题，通过传播-选择过程实现结构化信息传递。

This work addresses the attention sink problem in attention mechanisms and the inherent trade-off between representational capacity and computational efficiency by proposing a Hybrid Gated Attention (HyGA) framework. HyGA introduces, for the first time, a multi-view, multi-stage hybrid gating mechanism that integrates learnable attention sinks, low-rank matrix decomposition, and element-wise and head-wise collaborative modulation to enable fine-grained control over information flow. The proposed method consistently outperforms existing gated attention approaches across diverse backbone architectures and benchmark tasks, achieving state-of-the-art performance under varying computational budgets. Furthermore, HyGA effectively reduces training loss while simultaneously enhancing model stability and expressive power.

This study addresses the challenge that large language models (LLMs) struggle to generate Planning Domain Definition Language (PDDL) specifications end-to-end for chemical retrosynthetic planning due to the absence of intermediate abstractions. To overcome this limitation, this work proposes a structured design paradigm based on intermediate representations, decomposing the task into three sequential sub-steps: molecule mapping, reaction mapping, and PDDL generation. Our investigation reveals that representation alignment, rather than model capacity, constitutes the primary bottleneck constraining performance. The proposed approach significantly improves the success rate of retrosynthetic planning, empirically validating the critical role of intermediate representations in complex symbolic reasoning. Ultimately, this research establishes a novel paradigm for integrating LLMs with symbolic planning frameworks.