projection head design

Aligning low-dimensional manifolds in neural implicit spaces—specifically, measuring intrinsic similarity, establishing cross-space point correspondences, and enabling effective representation transfer—remains challenging. Method: This paper introduces the Functional Maps paradigm to neural implicit space modeling for the first time, proposing a spectral-geometry-based, interpretable multi-task alignment framework. It constructs functional bases in implicit spaces via Laplacian eigen-decomposition and jointly optimizes weakly supervised correspondences to unify cross-manifold mapping in the functional domain. Contribution/Results: The framework supports unsupervised and weakly supervised correspondence discovery, geometry-consistent similarity measurement, and cross-modal representation transfer. Empirically, it achieves significant performance gains on image stitching and cross-modal retrieval tasks, demonstrating its effectiveness and generalizability as a universal representation alignment method.

Neural models trained on identical tasks exhibit geometrically heterogeneous latent representations due to stochasticity and architectural differences, rendering cross-model representations incomparable. To address this, we propose relative geodesic representations grounded in pullback metrics—a novel application of differential-geometric pullback metrics to latent space alignment—explicitly modeling intrinsic geometric transformations between latent manifolds of distinct models. Unlike conventional linear alignment methods, our approach operates without supervision and generalizes across diverse architectures and pretraining paradigms. Experiments on autoencoders and vision foundation discriminative models demonstrate substantial improvements in cross-model retrieval accuracy and model stitching performance. Moreover, the method scales effectively to large-scale settings.

Existing image representation methods often struggle to simultaneously support both recognition and generation tasks. This work proposes a hypernetwork architecture based on Implicit Neural Representations (INRs), which encodes images into compact model weights that enable efficient reconstruction. By integrating knowledge distillation with pixel-level and perceptual losses, the method establishes a unified visual representation framework. It is the first approach to achieve high-accuracy recognition and high-quality image generation within a single shared embedding space, demonstrating state-of-the-art performance across diverse vision tasks while maintaining a highly compressed embedding dimensionality.

This work addresses the geometric incompatibility of latent representation spaces across neural networks—arising from differences in architecture, training protocols, or data—which impedes cross-model semantic alignment. To this end, the authors introduce SEMASIA, a large-scale benchmark dataset comprising embeddings from approximately 1,700 pretrained vision models evaluated on eight image classification benchmarks, along with rich structured metadata. Leveraging supervised alignment, reconstruction error analysis, downstream task evaluation, and regression modeling, the study reveals consistent prototype clustering and hierarchical semantic structures across diverse models, establishes a standardized benchmark for alignment methods, and quantifies the influence of pretraining data composition and model scale on embedding geometry.

Existing weight-space learning approaches struggle to align model weights with the information contained in training datasets, limiting their effectiveness in downstream tasks. This work proposes WeightCLIP, the first method to explicitly incorporate dataset-level information into weight-space learning. It employs an autoencoder to encode model weights and a dedicated dataset encoder to extract sample features, then leverages contrastive learning to align these two representations. Furthermore, a latent-space refinement procedure is introduced to enhance the quality of generated weights. Evaluated across model retrieval, generation, and refinement tasks, WeightCLIP significantly outperforms standard fine-tuning and enables the generation of models with strong generalization capabilities conditioned on dataset information.

Existing approaches typically rely solely on the final layer of deep models or employ simplistic fusion of shallow representations, overlooking the fact that task-relevant information is non-monotonically distributed across intermediate layers and cannot be reliably recovered through naive aggregation. This work systematically uncovers, for the first time, the distributional patterns of informative representations in intermediate layers and introduces a Layer-wise Optimal Embedding Selection (LOES) strategy grounded in spectral analysis, complemented by a geometric regularization loss (GeoReg). By leveraging constructive spectral methods along with orthogonality and isotropy constraints, the proposed framework precisely identifies task-discriminative subspaces and stabilizes their geometric structure. The method consistently outperforms baselines across diverse architectures, modalities, and data scales, exhibits performance gains with increasing model depth, and enables interpretable cross-lingual and cross-modal analyses.

This work addresses the limitations of inefficient textual communication in multi-model collaboration, which often leads to information loss and constrained coordination. The authors propose a parameter-efficient method that aligns the key-value (K-V) caches of multiple large language models within a shared latent space without modifying their pre-trained parameters. By introducing lightweight adapters, the approach enables efficient cross-model translation of internal states. This is the first method to achieve latent-space alignment of K-V caches across distinct models, facilitating high-bandwidth communication and direct transfer of capabilities such as soft prompting. Experiments with the Gemma-2 model family demonstrate that the framework not only enables seamless inter-model collaboration and knowledge sharing but also significantly enhances individual model performance on downstream tasks, thereby validating the effectiveness and scalability of latent-space collaborative mechanisms.

This work establishes that feedforward networks constitute a strict subset of generalized convolutional networks and identifies a parameterization mismatch between the two architectures. To address this, the authors propose a model projection method based on a unified tensor-activation formulation: by freezing pretrained convolutional filters and learning only scalar gating parameters, the approach enables parameter-efficient transfer learning. This strategy effectively inherits optimization techniques developed for feedforward networks while drastically reducing the number of trainable parameters. Using only simple training protocols, the method achieves strong transfer performance across multiple ImageNet-pretrained backbones and downstream image classification tasks.

Existing multimodal embedding spaces, such as CLIP, tend to prioritize dominant objects during semantic compression while suppressing fine-grained attributes like viewpoint or color tone, thereby limiting precise retrieval and manipulation. To address this, this work proposes a text-conditioned affine transformation network that dynamically modulates visual features within a frozen embedding space, enabling explicit disentanglement and emphasis of any user-specified attribute—such as color or artistic style—without requiring re-encoding. This approach achieves, for the first time, real-time control over the embedding space via natural language instructions at zero inference cost, supporting structured organization and retrieval across multiple attributes. Experiments demonstrate state-of-the-art performance on attribute retrieval and multi-attribute clustering tasks, while maintaining high computational efficiency and compatibility with existing models.

This work addresses the challenge of low sample efficiency in fMRI-based encoding and decoding models, which stems from the scarcity of paired fMRI–stimulus data and substantial inter-subject variability. To overcome this, the authors propose a lightweight latent embedding alignment framework that operates with frozen pre-trained encoders and decoders. By leveraging abundant unpaired stimulus embeddings through a reverse semi-supervised learning strategy, the method introduces a novel meta-transfer mechanism that integrates residual debiasing and sparse aggregation to enable effective cross-subject knowledge transfer and alignment refinement. Theoretical analysis provides generalization bounds and safety guarantees under limited sample regimes. Extensive experiments on large-scale fMRI image reconstruction benchmarks demonstrate significant improvements in both sample efficiency and reconstruction performance, confirming the approach’s effectiveness and robustness.