See What I See, Know What I Think: Dense Latent Communication Across Heterogeneous Agents

2026-06-11

AI Summary

This work addresses the challenges of heterogeneous multi-agent communication—namely, substantial information loss, high computational overhead, and misalignment in latent spaces—by proposing a dense KV-cache communication method tailored for heterogeneous agents. The approach leverages lightweight cross-model cache transformation and a two-stage training scheme (reconstruction followed by generation) to jointly transfer both perceptual content and reasoning logic. It reveals, for the first time, a duality in information structure between context-aware and context-agnostic transmission, enabling efficient “mind-reading” communication without requiring shared inputs or homogeneity assumptions. Evaluated across six heterogeneous agent pairs based on the Qwen3 series and six in- and out-of-domain benchmarks, the method matches or surpasses text-based communication while using only half to one-third of the computational cost, and remains effective even when receivers lack contextual input, significantly outperforming existing heterogeneous baselines.

Abstract

Multi-agent systems communicate mostly through text, paying a lossy and expensive decode and re-encode cost. KV-cache communication is a promising alternative, yet most prior work is homogeneous, using duplicate copies of the same model, and avoids the central challenge of cross-model latent alignment; existing heterogeneous methods are also restrictive, typically assuming shared input and using transferred caches mainly for steering. We study a more fundamental question: can heterogeneous agents be aligned well enough to perform real "mind reading" and transfer both what one agent sees and how it thinks? Our information-structure analysis reveals a duality: context-aware transfer is driven by sparse reasoning signals, while context-unaware transfer, where the receiver sees no input, requires dense contextual knowledge preservation. Motivated by this, we propose dense alignment for heterogeneous KV-cache communication via a lightweight cross-model cache transformation and two-phase training: reconstruction followed by generation. Across all six directions of {Qwen3-4B, 8B, 14B} and six in-domain and out-of-domain benchmarks, our method outperforms prior heterogeneous baselines, matches or exceeds text communication in context-aware settings at roughly 2 to 3 times lower compute, and remains effective in context-unaware transfer where prior methods collapse.

Problem

• heterogeneous agents
• latent communication
• KV-cache transfer
• cross-model alignment
• context-unaware transfer

Innovation

• KV-cache communication
• heterogeneous agents
• dense alignment
• cross-model transformation
• latent communication