🤖 AI Summary
This work addresses the unobservability of representations arising from partial observations in representation learning by introducing the Plato Projection Structure (PPS). PPS provides the first unified characterization of equivalence classes of indistinguishable latent states through the quotient geometry induced by self-adjoint, positive semi-definite observation operators. The framework reveals a fundamental limitation: outputs cannot fully capture latent representations, thereby establishing the theoretical foundation for geometric preservation mechanisms in knowledge distillation and representation transfer. It also highlights intrinsic limitations of current interpretability methods. Combining kernel invariance analysis with rank-controlling techniques, experiments validate kernel-invariant observability, attribution gaps induced by projection, and rank-controllable observable geometry, offering rigorous theoretical support for representation accessibility and interpretability.
📝 Abstract
We characterize observability in representation learning through Platonic Projection Structures (PPS), an operator-theoretic framework for analyzing representation accessibility under partial observation. Rather than treating observable outputs as direct reflections of latent representations, PPS models observation through a self-adjoint positive semidefinite operator acting on a latent representation space. A system is represented as a triple $(H, Π, O)$, where $H$ is a latent representation space, $Π\succeq 0$ is an observation operator, and $O(v)=\langle v,Πv\rangle$ defines an induced scalar observable. Observability is characterized by the quotient geometry $H/\ker(Π)$, representing equivalence classes of latent states indistinguishable under observation. We show that quantum measurement and representation inference under linear observation models share this operator-theoretic structure while differing in the algebraic properties of their observation operators; the correspondence is structural rather than physical. Representation transfer and knowledge distillation can likewise be interpreted as approximate preservation of observable geometry through $ΦΠ_T \approx Π_S Φ$. PPS also reveals a structural limitation of output-based interpretability: latent components in $\ker(Π)$ are inaccessible from induced observables, imposing intrinsic constraints on attribution and explanation methods. Controlled empirical validations demonstrate kernel-invariant observability, projection-induced attribution gaps, and rank-controlled observable geometry in latent representation spaces. PPS thus provides an explicit characterization of observability through operator-induced quotient geometry and a unified perspective on representation accessibility, interpretability, and projection-mediated inference.