Reasoning on the Simplex: Geometric Fixed-Point Models

2026-09-27

AI Summary

This study addresses the issue wherein residual updates of recurrent reasoners in unconstrained latent spaces vanish without guaranteeing strict fixed points. To resolve this, we propose a geometric fixed-point reasoning framework that defines states as belief fields over simplices, imposes task structure via convex relaxation, and constrains updates to continuous self-mappings, thereby ensuring fixed-point existence through Brouwer’s theorem. The resulting architecture is both parameter-efficient and interpretable. With merely 7M parameters, it achieves 95.1% accuracy on Sudoku and surpasses GPT-2 Small and Medium in language modeling performance.

Abstract

Looped reasoners spend test-time compute by iterating a weight-tied map, but a small residual does not mean the state is a fixed point when that map lives in unconstrained latent space. We propose Geometric Fixed-Point Reasoning (GFPR), in which the iterated state is the prediction itself: a field of categorical beliefs on a product of simplices, whose argmax is the answer at every step. Because the state is a belief, task structure can be imposed through compact convex relaxations, either as structured readouts or directly in the recurrent state; in the latter case the update remains a continuous self-map, so a fixed point exists for any parameters. At about 7M parameters, GFPR reaches 95.1% exact match on Sudoku-Extreme, 92.0% on Maze-Hard, and 100% sequence accuracy on S_5 length 128, above the published FPRM numbers at the same scale. The same update also trains a 201M language model on FineWeb-Edu in which each site is a distribution over the vocabulary; with 24 Picard steps it is above GPT-2 small on four zero-shot multiple-choice tasks and above GPT-2 medium on ARC-Easy.

Problem

• looped reasoners
• fixed point
• unconstrained latent space
• test-time compute

Innovation

• Geometric Fixed-Point Reasoning
• Simplex
• Looped Reasoners
• Convex Relaxations
• Continuous Self-Map