A Tilted Bowl Is Not a Slippery Slope: Compressing Looped Models

2026-09-30

AI Summary

This study addresses the performance collapse in recurrent model compression caused by the accumulation of misjudged rounding errors. To this end, it proposes the "tilted bowl" theory, which reveals that fixed rounding errors within convergent loops merely shift the stopping point rather than accumulating progressively. Building upon this theoretical insight, the work integrates quantization-aware training with 8-bit mixed-precision inference to construct a dynamic depth controller based on unlabeled measurements, enabling effective error failure prediction and recovery. Evaluated on Sudoku and Maze tasks, the proposed method surpasses fixed-depth inference baselines by up to 15 points while requiring lower weight traffic, thereby significantly enhancing the inference efficiency of recurrent neural networks.

Abstract

Looped models reason by applying the same block of weights many times, so compressing that block saves memory traffic on every loop. Compressed looped models, however, often collapse, and the collapse is usually blamed on rounding error that accumulates from loop to loop. In this work we test that account on more than 30 models from five families and find, to our surprise, that it holds only for loops that never settle. When a loop settles, a fixed rounding error does not accumulate. It moves the point where the loop settles, much as tilting a bowl moves where a ball comes to rest, and the answer is lost only when the shift is larger than the readout tolerates. This picture lets us predict which models fail from a single label-free measurement, and it tells us why failed models recover: their loops still settle, so a few final loops with 8-bit weights bring the answer back. Motivated by these findings, we build a controller that stops when the model's halting head fires and then finishes with 8-bit loops. On Sudoku-Extreme and Maze-Hard it beats fixed-depth inference by up to 15 points under a third of the weight traffic.

Problem

• Looped models
• Model compression
• Rounding error
• Performance collapse

Innovation

• Looped models
• Model compression
• Rounding error
• Halting head
• Fixed-point convergence