Random Recursive Models

2026-09-30

AI Summary

Traditional recurrent models are constrained by fixed layer hierarchies, limiting their computational flexibility. This work proposes the Random Recurrent Model (RRM), which breaks these fixed-depth constraints through independent sampling with replacement from a shared parameter pool, thereby enabling dynamic depth reasoning and test-time scaling. Furthermore, RRM incorporates Monte Carlo inference to enhance predictive robustness. Experimental results demonstrate that RRM outperforms existing baselines across multiple reasoning tasks while reducing parameter counts by 50%–75%. Notably, the model achieves continuous performance improvements at inference time simply by increasing the number of sampling steps, without requiring retraining.

Abstract

Recursive models create computational depth through parameter reuse, offering a parameter-efficient alternative to increasing model size. However, most recursive models repeatedly apply one learned transformation or a prescribed sequence of transformations, restricting computation to a fixed layer order. We introduce the Random Recursive Model (RRM), which maintains a pool of $L$ learned layers and performs $T$ recursive steps by sampling one layer independently with replacement for each example and step. This enables flexible layer reuse while retaining the parameter efficiency of recurrence. We evaluate RRM on challenging reasoning tasks, where it matches or exceeds the baselines, often with 50-75 % fewer parameters. RRM can vary its depth at inference, including beyond that seen during training, without retraining or adding parameters, improving tasks that benefit from deeper iterative computation. RRM also supports Monte Carlo inference and probabilistic test-time scaling, both of which improve performance without retraining. These insights may open new directions in neural network architecture design.

Problem

• Recursive Models
• Parameter Efficiency
• Computational Depth
• Reasoning Tasks
• Neural Network Architecture

Innovation

• Random Recursive Model
• Parameter Efficiency
• Dynamic Inference Depth
• Monte Carlo Inference
• Test-Time Scaling