Recursive Vision Language Models for General Symbolic Reasoning

2026-08-02

AI Summary

This work addresses the limitations of large language models in symbolic reasoning tasks—such as Sudoku, maze solving, and ARC—where fixed-depth autoregressive decoding impedes systematic search and backtracking. To overcome this, the authors propose R-Qwen, the first general-purpose recursive reasoning framework that integrates pre-trained language model priors with a programmatic self-recursion mechanism to enable structured iterative refinement. The framework further incorporates a Hierarchical Supervision Weighting (HSW) strategy to reduce gradient variance and accelerate convergence. Built upon the Qwen backbone and enhanced with deep supervision, R-Qwen outperforms existing recursive models and even larger-scale LLMs across eight challenging benchmarks, achieving a 27.6% improvement over the baseline on ARC-AGI, thereby demonstrating the efficacy of recursive refinement for general symbolic reasoning.

Abstract

Hard symbolic-reasoning tasks such as Sudoku, maze pathfinding, and ARC remain challenging for LLMs due to their fixed-depth autoregressive reasoning, which limits systematic search, refinement, and backtracking. While recursive models such as Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM) address this limitation through iterative latent-state refinement, they are typically task-specific and do not leverage pretrained language priors. We propose R-Qwen, a recursive reasoning framework built upon a pretrained Qwen backbone. R-Qwen repeatedly refines a candidate solution through programmatic self-recursion and deep supervision, combining the structured iterative computation of recursive models with the linguistic and reasoning priors of pretrained LLMs. We further adapt Hierarchical Supervision Weighting (HSW) to autoregressive models by exponentially weighting losses across recursive steps. HSW reduces gradient variance by at least 50\%, improves the signal-to-noise ratio of stochastic gradients, and accelerates convergence. Across eight challenging benchmarks, R-Qwen consistently outperforms prior recursive reasoning models and substantially larger LLMs while using a comparable number of trainable parameters. Notably, on ARC-AGI dataset, our model achieves a 27.6\% improvement over the baseline, highlighting the effectiveness of recursive refinement for general symbolic reasoning. These results suggest that recursive reasoning mechanisms and pretrained language model priors are complementary approaches for improving symbolic puzzle-solving. Code and models will be released after acceptance.

Problem

• symbolic reasoning
• recursive models
• autoregressive reasoning
• systematic search
• backtracking

Innovation

• Recursive Reasoning
• Pretrained Language Models
• Hierarchical Supervision Weighting
• Symbolic Reasoning
• Self-Recursion