TRAM: Enhancing Multimodal Reasoning with Trajectory-Derived Auxiliary Memory

2026-08-03

AI Summary

This work addresses the challenge that multimodal large reasoning models often fail to effectively leverage early-generated relations, constraints, and intermediate conclusions during long-horizon reasoning, leading to error accumulation. To mitigate this, the authors propose TRAM, a novel method that explicitly models semantic information derived from the reasoning trajectory as auxiliary memory. TRAM employs a fast-slow recurrent mechanism to online-update a compact implicit memory representation and injects it into designated decoder layers via a lightweight residual pathway, enabling efficient integration of cross-stage reasoning information. Notably, TRAM requires no additional training and consistently enhances performance across four prominent multimodal large language models and eight benchmarks, significantly improving accuracy and consistency on mathematical, scientific, and general visual reasoning tasks.

Abstract

Multimodal Large Reasoning Models (MLRMs) have achieved strong performance on tasks requiring visual understanding and multi-step inference. However, as reasoning trajectories grow, models may become less effective at using information established earlier in the context, increasing the risk of reasoning errors. Existing approaches primarily address this problem by sustaining visual grounding throughout reasoning. However, reasoning also transforms visual observations into task-specific relations, constraints, and intermediate conclusions whose influence may weaken over long trajectories. Our attribution analysis suggests that correctness is not consistently separated by image attribution alone, but is more closely associated with whether trajectories retain and integrate such reasoning-derived information across stages. Motivated by this, we introduce TRAM (TRajectory-derived Auxiliary Memory), a training-free method that augments standard decoding with an auxiliary memory pathway derived from the model's own reasoning trajectory. TRAM consolidates completed reasoning into a compact latent memory, updates it online through fast and slow recurrent streams, and feeds it back into selected decoder layers through a lightweight residual pathway. Experiments across four MLRM variants on eight benchmarks show that TRAM improves performance over vanilla decoding on mathematical, scientific, and general visual reasoning tasks without additional training.

Problem

• multimodal reasoning
• reasoning trajectory
• information retention
• visual reasoning
• reasoning errors

Innovation

• trajectory-derived memory
• multimodal reasoning
• auxiliary memory
• training-free enhancement
• reasoning consistency