Can Computation from Earlier Problems Help LLMs Solve New Ones?

2026-09-30

AI Summary

This study addresses how computational histories from earlier queries interfere with subsequent reasoning accuracy in multi-turn dialogues with large language models. By analyzing internal state evolution through a controlled replay mechanism, this work proposes STAIR, which introduces a stale token attention mechanism to enable cross-query reuse. Built upon Qwen-based architectures with fixed key-value cache repositories, the method fine-tunes only a minimal set of parameters to preserve historical information while optimizing reasoning capabilities. Evaluated across four benchmarks, STAIR improves average accuracy in subsequent turns by up to 11.67 percentage points over baseline models, demonstrating the effectiveness of lightweight, parameter-efficient fine-tuning for multi-turn context management.

Abstract

Large language models often solve independent problems in the same conversation. Can computation from earlier problems help them solve new ones? To answer this question, we first conduct preliminary experiments showing that retained history can raise or lower later-turn accuracy, even within the same domain. To understand these effects, we use controlled replay to isolate internal state changes specific to each problem-history pairing. Across different histories, these changes preserve similar relationships among current problems. To improve reasoning under retained history, we introduce STAIR (Stale-Token Attention for Inter-query Reuse). STAIR captures keys and values from earlier response generation in a fixed bank. It learns to redirect current queries when they read this bank during prompt processing. The base model remains frozen; only 12,288 parameters are trained. Across three Qwen models and four benchmarks, STAIR improves average later-turn accuracy by up to 11.67 percentage points over the unmodified model with history.

Problem

• Large Language Models
• Multi-turn Reasoning
• Context Retention
• Inter-query Reuse

Innovation

• STAIR
• Inter-query Reuse
• Key-Value Bank
• Parameter Efficiency
• Context History