Generator-Assistant Stepwise Rollback Framework for Large Language Model Agent

2025-03-04

AI Summary

LLM agents suffer from the “irreversible single-step generation” problem in stepwise reasoning: erroneous reasoning steps, once generated, become fixed in the execution trace, leading to error propagation. To address this, we propose the Generator-Assistant Rollback (GAR) framework—a novel two-role runtime collaboration mechanism wherein a generator produces actions while an assistant continuously monitors and triggers conditional rollback upon detecting inconsistencies. GAR introduces trajectory recalibration and context reconstruction techniques to enable dynamic error identification and precise trace correction. The method is plug-and-play, requiring no modification to the underlying LLM. Evaluated on three mainstream benchmarks, GAR significantly outperforms strong baselines, simultaneously improving task completion rate and reasoning reliability. This work establishes a new paradigm for robust, self-correcting reasoning in LLM-based agents.

Technology Category

• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

Large language model (LLM) agents typically adopt a step-by-step reasoning framework, in which they interleave the processes of thinking and acting to accomplish the given task. However, this paradigm faces a deep-rooted one-pass issue whereby each generated intermediate thought is plugged into the trajectory regardless of its correctness, which can cause irreversible error propagation. To address the issue, this paper proposes a novel framework called Generator-Assistant Stepwise Rollback (GA-Rollback) to induce better decision-making for LLM agents. Particularly, GA-Rollback utilizes a generator to interact with the environment and an assistant to examine each action produced by the generator, where the assistant triggers a rollback operation upon detection of incorrect actions. Moreover, we introduce two additional strategies tailored for the rollback scenario to further improve its effectiveness. Extensive experiments show that GA-Rollback achieves significant improvements over several strong baselines on three widely used benchmarks. Our analysis further reveals that GA-Rollback can function as a robust plug-and-play module, integrating seamlessly with other methods.

Problem

• Addresses irreversible error propagation in LLM agents.
• Proposes GA-Rollback for better decision-making in LLM agents.
• Introduces strategies to improve rollback effectiveness in LLM agents.

Innovation

• Introduces Generator-Assistant Stepwise Rollback framework
• Uses assistant to detect and rollback incorrect actions
• Enhances decision-making with tailored rollback strategies