LSTM-MAS: A Long Short-Term Memory Inspired Multi-Agent System for Long-Context Understanding

2026-01-17

AI Summary

This work addresses the challenges faced by large language models in processing long-context inputs, including high computational costs, length limitations, error accumulation, and hallucination propagation. The authors propose LSTM-MAS, a novel approach that, for the first time, maps the gating mechanisms of Long Short-Term Memory (LSTM) networks onto a multi-agent system. This framework comprises a chain of four specialized agents—Worker, Filter, Judge, and Manager—that emulate the input gate, forget gate, constant error carousel, and output gate of LSTMs, enabling segmented comprehension and controlled information flow. Through hierarchical information regulation, redundancy filtering, and continuous error detection, LSTM-MAS substantially mitigates hallucinations and error propagation. Experimental results demonstrate significant performance improvements over the prior multi-agent method CoA, with gains of 40.93%, 43.70%, 121.57%, and 33.12% on NarrativeQA, Qasper, HotpotQA, and MuSiQue benchmarks, respectively.

Technology Category

• Multiagent Systems: Multiagent Learning
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Effectively processing long contexts remains a fundamental yet unsolved challenge for large language models (LLMs). Existing single-LLM-based methods primarily reduce the context window or optimize the attention mechanism, but they often encounter additional computational costs or constrained expanded context length. While multi-agent-based frameworks can mitigate these limitations, they remain susceptible to the accumulation of errors and the propagation of hallucinations. In this work, we draw inspiration from the Long Short-Term Memory (LSTM) architecture to design a Multi-Agent System called LSTM-MAS, emulating LSTM's hierarchical information flow and gated memory mechanisms for long-context understanding. Specifically, LSTM-MAS organizes agents in a chained architecture, where each node comprises a worker agent for segment-level comprehension, a filter agent for redundancy reduction, a judge agent for continuous error detection, and a manager agent for globally regulates information propagation and retention, analogous to LSTM and its input gate, forget gate, constant error carousel unit, and output gate. These novel designs enable controlled information transfer and selective long-term dependency modeling across textual segments, which can effectively avoid error accumulation and hallucination propagation. We conducted an extensive evaluation of our method. Compared with the previous best multi-agent approach, CoA, our model achieves improvements of 40.93%, 43.70%,121.57% and 33.12%, on NarrativeQA, Qasper, HotpotQA, and MuSiQue, respectively.

Problem

• long-context understanding
• large language models
• multi-agent systems
• error accumulation
• hallucination propagation

Innovation

• LSTM-inspired
• multi-agent system
• long-context understanding
• gated memory mechanism
• error hallucination mitigation