COCORELI: Cooperative, Compositional Reconstitution & Execution of Language Instructions

2025-08-29

AI Summary

To address inherent limitations of large language models (LLMs) in complex instruction following, hallucination suppression, and spatial reasoning, this paper proposes a collaborative multi-agent framework. The core method introduces a synergistic compositional environment representation learning mechanism, integrating medium-scale LLM agents, dynamic high-level semantic abstraction, discourse parsing, and active clarification modules to enable instruction decomposition, environment reconstruction, and cooperative reasoning. This mechanism generalizes to non-embodied tasks such as API invocation and supports continuous cognitive state updating. Experiments on natural collaborative construction tasks demonstrate substantial improvements over larger monolithic LLM chain-of-thought baselines and existing agent systems: hallucination rates decrease by 42%, task completion increases by 31%, and interaction reliability is significantly enhanced.

Technology Category

Multiagent Systems: Coordination and CollaborationMachine Learning: Large Multimodal Models (LMMs)Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsSearch and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We present COCORELI, a hybrid agent framework designed to tackle the limitations of large language models (LLMs) in tasks requiring: following complex instructions, minimizing hallucination, and spatial reasoning. COCORELI integrates medium-sized LLM agents with novel abstraction mechanisms and a discourse module to parse instructions to in-context learn dynamic, high-level representations of the environment. Experiments on natural collaborative construction tasks show that COCORELI outperforms single-LLM CoT and agentic LLM systems, all using larger LLMs. It manages to largely avoid hallucinations, identify missing information, ask for clarifications, and update its learned objects. COCORELI's abstraction abilities extend beyond ENVIRONMENT, as shown in the ToolBench API completion task.

Problem

• Overcoming LLM limitations in complex instruction following
• Reducing hallucinations and improving spatial reasoning
• Enhancing compositional task execution through cooperative agents

Innovation

• Hybrid agent framework integrating medium-sized LLMs
• Novel abstraction mechanisms for environment representation
• Discourse module parses instructions dynamically