Exploring Design of Multi-Agent LLM Dialogues for Research Ideation

2025-07-11

AI Summary

Enhancing the novelty and feasibility of scientific ideation through multi-agent large language model (LLM) systems remains challenging due to unstructured dialogue patterns and suboptimal role design. Method: We propose a role-based, iterative multi-agent architecture featuring explicit functional specialization—namely, idea generators, multi-dimensional critics, and integrative revisers—establishing a closed-loop “generate–critique–revise” workflow. We systematically modulate agent count, interaction depth, and role heterogeneity. Contribution/Results: Experiments demonstrate that increasing critic diversity significantly improves proposal feasibility (+23.6% feasibility score), while role heterogeneity and interaction depth jointly enhance idea diversity. This work provides the first quantitative validation of critic-side diversity as a critical lever in multi-agent creative systems. It establishes a reproducible design paradigm and empirically grounded guidelines for LLM-augmented scientific assistance. The implementation is open-sourced.

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Multiagent Systems: Teamwork
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Architectures and workflows that use LLMs for crowd work

Abstract

Large language models (LLMs) are increasingly used to support creative tasks such as research idea generation. While recent work has shown that structured dialogues between LLMs can improve the novelty and feasibility of generated ideas, the optimal design of such interactions remains unclear. In this study, we conduct a comprehensive analysis of multi-agent LLM dialogues for scientific ideation. We compare different configurations of agent roles, number of agents, and dialogue depth to understand how these factors influence the novelty and feasibility of generated ideas. Our experimental setup includes settings where one agent generates ideas and another critiques them, enabling iterative improvement. Our results show that enlarging the agent cohort, deepening the interaction depth, and broadening agent persona heterogeneity each enrich the diversity of generated ideas. Moreover, specifically increasing critic-side diversity within the ideation-critique-revision loop further boosts the feasibility of the final proposals. Our findings offer practical guidelines for building effective multi-agent LLM systems for scientific ideation. Our code is available at https://github.com/g6000/MultiAgent-Research-Ideator.

Problem

• Optimizing multi-agent LLM dialogue design for research ideation
• Evaluating agent roles and interaction depth impact on ideas
• Enhancing idea novelty and feasibility through diverse agents

Innovation

• Multi-agent LLM dialogues enhance idea generation
• Iterative critique-revision loop improves idea feasibility
• Agent role and depth configuration optimize ideation