Post-Training Recipe, More Than Model Family, Shapes Multi-Agent LLM Conversational Behavior

Unknown Date

AI Summary

This study investigates the sources of behavioral diversity in multi-agent large language model (LLM) systems, challenging the common assumption that model family labels adequately capture such variation. The authors construct a large-scale multi-agent dialogue corpus comprising 940,000 conversation chains across 11 checkpoints and conduct a factorial experiment with 1.6 million chains using Llama-based models. Through closed-API validation, runtime detection, and surface cue analysis, they empirically demonstrate—for the first time—that post-training recipes exert a significantly stronger influence on behavioral differences than model family affiliation. For instance, evasiveness varies by up to 18% between models sharing the same base architecture but differing in post-training, exceeding cross-family discrepancies. These findings contest the use of model family as a proxy for diversity and advocate prioritizing post-training strategies as a core dimension in designing multi-LLM systems.

Abstract

Multi-LLM systems use multiple language models to deliberate, judge each other's outputs, or coordinate as agents. Their value depends on the models producing measurably different conversational behaviors when given the same input. Prior offline studies recommend drawing one model per family for behavioral diversity, because LLMs prefer outputs from their own family when rating one another in isolation. Whether the same family label predicts behavior in interactive multi-LLM systems, the setting that real deployed systems use, has not been tested. We study this with a 940,000-chain 11-checkpoint corpus and a 1.6M-chain same-base Llama factorial. On our validated headline metric, hedging, a reasoning-distilled Llama checkpoint shifts by 18% depending on which same-base partner it replies to, more than any cross-family hedging gap in the controlled subset. Qwen, closed-API, and runtime checks suggest the pattern is not isolated, while repair and challenge analyses remain exploratory because their surface-cue detectors are weaker. Overall, the results identify post-training recipe as a first-class axis for multi-LLM panel composition and show that model family alone is an incomplete proxy for conversational diversity.

Problem

• multi-agent LLM
• conversational behavior
• behavioral diversity
• model family
• post-training recipe

Innovation

• post-training recipe
• multi-agent LLM
• conversational diversity
• behavioral variation
• model family