Diversity Combining for Multi-Path LLM Reasoning

2026-09-29

AI Summary

This study addresses the rapid saturation of performance gains in multi-path large language model (LLM) reasoning as the number of sampled paths increases, compounded by the absence of mechanisms to predict computational overhead. By formulating multi-path reasoning as a communication diversity combining problem, this work reveals how inter-path correlation constrains the effective sample size. It further introduces generalized least squares to establish the optimality of majority voting and proposes an adaptive path-selection rule based on a four-path pilot scheme. Additionally, leveraging prompt template diversity significantly reduces inter-path correlation. The proposed approach preserves the accuracy achieved with 32 sampling paths while substantially decreasing computational costs, thereby providing both a theoretical framework and a practical solution for efficient LLM inference.

Abstract

Multi-path reasoning methods such as self-consistency (SC) sample $K$ reasoning paths and choose the most frequent answer. However, their gains quickly plateau as $K$ increases, and existing methods do not predict when this saturation will occur. We formalize multi-path LLM reasoning as a diversity combining problem from wireless communications: each path is a noisy channel observation, and the pairwise correlation of path correctness caps the design-effect effective sample size of the vote at a finite ceiling. Generalized least squares (GLS) analysis shows that, under exchangeability, the optimal symmetric linear combiner of latent embeddings is uniform, supporting majority vote as the natural default in standard SC while leaving room for weighting or pruning under heterogeneous prompt-template branches. Across 5 models and 12 benchmarks, prompt-template diversity reduces path correlation in $55$ of $57$ valid cells, with the strongest effect on open-ended QA. We derive an Adaptive-K rule that uses a four-path pilot to select $K^*$, retaining $96$--$103\%$ of MV@$K{=}32$ accuracy across Math, QA, and NLU.

Problem

• Multi-path reasoning
• Diversity combining
• Self-consistency
• Performance saturation
• Path correlation

Innovation

• Diversity Combining
• Multi-path Reasoning
• Adaptive-K Rule
• Generalized Least Squares
• Prompt-template Diversity