Equitable System-Prompt Selection via Constrained Mixed-Strategy GroupDRO

2026-08-04

AI Summary

This work addresses the inconsistent response quality of large language models (LLMs) on semantically equivalent but linguistically diverse queries, a problem exacerbated by the neglect of underrepresented groups in existing system prompt optimization methods. To mitigate this, the paper introduces fairness into system prompt selection through a constrained mixture-policy Group Distributionally Robust Optimization (GroupDRO) framework. This approach decouples prompt generation from selection, enables ensemble prompting, and reveals complementary strengths of different prompts across metric–group combinations. By minimizing worst-case losses over predefined question groups while maintaining overall performance near average-optimal levels, the method significantly improves robustness. Experiments across five LLMs and two multilingual benchmarks demonstrate average reductions of 13.1%, 13.2%, and 13.7% in overall mean error, worst 25% mean error, and worst-group loss, respectively, compared to non-mitigated baselines.

Abstract

Large language models are increasingly used for information seeking, yet semantically equivalent questions phrased in different ways can receive answers of considerably different quality. System prompts are widely employed to steer response behavior, but they are typically optimized for average-case quality, so some question phrasings may still receive incomplete or low-quality answers. To address this, we formulate a constrained mixed-strategy GroupDRO framework for system-prompt selection. Instead of optimizing the system-prompt text, the framework assigns weights to system prompts in an existing pool to minimize the worst-case information-quality loss across evaluation metrics and groups, while constraining the mean loss to stay close to that of average-based selection. Because pool generation and selection are decoupled, the method applies to any system-prompt pool and can leverage an ensemble of complementary system prompts rather than a single one. Across five LLMs on two bilingual medical and consumer-finance benchmarks, the constrained method reduces the Overall Mean, Worst 25% Mean, and Worst by 13.1%, 13.2%, and 13.7% on average relative to no mitigation while keeping overall quality close to Average selection. Its multi-prompt weights reveal complementarity across metric-group pairs. Code and data are available at https://github.com/Rainxu09/equitable-system-prompt-selection.

Problem

• equitable prompt selection
• system prompts
• large language models
• quality disparity
• fairness

Innovation

• Constrained GroupDRO
• System Prompt Selection
• Mixed-Strategy Optimization
• Equitable LLM Responses
• Prompt Ensemble