Set-LLM: A Permutation-Invariant LLM

2025-05-21

AI Summary

Large language models (LLMs) exhibit significant order sensitivity—reordering answer options induces inconsistent decisions, undermining reliability in multiple-choice question answering and automated evaluation of AI-generated outputs. Method: We propose the first approach to enforce strict permutation invariance in pretrained LLMs, theoretically guaranteeing that output distributions remain unchanged under arbitrary input set permutations. Our method comprises three key components: (i) set-specific attention masking, (ii) permutation-invariant positional encoding, and (iii) a lightweight fine-tuning adapter architecture compatible with mainstream LLM backbones and supporting joint set–text modeling. Contribution/Results: This is the first work to introduce strict permutation invariance into general-purpose LLMs. On multiple-choice QA and model-output evaluation benchmarks, our method eliminates ordering bias entirely while maintaining or improving accuracy and incurring zero additional inference latency. It establishes a new paradigm for robust, trustworthy automated assessment.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

While large language models (LLMs) demonstrate impressive capabilities across numerous applications, their robustness remains a critical concern. This paper is motivated by a specific vulnerability: the order sensitivity of LLMs. This vulnerability manifests itself as the order bias observed when LLMs decide between possible options (for example, a preference for the first option) and the tendency of LLMs to provide different answers when options are reordered. The use cases for this scenario extend beyond the classical case of multiple-choice question answering to the use of LLMs as automated evaluators in AI pipelines, comparing output generated by different models. We introduce Set-LLM, a novel architectural adaptation for pretrained LLMs that enables the processing of mixed set-text inputs with permutation invariance guarantees. The adaptations involve a new attention mask and new positional encodings specifically designed for sets. We provide a theoretical proof of invariance and demonstrate through experiments that Set-LLM can be trained effectively, achieving comparable or improved performance and maintaining the runtime of the original model, while eliminating order sensitivity.

Problem

• LLMs exhibit order sensitivity in decision-making
• Order bias affects multiple-choice and evaluation tasks
• Set-LLM ensures permutation invariance in set-text processing

Innovation

• Novel attention mask for set-text inputs
• New positional encodings for permutation invariance
• Theoretical proof and experimental validation