Frustratingly Simple Black-Box Adaptation of Language Models via Logit Bias

2026-07-24

AI Summary

This work proposes a lightweight language model adaptation method that requires neither fine-tuning nor complex prompt engineering. By introducing user-defined, context-independent logit bias vectors during decoding, the approach enhances model performance on domain-specific tasks through API-level intervention while preserving data privacy. It formalizes fixed logit bias as an approximation to optimal prefix correction and integrates a KL-regularized reinforcement learning objective with a closed-form inverse propensity estimator based on trajectories, rewards, and token probabilities to enable efficient black-box adaptation. Experimental results demonstrate that the method significantly outperforms baseline approaches on mathematical and reasoning benchmarks, achieving strong performance with substantially fewer trainable parameters than conventional fine-tuning, thereby validating logit bias as a highly effective minimalistic adaptation mechanism.

Abstract

Many organizations aim to adapt language models for internal use, both to improve performance on domain-specific tasks and to address privacy concerns around sensitive data. However, such adaptation remains non-trivial: it often requires operationally challenging fine-tuning of open-source models or ad hoc prompt optimization. We study a minimal alternative based on a simple API-level control: allowing users to bias the model's logits with a user-defined vector. We develop a black-box method for learning a single context-independent logit-bias vector, added at every decoding step, without modifying model weights or requiring gradients. Starting from a KL-regularized reinforcement learning (RL) objective, we characterize when such a fixed logit-bias vector can approximate the optimal prefix-dependent correction and derive a closed-form inverse-propensity estimator from rollouts, rewards, and token probabilities. Empirically, this simple decoding-time intervention improves over base models on mathematical and reasoning benchmarks while using far fewer trainable parameters than conventional fine-tuning. Our results suggest that learned logit bias is a lightweight mechanism for adapting language models under minimal access requirements.

Problem

• language model adaptation
• logit bias
• black-box adaptation
• domain-specific tasks
• privacy

Innovation

• logit bias
• black-box adaptation
• reinforcement learning
• decoding-time intervention
• parameter-efficient adaptation