Self-Evaluation Is Already There: Eliciting Latent Judge Calibration in Base LLMs with Minimal Data

2026-06-03

AI Summary

This work addresses the challenge of enabling large language models to accurately predict external evaluators’ scores for their generated content in data-scarce settings. The authors propose Self-Evaluation Elicitation (SEE), a method that leverages only 160 labeled samples—approximately 1/31 of the baseline data size—to elicit the model’s intrinsic self-evaluation capability. SEE integrates calibrated, coupled reinforcement learning with masked knowledge distillation in a two-stage training framework, significantly improving prediction calibration without compromising generation quality. Requiring minimal training data, SEE demonstrates high efficiency, strong transferability, and robust generalization across diverse evaluators, supporting multi-attribute quality score prediction.

Abstract

Large language models are increasingly evaluated by other models, raising a natural question: can a model predict how a judge will score its own output? We find that the ability is largely present before any targeted training: prompted few-shot, a base model already predicts an external judge's multi-attribute quality scores on open-ended responses well above chance across three benchmarks. We introduce Self-Evaluation Elicitation (SEE), a method that surfaces this latent ability through a short cycle comprising a calibration-coupled reinforcement learning phase that improves the answer and predicts the judge, followed by a masked distillation phase that sharpens the prediction while leaving the answer untouched. From 160 unique examples, roughly 31x fewer than a reinforcement learning baseline, SEE improves held-out calibration across three benchmarks while preserving answer quality. The elicited self-evaluation is sharply localized within the model's own token distribution and stable across judges it was never trained against, indicating a transferable notion of quality rather than a single judge's preference. These results reframe judge-aligned self-evaluation as a problem of elicitation rather than acquisition.

Problem

• self-evaluation
• judge calibration
• large language models
• quality prediction
• latent ability

Innovation

• Self-Evaluation Elicitation
• Judge Calibration
• Latent Ability
• Masked Distillation
• Reinforcement Learning