How the Audit Rule Shapes Faithful Factor Explanations in LLMs

2026-10-01

AI Summary

This study addresses the limitation of report-dependent auditing under finite budgets, which incentivizes large language models (LLMs) to suppress truthful explanations to evade penalties. Grounded in a verification game-theoretic framework and leveraging counterfactual Brier scores alongside synthetic rational agent simulations, this work systematically characterizes such suppression mechanisms. To mitigate this issue, it proposes a hybrid auditing rule incorporating a report-independent auditing component. Both theoretical analysis and empirical results consistently demonstrate that this hybrid mechanism effectively eliminates the model's incentives for concealment, thereby encouraging faithful reporting of factor influences. Ultimately, this research provides a rigorous theoretical foundation and a practical solution for safeguarding the faithfulness of LLM explanations.

Abstract

Large language models are often asked which input factors influenced their outputs. For structured inputs, such reports can be checked by counterfactual perturbation, but each factor must be queried multiple times to estimate its effect, so verification is usually budget-limited. We study how this limited-budget setting changes the incentive to report factor-level influence truthfully. We formalize the interaction as a verification game and show that proper scoring alone is not enough when auditing depends on the report: report-dependent auditing creates a suppression incentive, because factors reported as important are more likely to be checked and penalized for estimation noise. In contrast, report-independent auditing, or a mixed rule with a small report-independent floor, removes this channel and makes truthful reporting preferable to full suppression. We instantiate the framework with the Counterfactual Brier Score (CBS) and evaluate its predictions on four NLP benchmarks. A synthetic rational agent matches the theoretical prediction exactly, and real LLMs follow the same incentives when they are made explicit. The main design implication is simple: under partial verification, factor-level explanation systems should include a report-independent audit component so that under-reporting cannot be used to avoid scrutiny.

Problem

• Large Language Models
• Faithful Explanations
• Verification Game
• Audit Rule
• Factor-level Influence

Innovation

• Verification Game
• Counterfactual Brier Score
• Report-independent Auditing
• Faithful Explanations
• Mechanism Design