Learning to Partially Defer for Sequences

2025-02-03

AI Summary

To address the trade-off between cost and accuracy in long-sequence forecasting, this paper proposes a fine-grained human-in-the-loop decision framework that dynamically identifies high-difficulty tokens within the output sequence for expert intervention—bypassing full-sequence deferral. It introduces, for the first time, a sequence-level partial deferral mechanism, instantiated via two post-hoc rejection classifiers: a token-level rejector and a truncation-based rejector—departing from conventional all-or-nothing deferral paradigms. Built upon pre-trained language models, the method jointly optimizes token-level next-token prediction and sequence-level truncation-aware deferral decisions. Evaluated on Traveling Salesman Problem (TSP) solving and news summarization, the fine-grained deferral strategy achieves superior cost-accuracy trade-offs compared to full-sequence deferral baselines, demonstrating both computational efficiency and predictive fidelity.

Technology Category

Reasoning under Uncertainty: Sequential Decision MakingPlanning, Routing, and Scheduling: Planning with Language ModelsSearch and Optimization: Learning to Search

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingEconomics, Online Markets and Human Computation: Cost models of using LLMs in production systemsSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

In the Learning to Defer (L2D) framework, a prediction model can either make a prediction or defer it to an expert, as determined by a rejector. Current L2D methods train the rejector to decide whether to reject the entire prediction, which is not desirable when the model predicts long sequences. We present an L2D setting for sequence outputs where the system can defer specific outputs of the whole model prediction to an expert in an effort to interleave the expert and machine throughout the prediction. We propose two types of model-based post-hoc rejectors for pre-trained predictors: a token-level rejector, which defers specific token predictions to experts with next token prediction capabilities, and a one-time rejector for experts without such abilities, which defers the remaining sequence from a specific point onward. In the experiments, we also empirically demonstrate that such granular deferrals achieve better cost-accuracy tradeoffs than whole deferrals on Traveling salesman solvers and News summarization models.

Problem

• Long Sequence Prediction
• Expert Systems
• Cost-Accuracy Tradeoff

Innovation

• Learning to Defer
• Enhanced Rejectors
• Cost-Efficient Prediction