Algorithmic Recourse of In-Context Learning for Tabular Data

2026-05-29

AI Summary

This work addresses the lack of actionable feedback in in-context learning (ICL)-based tabular prediction models for high-stakes decision-making by proposing the first algorithmic recourse framework tailored to ICL tabular tasks. The framework introduces Adaptive Subspace Recourse for ICL (ASR-ICL), a training-free approach that leverages zeroth-order optimization to generate recourses for black-box ICL models in multi-class settings. Theoretical analysis establishes the existence and boundedness of the recourse solutions and demonstrates their convergence to classical recourse solutions as the context size increases. Empirical evaluations on multiple real-world datasets show that ASR-ICL achieves recourse quality comparable to existing methods while requiring significantly fewer model queries, thereby validating both its practical efficacy and theoretical convergence guarantees.

Abstract

As predictive models are increasingly deployed in high-stakes settings such as credit approval, there is a growing need for post-hoc methods that provide recourse to affected individuals. Many such models operate on tabular data, where features correspond to real-world attributes. Recently, in-context learning (ICL) has enabled large language models to perform tabular prediction by conditioning on labeled examples at inference time, without explicit training. However, algorithmic recourse for tabular decision-making under ICL remains largely unexplored. In this work, we present the first study of algorithmic recourse for tabular data under ICL. We carry out a theoretical analysis, showing that recourse remains well-defined and bounded, and we characterize how recourse converges toward classical solutions as the context size increases. In practice, we propose a novel zeroth-order recourse framework, Adaptive Subspace Recourse for In-Context Learning (ASR-ICL), that efficiently generates actionable and sparse recourse for black-box ICL models. The proposed framework naturally extends to multi-class tabular tasks. Experiments across multiple real-world datasets and models demonstrate that ASR-ICL achieves recourse quality comparable to existing methods with fewer queries and empirically confirm the predicted convergence behavior, supporting our theoretical analysis.

Problem

• algorithmic recourse
• in-context learning
• tabular data
• post-hoc explanation
• actionable recommendations

Innovation

• in-context learning
• algorithmic recourse
• tabular data
• zeroth-order optimization
• adaptive subspace