FACTER: Fairness-Aware Conformal Thresholding and Prompt Engineering for Enabling Fair LLM-Based Recommender Systems

2025-02-05

AI Summary

This paper addresses pervasive demographic bias in large language model (LLM)-based recommender systems, proposing a dynamic fairness regulation framework that requires no model retraining. Methodologically, it integrates conformal prediction with dynamic prompt engineering to introduce a novel violation-driven fairness constraint mechanism: a semantic variance-based proxy metric quantifies bias; adaptive thresholds and violation-triggered mechanisms dynamically tighten fairness constraints in real time; and an adversarial prompt generator actively suppresses biased outputs by leveraging historical bias patterns. Experiments on MovieLens and Amazon datasets demonstrate a 95.5% reduction in fairness violations while preserving high recommendation accuracy, empirically validating semantic variance as a robust bias proxy.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: Ethics — Bias, Fairness, Transparency & PrivacyComputer Vision: Bias, Fairness & Privacy

Application Category

User Modeling, Personalization and Recommendation: Fairness-aware retrieval and rankingSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We propose FACTER, a fairness-aware framework for LLM-based recommendation systems that integrates conformal prediction with dynamic prompt engineering. By introducing an adaptive semantic variance threshold and a violation-triggered mechanism, FACTER automatically tightens fairness constraints whenever biased patterns emerge. We further develop an adversarial prompt generator that leverages historical violations to reduce repeated demographic biases without retraining the LLM. Empirical results on MovieLens and Amazon show that FACTER substantially reduces fairness violations (up to 95.5%) while maintaining strong recommendation accuracy, revealing semantic variance as a potent proxy of bias.

Problem

• Ensures fairness in LLM-based recommendations
• Reduces demographic biases without retraining
• Maintains accuracy while improving fairness

Innovation

• Fairness-aware conformal prediction
• Dynamic prompt engineering
• Adversarial prompt generator