How Reasoning Shapes Social Bias in LLM-Generated Code?

2026-08-07

AI Summary

This study investigates how large language models introduce and propagate social biases during reasoning-driven code generation, particularly in human decision-making scenarios involving sensitive demographic attributes. Addressing the limitation of existing debiasing approaches that focus solely on model outputs, this work proposes ProbeDebias—a novel framework that detects and corrects bias trajectories during the reasoning process itself. Through systematic evaluation across nine mainstream models on three real-world tasks, combined with automated metrics for both bias and code quality alongside reasoning trajectory analysis, experiments demonstrate that ProbeDebias achieves an 87.76% F1 score in reasoning bias detection, reduces code-level bias by 83.73% on average, and further decreases bias by 52.70%–54.42% compared to current state-of-the-art methods, while simultaneously improving code quality by 9.79%–36.79%.

Abstract

Large language models (LLMs) are increasingly used for code generation, yet generated programs may exhibit social bias through unfair or differential treatment of sensitive demographic attributes. While prior work mainly studies direct code generation, bias in reasoning-based generation remains underexplored. We conduct the first systematic study of social bias in reasoning-based code generation, evaluating 9 standard LLMs and large reasoning models (LRMs) on realistic bias-sensitive tasks across three human-centered decision scenarios. We find that reasoning generally reduces bias, lowering the average bias rate from 0.64 to 0.40, but the effect varies substantially across models. Meanwhile, code quality is not consistently preserved, with the average quality dropping from 0.72 to 0.59. Biased reasoning strongly predicts biased code, and adjusting generation configurations alone is insufficient for robust mitigation. Based on these findings, we propose ProbeDebias, a reasoning-aware framework that detects and rewrites biased reasoning traces before code generation. ProbeDebias achieves 87.76% F1 for reasoning-bias detection and reduces code bias by 83.73% on average while largely preserving quality. Compared with SOTA baselines, it further reduces average bias by 52.70%-54.42% and improves quality by 9.79%-36.79%. These results highlight the value of reasoning-stage analysis for trustworthy code generation.

Problem

• social bias
• reasoning-based code generation
• large language models
• biased reasoning
• code generation

Innovation

• reasoning-based code generation
• social bias mitigation
• ProbeDebias
• reasoning trace analysis
• trustworthy AI