Beyond "What to Retrieve": Uncertainty in Retrieval-Augmented Code Generation

2026-07-27

AI Summary

This work addresses the limitation of existing retrieval-augmented code generation methods, which fail to explicitly model uncertainty arising from heterogeneous information sources, thereby struggling with noise, redundancy, and conflicting evidence. To overcome this, we propose OpenCoder, a novel framework that, for the first time, treats uncertainty as a controllable signal in repository-scale code generation. OpenCoder dynamically filters and ranks multi-source evidence through uncertainty-aware retrieval, factor analysis, and goal-aware API refinement, guiding large language models in generation, verification, and repair. Experimental results demonstrate that our approach significantly improves the accuracy of a GPT-based backend on 32 RepoExec-inline tasks, increasing correctness from 56.25% to 78.13%, while also revealing strong, non-additive interactions among multi-source evidence.

Abstract

Repository-level code generation relies on heterogeneous evidence whose relevance, compatibility, and completeness are inherently uncertain. Similar-code examples, repository context, and project-specific APIs may provide complementary information, but can also introduce noisy, redundant, or conflicting signals. Existing retrieval-augmented approaches primarily optimize retrieval relevance without explicitly modeling how uncertainty in retrieved evidence affects downstream generation. We introduce OpenCoder, an uncertainty-aware framework that estimates source-specific uncertainty, uses it to filter and rank heterogeneous evidence, and guides generation, verification, and repair. A factorial analysis over API knowledge, repository context, and similar-code evidence reveals no universal additive source ranking; instead, significant cross-source interactions depend on the accompanying evidence and LLM backend. On an expanded 32-task RepoExec-inline evaluation, OpenCoder improves GPT selected-output correctness over Baseline RAG from 56.25\% to 78.13\%. However, it matches a verification-and-repair control, and the corresponding Gemini improvement is not statistically supported, indicating backend-dependent benefits. Target-aware API refinement also substantially improves API-set retrieval. These findings support treating uncertainty as an actionable control signal for repository-level retrieval, verification, and repair.

Problem

• retrieval-augmented code generation
• uncertainty
• heterogeneous evidence
• repository-level code generation
• evidence relevance

Innovation

• uncertainty-aware retrieval
• retrieval-augmented code generation
• heterogeneous evidence
• verification and repair
• repository-level code generation