SynH-Rank: Quality-Aware Code Search via Diverse Data Synthesis and Hierarchical Ranking Training

2026-07-19

AI Summary

Existing code search systems often neglect non-functional code quality attributes—such as execution efficiency, memory usage, and maintainability—during reranking, failing to meet developers’ demands for high-quality code. This work proposes a novel reranking framework that jointly optimizes semantic relevance and code quality by introducing a three-tier quality labeling scheme and a hierarchical ranking objective. Diverse training data are synthesized using large language models, and a hierarchical ranking loss integrating contrastive learning with quality annotations is designed. The study also establishes the first quality-aware evaluation benchmark for code search, along with two new metrics: Quality-Preserving Accuracy (QPA) and Multi-Condition Accuracy (MCA). Experiments demonstrate that the proposed method improves QPA by 20.15% and outperforms relevance-only baselines by 15.80%, while achieving significant gains in traditional relevance metrics and multi-condition generalization capability.

Abstract

Code search enhances developer productivity by enabling efficient code reuse. Current code search systems often use a retrieve-then-rerank pipeline, where rerankers focus on modeling semantic relevance between queries and code. However, these rerankers overlook critical non-functional qualities like execution speed, memory usage, and maintainability, which are essential for practical software development. Studies reveal developers expect results to maintain high coding standards and satisfy specific needs, such as resource optimization, highlighting the importance of quality-aware code search. Achieving quality-aware code search faces two major challenges: the scarcity of quality-annotated datasets for effective training and the limitations of standard contrastive learning objectives, which fail to capture the ordinal relationships among high-quality, low-quality, and irrelevant code. Although contrastive learning excels in distinguishing relevant from irrelevant code, its binary objective does not support nuanced quality distinctions.To address these challenges, we propose SynH-Rank, a quality-aware code reranking framework that combines LLM-driven diverse data synthesis with hierarchical ranking training. SynH-Rank employs a three-level labeling scheme to explicitly model the hierarchy: high-quality relevant > low-quality relevant > irrelevant. Additionally, we introduce a new benchmark with 4,209 pairs and two novel metrics: Quality Preference Accuracy (QPA) for assessing prioritization of high-quality code and Multi-Condition Accuracy (MCA) for evaluating performance under complex constraints.Experimental results show SynH-Rank improves QPA by 20.15\% over backbone models and outperforms standard relevance-only contrastive training by 15.80\%, while simultaneously enhancing traditional relevance metrics and multi-condition generalizability.

Problem

• code search
• code quality
• non-functional properties
• quality-aware ranking
• developer productivity

Innovation

• quality-aware code search
• hierarchical ranking
• data synthesis
• contrastive learning
• code reranking