Calibrated Fusion for Heterogeneous Graph-Vector Retrieval in Multi-Hop QA

2026-03-30

AI Summary

This work addresses the challenge of effectively fusing heterogeneous scores—such as vector similarity and graph-based relevance measures like personalized PageRank—in graph-augmented retrieval, where distributional mismatches hinder integration. To resolve this, the authors propose a calibration method based on Percentile Rank (PIT) normalization, which maps disparate scores onto a unified, dimensionless scale while preserving magnitude information and enabling stable alignment. Combined with linear and Boltzmann fusion strategies, the approach significantly improves last-hop retrieval performance in multi-hop question answering. On MuSiQue and 2WikiMultiHopQA benchmarks, it achieves LastHop@5 scores of 76.5% and 53.6%, respectively, substantially outperforming existing baselines.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Distributed Search
• Knowledge Representation and Reasoning: Preferences

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Querying, indexing, and retrieval in Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Graph-augmented retrieval combines dense similarity with graph-based relevance signals such as Personalized PageRank (PPR), but these scores have different distributions and are not directly comparable. We study this as a score calibration problem for heterogeneous retrieval fusion in multi-hop question answering. Our method, PhaseGraph, maps vector and graph scores to a common unit-free scale using percentile-rank normalization (PIT) before fusion, enabling stable combination without discarding magnitude information. Across MuSiQue and 2WikiMultiHopQA, calibrated fusion improves held-out last-hop retrieval on HippoRAG2-style benchmarks: LastHop@5 increases from 75.1% to 76.5% on MuSiQue (8W/1L, p=0.039) and from 51.7% to 53.6% on 2WikiMultiHopQA (11W/2L, p=0.023), both on independent held-out test splits. A theory-driven ablation shows that percentile-based calibration is directionally more robust than min-max normalization on both tune and test splits (1W/6L, p=0.125), while Boltzmann weighting performs comparably to linear fusion after calibration (0W/3L, p=0.25). These results suggest that score commensuration is a robust design choice, and the exact post-calibration operator appears to matter less on these benchmarks.

Problem

• heterogeneous retrieval
• score calibration
• multi-hop QA
• graph-vector fusion
• retrieval fusion

Innovation

• calibrated fusion
• percentile-rank normalization
• heterogeneous retrieval
• multi-hop QA
• graph-augmented retrieval