When Evidence Conflicts: Reliability-aware Meta-review Generation
本文针对同行评审中证据冲突和可靠性差异问题，提出一种基于证据可靠性聚合的元评审生成方法，有效识别并解决冲突，提升评审质量。
0 citationsRead paper
Selected work
Representative Papers
Contextual Relevance and Adaptive Sampling for LLM-Based Document Reranking
For complex queries involving multifaceted requirements and nuanced semantics, document relevance becomes highly context-dependent, rendering conventional re-ranking methods inadequate. To address this, we propose a “contextual relevance” modeling framework that formalizes relevance as the joint probability distribution over candidate document sets, revealing the substantial impact of document composition and ordering on large language model (LLM) relevance judgments. We introduce TS-SetRank—a novel algorithm integrating Bayesian uncertainty estimation with Thompson sampling—to enable uncertainty-aware, adaptive set-level sampling and re-ranking. Evaluated on BRIGHT and BEIR benchmarks, TS-SetRank achieves 15–25% and 6–21% improvements in nDCG@10, respectively, outperforming state-of-the-art retrieval and re-ranking approaches. Our core contributions are: (i) the first formal definition of contextual relevance, and (ii) the establishment of a set-level, uncertainty-driven re-ranking paradigm.
0 citationsRead paper
FrameOracle: Learning What to See and How Much to See in Videos
Existing vision-language models (VLMs) are constrained by fixed-frame inputs, and conventional uniform or budget-driven frame sampling strategies fail to adapt to varying video information density and task complexity—leading to either redundancy or loss of critical frames. To address this, we propose the first lightweight, adaptive module that jointly predicts both the optimal number of keyframes and their identities. We further introduce FrameOracle-41K, a large-scale VideoQA dataset with expert-annotated keyframes. Our method employs a four-stage curriculum learning framework, integrating cross-modal similarity as a proxy signal alongside ground-truth keyframe supervision, enabling plug-and-play deployment. Extensive evaluation across five VLMs and six benchmarks demonstrates: (1) compression of 16-frame inputs to an average of 10.4 frames with zero accuracy degradation; and (2) reduction of 64-frame candidates to 13.9 frames while improving accuracy by 1.4%, achieving state-of-the-art efficiency–accuracy trade-off.
0 citationsRead paper
FinChart-Bench: Benchmarking Financial Chart Comprehension in Vision-Language Models
Existing large vision-language models (LVLMs) exhibit poor performance in financial chart understanding, particularly struggling with complex temporal structures and domain-specific terminology. Method: We introduce FinChart-Bench—the first benchmark tailored to real-world financial scenarios—comprising 1,200 chart images and 7,016 human-annotated questions spanning true/false, multiple-choice, and open-ended formats. It systematically evaluates 25 open- and closed-source LVLMs, uniquely emphasizing temporal modeling, instruction following, and spatial reasoning—capabilities critical for financial chart interpretation. Contribution/Results: FinChart-Bench reveals fundamental limitations in current LVLMs’ reliability for automated financial analysis. Experiments show a narrowing performance gap between open- and closed-source models; however, several newly released models suffer from regression. Widespread deficiencies persist—including weak spatial reasoning and poor instruction comprehension—highlighting urgent needs for domain-adapted architectural and training innovations.
0 citationsRead paper
DuaShepherd: Integrating Stepwise Correctness and Potential Rewards for Mathematical Reasoning
Existing large language models (LLMs) for mathematical reasoning suffer from a disconnect between modeling step-wise correctness and final answer success probability. Method: This paper proposes DuaShepherd, a novel framework featuring a correctness-potential dual-signal co-modeling mechanism. It introduces a large-scale, human-verified reward dataset annotated with both step-wise correctness and potential success probability, and designs a unified multi-head, multi-task architecture to jointly optimize these two signals, augmented by a composite probability fusion strategy. Contribution/Results: Evaluated via an automated reward data pipeline and a rigorous RL benchmark—MATH500 and ProcessBench—DuaShepherd achieves state-of-the-art performance in mathematical reasoning, significantly outperforming single-signal baselines under equivalent computational resources.
0 citationsRead paper
Recent publications
Latest Papers
When Evidence Conflicts: Reliability-aware Meta-review Generation
本文针对同行评审中证据冲突和可靠性差异问题，提出一种基于证据可靠性聚合的元评审生成方法，有效识别并解决冲突，提升评审质量。
0 citationsRead paper
Contextual Relevance and Adaptive Sampling for LLM-Based Document Reranking
For complex queries involving multifaceted requirements and nuanced semantics, document relevance becomes highly context-dependent, rendering conventional re-ranking methods inadequate. To address this, we propose a “contextual relevance” modeling framework that formalizes relevance as the joint probability distribution over candidate document sets, revealing the substantial impact of document composition and ordering on large language model (LLM) relevance judgments. We introduce TS-SetRank—a novel algorithm integrating Bayesian uncertainty estimation with Thompson sampling—to enable uncertainty-aware, adaptive set-level sampling and re-ranking. Evaluated on BRIGHT and BEIR benchmarks, TS-SetRank achieves 15–25% and 6–21% improvements in nDCG@10, respectively, outperforming state-of-the-art retrieval and re-ranking approaches. Our core contributions are: (i) the first formal definition of contextual relevance, and (ii) the establishment of a set-level, uncertainty-driven re-ranking paradigm.
0 citationsRead paper
FrameOracle: Learning What to See and How Much to See in Videos
Existing vision-language models (VLMs) are constrained by fixed-frame inputs, and conventional uniform or budget-driven frame sampling strategies fail to adapt to varying video information density and task complexity—leading to either redundancy or loss of critical frames. To address this, we propose the first lightweight, adaptive module that jointly predicts both the optimal number of keyframes and their identities. We further introduce FrameOracle-41K, a large-scale VideoQA dataset with expert-annotated keyframes. Our method employs a four-stage curriculum learning framework, integrating cross-modal similarity as a proxy signal alongside ground-truth keyframe supervision, enabling plug-and-play deployment. Extensive evaluation across five VLMs and six benchmarks demonstrates: (1) compression of 16-frame inputs to an average of 10.4 frames with zero accuracy degradation; and (2) reduction of 64-frame candidates to 13.9 frames while improving accuracy by 1.4%, achieving state-of-the-art efficiency–accuracy trade-off.
0 citationsRead paper
FinChart-Bench: Benchmarking Financial Chart Comprehension in Vision-Language Models
Existing large vision-language models (LVLMs) exhibit poor performance in financial chart understanding, particularly struggling with complex temporal structures and domain-specific terminology. Method: We introduce FinChart-Bench—the first benchmark tailored to real-world financial scenarios—comprising 1,200 chart images and 7,016 human-annotated questions spanning true/false, multiple-choice, and open-ended formats. It systematically evaluates 25 open- and closed-source LVLMs, uniquely emphasizing temporal modeling, instruction following, and spatial reasoning—capabilities critical for financial chart interpretation. Contribution/Results: FinChart-Bench reveals fundamental limitations in current LVLMs’ reliability for automated financial analysis. Experiments show a narrowing performance gap between open- and closed-source models; however, several newly released models suffer from regression. Widespread deficiencies persist—including weak spatial reasoning and poor instruction comprehension—highlighting urgent needs for domain-adapted architectural and training innovations.
0 citationsRead paper
DuaShepherd: Integrating Stepwise Correctness and Potential Rewards for Mathematical Reasoning
Existing large language models (LLMs) for mathematical reasoning suffer from a disconnect between modeling step-wise correctness and final answer success probability. Method: This paper proposes DuaShepherd, a novel framework featuring a correctness-potential dual-signal co-modeling mechanism. It introduces a large-scale, human-verified reward dataset annotated with both step-wise correctness and potential success probability, and designs a unified multi-head, multi-task architecture to jointly optimize these two signals, augmented by a composite probability fusion strategy. Contribution/Results: Evaluated via an automated reward data pipeline and a rigorous RL benchmark—MATH500 and ProcessBench—DuaShepherd achieves state-of-the-art performance in mathematical reasoning, significantly outperforming single-signal baselines under equivalent computational resources.
0 citationsRead paper