span extraction

This work addresses the challenge of extracting scientific hypotheses and their supporting statistical evidence from research papers, a task hindered by the documents’ length and dispersed information. The authors propose a two-stage retrieve-and-extract framework that incorporates a paper-structure-aware context selection strategy to effectively link key findings in abstracts with corresponding hypotheses and evidence in the main text. Through systematic evaluation of various configurations—including standard RAG, re-ranking, fine-tuned retrievers, and large language model extractors—and by decoupling retrieval from extraction performance using oracle passages, the study demonstrates that high-quality contextual passages substantially improve hypothesis extraction. However, extracting statistical evidence remains challenging, revealing limitations in current models when processing hybrid numerical-textual statements.

This work investigates the interplay between retrieval and comprehension of in-document extractive evidence by large language models (LLMs) in few-shot learning, specifically examining whether prediction errors stem from retrieval failures and their underlying causes. We conduct error attribution analysis and two-stage ablation studies across five datasets using two representative closed-source LLMs, with human-annotated gold-standard evidence as ground truth. Our key finding—novel and empirically validated—is that prediction errors are strongly coupled with retrieval errors; however, retrieval failure is primarily attributable not to model comprehension deficits, but rather to suboptimal evidence quality (e.g., low clarity or incompleteness). Crucially, improving retrieval accuracy yields significant gains in final prediction performance. These results provide foundational theoretical support and actionable optimization directions for evidence-retrieval–based downstream tasks.

This work addresses the challenge of efficiently selecting evidence in multi-hop retrieval-augmented generation (RAG) systems under fixed context-length constraints, where conventional retrieval metrics poorly predict answer generation quality. The authors propose an answer-in-context diagnostic metric and formulate context construction as a budget-constrained monotone submodular maximization problem, jointly optimizing relevance, query coverage, representativeness, and diversity. Evaluated on HotpotQA with a 160-token budget, their approach achieves up to a 5.1 F1 improvement over strong baselines such as MMR when paired with a 3B reader. However, this advantage diminishes with increasing model scale—vanishing at 7B and reversing at 14B—a performance boundary consistently explained by the proposed answer-in-context metric.

Existing evaluation metrics struggle to accurately assess whether long-context or retrieval-augmented language models genuinely utilize evidence, often conflating memorization, superficial citation, and substantive integration. This work proposes a four-condition diagnostic protocol—comprising no-evidence, full-context, retrieved-evidence, and oracle-evidence settings—to disentangle evidence utilization into four diagnosable dimensions for the first time. It introduces the ONCU metric to quantify the degree to which models recover the performance advantage afforded by oracle evidence. Through carefully matched experimental designs, effective denominator group filtering, and multidimensional audit metrics, the study evaluates five families of open-source models across 18,000 predictions. Results reveal that synthetic tasks are primarily limited by failures in leveraging full context, whereas real multi-hop tasks suffer from insufficient retrieval chain coverage—a distinction consistently corroborated by ONCU scores.

Large language models lack explicit mechanisms to refer to specific spans in the input text, leading to inconsistent performance with existing span annotation prompting strategies. This work systematically examines three categories of approaches: input tagging, numerical indexing, and content matching, and proposes LogitMatch—a novel constrained decoding method that enforces alignment between model outputs and valid input spans in logit space to address the inconsistency inherent in content matching. Experiments across four diverse tasks demonstrate that LogitMatch significantly outperforms existing content matching methods and, in certain settings, surpasses other strategies, while also confirming that input tagging remains a robust baseline.

This work addresses a key challenge in multi-hop retrieval-augmented generation (RAG): disentangling whether performance bottlenecks stem from missing supporting evidence or the reader’s inability to effectively utilize available evidence. To this end, the authors introduce the concept of an “evidence interface” and, for the first time, systematically decouple the impact of missing support chains from evidence interface effects by evaluating adapter-trained readers across three multi-hop QA benchmarks with annotated support chains. Using original contexts, retrieved passages, and oracle-supported inputs, they demonstrate that a support-supervised ranker significantly improves support coverage on 2Wiki and MuSiQue, recovers original-context performance at substantially lower prompting costs, and approaches the upper bound set by gold support annotations.

This work investigates how different sequence representations—such as bytes, characters, and subwords—affect the information acquisition capacity of Transformer models under a fixed context window, a question that remains poorly understood. From an information-theoretic perspective, the paper introduces the notion of “fragmentation” and formally demonstrates that it inherently increases the log-loss of the optimal finite-context model. It establishes theoretical guarantees linking tokenization compression rates to the reliability of source context coverage, thereby constructing the first information-theoretic framework for representation selection in finite-context settings. Through Markov source modeling and comparative analysis of various tokenization strategies—including BPE, WordPiece, and byte-level methods—the study reveals the theoretical underpinnings of performance differences observed in models like ByT5 and CANINE, and proposes practical metrics to evaluate the effective context coverage of real-world tokenizers.

This study addresses the confounding effects of metadata, structured representations, and retrieval mechanisms in current RAG systems, which often combine multiple context-augmentation strategies, obscuring their individual contributions to answer quality. Through controlled experiments across six benchmarks, four models, and five augmentation levels—totaling over 24,000 evaluations—the work reveals that increased contextual richness does not necessarily improve accuracy. It introduces the “tractability hierarchy” theory, emphasizing that context must align with model capacity. The findings demonstrate that most augmentation strategies actually degrade performance; however, when metadata and retrieval strategies are carefully matched to a model’s capabilities, smaller models can outperform state-of-the-art large models by up to 19 F1 points on specific tasks, challenging the prevailing RAG design paradigm centered on stacking metadata.

Existing evidence construction systems are difficult to diagnose across paradigms due to inconsistent corpora and evaluation metrics. This work proposes AuthTrace, a unified diagnostic benchmark built upon a single-author, topic-dense corpus, comprising 2,099 annotated instances. For the first time, it enables consistent evaluation of mainstream evidence construction paradigms under identical corpora and query sets, introducing fan-in gradient as a core diagnostic dimension. Through exhaustive gold-standard evidence annotation and multi-system comparative experiments, the study identifies evidence recall as a strong predictor of answer quality (r = 0.96), reveals distinct performance degradation patterns across paradigms under high fan-in conditions, and demonstrates the general ineffectiveness of full-context prompting.

This work addresses document-level conspiracy theory detection by proposing a joint framework that integrates multi-label span classification with sequence classification. For extracting conspiracy-related markers—such as roles and actions—the approach formulates the task as boundary-aware multi-label span classification, incorporating IoU-based positive labeling, hard negative sampling, and an inclusion-aware non-maximum suppression strategy, while distinguishing between entity-like and abstract roles. Document-level classification is performed using a RoBERTa model enhanced with label smoothing. Evaluated on SemEval-2026 Task 10, the method achieves 7th place in Subtask 1 (macro F1 = 0.2251) and 11th place in Subtask 2 (weighted F1 = 0.7694), demonstrating the effectiveness of the proposed techniques.