retrieval-augmented interpretation

Large language models (LLMs) exhibit weak multi-hop reasoning capabilities and struggle to locate and integrate critical information in ultra-long contexts (>100K tokens). Method: This paper proposes a purely prompt-driven, end-to-end reasoning framework that synergistically combines structured prompt engineering with chain-of-thought (CoT) prompting to intrinsically enable key passage localization, stepwise evidence integration, and lightweight inference—all within a single forward pass. It emulates the retrieval-and-reasoning functionality of RAG without external retrievers. Contribution/Results: The work identifies the decisive impact of prompt elements—such as question, label, and instruction ordering—on long-range comprehension. Evaluated on the BABILong benchmark, it significantly outperforms both retrieval-free baselines and naive RAG across multi-fact question answering tasks—including object position tracking, dynamic counting, and uncertain knowledge reasoning—demonstrating strong robustness. Results empirically validate that optimized prompting can substantively replace conventional retrieval pipelines.

The impact of key design decisions—RAG activation, retrieval granularity, and knowledge integration strategy—on RAG system performance remains poorly understood. Method: We conduct systematic ablation studies across three code/qa benchmarks and two state-of-the-art LLMs, quantitatively evaluating how document type, recall rate, document selection strategy, and prompt engineering jointly affect answer correctness and confidence via multi-dimensional analysis, cross-model/dataset comparison, and joint prompt-retrieval analysis. Contribution/Results: We identify precise interaction patterns and operational boundaries among these factors and propose nine actionable, empirically grounded guidelines for diagnosing and optimizing RAG failures. Our findings significantly improve RAG system stability, debuggability, and reliability, offering rigorous empirical evidence and a principled methodology to support the engineering deployment of LLM-augmented systems.

To address core limitations of large language models (LLMs)—including hallucination, knowledge obsolescence, and poor domain adaptability—this work systematically advances the Retrieval-Augmented Structured (RAS) generation paradigm. We propose a multi-granularity knowledge acquisition mechanism integrating sparse, dense, and hybrid retrieval, coupled with text structuralization, taxonomy construction, knowledge embedding, and prompt-driven reasoning to enable efficient external knowledge retrieval, semantic alignment, and controllable integration. Crucially, we deeply embed structured modeling into the augmentation pipeline, enhancing factual accuracy, temporal freshness, and domain-specific competence of generated outputs. Our contributions include: (1) a unified methodological framework for RAS generation; (2) principled pathways toward multimodal, cross-lingual, and interactive augmented generation; and (3) empirically validated improvements in reliability and specialization across diverse domains. This work establishes foundational design principles and future research directions for next-generation RAS systems.

Current prompt engineering methodologies overemphasize automation techniques (e.g., role-playing, chain-of-thought) while neglecting users’ ability to articulate clear, customized requirements—resulting in low-quality prompts for complex tasks. Method: This paper introduces Requirement-Oriented Prompt Engineering (ROPE), a novel paradigm centered on *requirement quality* as the core training objective. ROPE establishes a human-centered instructional framework integrating expert annotation, structured training tasks, and LLM-driven real-time feedback to iteratively refine requirement formulation. Contribution/Results: Empirical analysis confirms a strong positive correlation between input requirement quality and downstream LLM performance. A randomized controlled trial with 30 novices demonstrates that ROPE improves task success rate by 20%—significantly outperforming conventional prompt training (+1%)—and this gain is not replicable via automated prompt optimization alone. The framework yields a scalable, pedagogically grounded teaching toolkit for effective prompt authoring.

Existing tool documentation is often inaccurate or incomplete, hindering large language models (LLMs) from effectively invoking external tools. To address this, we propose DRAFT, a framework that enables dynamic optimization of tool documentation through a self-driven, closed-loop interaction between LLMs and tools. Its core is a three-stage self-iterative mechanism: experience collection → experience learning → documentation rewriting, augmented by diversity-aware exploration and tool-adaptive early stopping. The method integrates LLM self-feedback analysis, trial-and-error-driven rewriting, and tool-aware sampling. On multiple benchmarks, DRAFT significantly improves tool-call accuracy; the refined documentation exhibits strong cross-model generalization, more efficient iteration, and robustness against overfitting. This work constitutes the first approach enabling LLM-led autonomous evolution of tool documentation, establishing a high-quality knowledge foundation for tool-augmented AI systems.

This work addresses the challenge of generating personalized, factually accurate, and topically relevant reading materials tailored to user-specified queries and target readability levels. To this end, the authors propose a four-module system that integrates retrieval-augmented generation (RAG) with large language models (LLMs), uniquely combining RAG with multiple prompting strategies—including Chain-of-Thought, zero-shot, and few-shot prompting—for personalized reading recommendations. The system further incorporates an LLM-as-a-Judge mechanism to automatically evaluate the factual accuracy, relevance, and readability alignment of generated content. Experimental results demonstrate that RAG consistently enhances the performance of diverse models—including LLaMA 4 Scout, LLaMA 3.1 8B, and Gemma2 9B—across all prompting strategies, yielding improvements of up to 26–35 percentage points in relevance and factual accuracy, thereby enabling high-quality customized reading material generation.

This work addresses the challenge that large language models often struggle to simultaneously satisfy content relevance and formal constraints, leading to procedural errors. To overcome this, the authors propose a multi-agent workflow that, for the first time, decouples the primary task description from fine-grained constraints and iteratively refines prompts through an evaluation-driven collaborative mechanism. By integrating automated scoring feedback, prompt rewriting, and multi-agent coordination, the approach significantly enhances adherence to formal constraints in model outputs. Experiments on Llama 3.1 8B and Mixtral-8x 7B demonstrate substantial improvements, validating the effectiveness of constraint decoupling and evaluation-guided refinement in boosting instruction-following performance.

This work addresses the fragmented collaboration between domain experts and developers in large language model (LLM) application development by proposing an end-to-end open-source platform that seamlessly integrates collaborative prompt editing, one-click batch experimentation, and human–AI hybrid evaluation for the first time. The platform supports batch scheduling across multiple models and prompts, real-time consistency metrics, version control, cost tracking, and result provenance. User studies demonstrate that the system significantly enhances cross-role collaboration efficiency, offers an intuitive interface, reduces time overhead, and has been successfully deployed in an online psychological counseling scenario, validating its effectiveness and practical utility.

This work addresses the lack of standardized documentation and evaluation methodologies in prompt engineering, which hinders the reproducibility and interpretability of complex prompts. To remedy this, the authors propose “Prompt Cards,” a novel framework that adapts the model card concept to prompt engineering by introducing a structured template to explicitly document a prompt’s design objectives, contextual strategies, evaluation protocols, and ethical considerations. Demonstrated through a “wordification” task, the approach integrates natural language generation with qualitative assessment to enable systematic recording and analysis of the entire prompting pipeline. Prompt Cards substantially enhance transparency, reproducibility, and methodological rigor, offering the research community a scalable standard for prompt documentation and a new paradigm for benchmarking beyond conventional metrics.

To address the dual challenges of high manual effort in constructing interactive software guides for enterprises and the hallucination-prone, hard-to-finetune nature of large language models (LLMs), this paper proposes a retrieval-augmented generation (RAG) framework grounded in a state-action knowledge graph. Our method automatically parses web interfaces to model complex enterprise systems—such as CRM and ERP platforms—as structured, queryable knowledge graphs, enabling context-aware navigation and precise reasoning over black-box LLMs without fine-tuning. Key contributions include: (1) the first formulation of dynamic UI states and user actions as a jointly modeled, retrievable graph structure; and (2) the integration of graph-based retrieval with RAG to enhance both interpretability and robustness of generated guidance. The framework has been deployed in production learning platforms RAKAM and Lemon Learning. Empirical evaluation demonstrates its high scalability and operational effectiveness in real-world industrial settings.