HtmlRAG: HTML is Better Than Plain Text for Modeling Retrieved Knowledge in RAG Systems

2024-11-05

arXiv.org

AI Summary

Conventional RAG systems convert HTML sources to plain text, discarding structural and semantic information (e.g., headings, tables), thereby degrading retrieval and answer generation. Method: This paper introduces a novel knowledge modeling paradigm that directly ingests raw HTML. It (i) applies lightweight HTML cleaning and compression; (ii) proposes a two-stage, semantics-aware tree pruning algorithm to retain critical structural units while suppressing noise; and (iii) systematically evaluates the impact of native LLM HTML comprehension on RAG performance—first such investigation. Results: Experiments across six QA benchmarks show consistent improvements: our method outperforms plain-text RAG baselines by 3.2–7.8 percentage points in accuracy, with marked gains in answer correctness and robustness. These results empirically validate HTML’s efficacy, generalizability, and practicality as a structured knowledge representation for RAG.

Technology Category

• Data Mining & Knowledge Management: Knowledge Acquisition from the Web
• Knowledge Representation and Reasoning: Knowledge Acquisition
• Natural Language Processing: Discourse, Pragmatics & Argument Mining

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Retrieval-Augmented Generation (RAG) has been shown to improve knowledge capabilities and alleviate the hallucination problem of LLMs. The Web is a major source of external knowledge used in RAG systems, and many commercial RAG systems have used Web search engines as their major retrieval systems. Typically, such RAG systems retrieve search results, download HTML sources of the results, and then extract plain texts from the HTML sources. Plain text documents or chunks are fed into the LLMs to augment the generation. However, much of the structural and semantic information inherent in HTML, such as headings and table structures, is lost during this plain-text-based RAG process. To alleviate this problem, we propose HtmlRAG, which uses HTML instead of plain text as the format of retrieved knowledge in RAG. We believe HTML is better than plain text in modeling knowledge in external documents, and most LLMs possess robust capacities to understand HTML. However, utilizing HTML presents new challenges. HTML contains additional content such as tags, JavaScript, and CSS specifications, which bring extra input tokens and noise to the RAG system. To address this issue, we propose HTML cleaning, compression, and a two-step block-tree-based pruning strategy, to shorten the HTML while minimizing the loss of information. Experiments on six QA datasets confirm the superiority of using HTML in RAG systems.

Problem

• HTML vs plain text in RAG
• Preserving HTML structural information
• Handling HTML noise in RAG

Innovation

• HTML replaces plain text
• HTML cleaning and compression
• Block-tree-based pruning strategy