🤖 AI Summary
Chunk size selection in long-document retrieval exhibits dataset- and embedding-model-dependent sensitivity, causing significant performance fluctuations. Method: We conduct a systematic empirical analysis across diverse datasets (short-form vs. long-form question answering) and state-of-the-art embedding models (e.g., Stella, Snowflake), quantifying model-specific chunk-size sensitivity for the first time. Contribution/Results: We identify an intrinsic trade-off: smaller chunks (64–128 tokens) yield superior performance on factoid QA, whereas larger chunks (512–1024 tokens) substantially improve retrieval accuracy for long-context queries. Based on these findings, we propose the “chunk–model–data” triadic co-adaptation principle—a principled, reproducible, and transferable framework for optimizing chunking strategies in long-document retrieval. This work provides actionable guidelines for aligning chunk granularity with model architecture and task characteristics, advancing robustness and generalizability in retrieval systems.
📝 Abstract
Chunking is a crucial preprocessing step in retrieval-augmented generation (RAG) systems, significantly impacting retrieval effectiveness across diverse datasets. In this study, we systematically evaluate fixed-size chunking strategies and their influence on retrieval performance using multiple embedding models. Our experiments, conducted on both short-form and long-form datasets, reveal that chunk size plays a critical role in retrieval effectiveness -- smaller chunks (64-128 tokens) are optimal for datasets with concise, fact-based answers, whereas larger chunks (512-1024 tokens) improve retrieval in datasets requiring broader contextual understanding. We also analyze the impact of chunking on different embedding models, finding that they exhibit distinct chunking sensitivities. While models like Stella benefit from larger chunks, leveraging global context for long-range retrieval, Snowflake performs better with smaller chunks, excelling at fine-grained, entity-based matching. Our results underscore the trade-offs between chunk size, embedding models, and dataset characteristics, emphasizing the need for improved chunk quality measures, and more comprehensive datasets to advance chunk-based retrieval in long-document Information Retrieval (IR).