🤖 AI Summary
This study addresses the limitations of existing text-to-MQL benchmarks, where mechanical translation causes information loss and query inefficiency, hindering natural language querying over document databases. We propose an access-pattern-driven construction paradigm that abandons heuristic conversion in favor of a human-in-the-loop workflow with coding agents. By designing native MongoDB schemas according to anticipated access patterns and performing query rewriting, our approach achieves lossless migration from SQLite to document databases. Based on this methodology, we introduce AptMQL-Bench, comprising 21 databases and 3,186 samples. Evaluations reveal that even the strongest model attains only 57.38% accuracy, demonstrating the substantial challenge and research value of this task.
📝 Abstract
Document databases such as MongoDB are core infrastructure for modern applications, and natural-language interfaces to them---text-to-MQL---would let non-experts query complex, semi-structured data without mastering the query language. Progress on this task depends on high-quality benchmarks, which are most practically obtained by converting an existing text-to-SQL benchmark to the document setting. Unfortunately, existing efforts rely on heuristics for mechanical conversion: the document schema mirrors the relational foreign-key graph, and each query mirrors its source SQL. As a result in our experiments, these approaches fail to migrate 6 of 21 BIRD databases outright, silently drop up to 25.9\% of rows on others, and yield schemas whose ground-truth queries run over an order of magnitude slower as the data scales. We instead propose a conversion pipeline, driven by coding agents with human-in-the-loop verification, that designs each document schema from expected access patterns and rewrites queries to be MongoDB-native. Applying it to BIRD, we build an access-pattern-based text-to-MQL benchmark (AptMQL-Bench). It includes 21 document-oriented databases, 3,186 natural-language requests, and their associated MQL queries---whose databases are migrated from SQLite without data loss and scale efficiently. The strongest model, Claude Opus 4.5, achieves only 57.38\% accuracy without external knowledge evidence and 70.34\% with it. This indicates that realistic text-to-MQL generation remains challenging.