🤖 AI Summary
This study investigates how to enhance retrieval effectiveness over RDF datasets while preserving the semantic faithfulness of metadata generated by large language models (LLMs) to the original data. It formulates metadata generation for the first time as a system-level information retrieval problem and systematically evaluates six LLM-based strategies—ranging from unconstrained rewriting to knowledge graph–grounded agent approaches—in terms of the trade-off between retrieval performance and content faithfulness. Experimental results show that unconstrained rewriting yields the greatest retrieval gains but suffers from the lowest faithfulness, whereas profile-guided rewriting achieves the best balance between the two objectives. The work reveals that retrieval improvements may stem from unfaithful semantic expansions and proposes a new paradigm that jointly optimizes effectiveness and trustworthiness in metadata generation for semantic data retrieval.
📝 Abstract
Dataset search depends heavily on metadata, making LLM-generated metadata a consequential form of synthetic content in retrieval systems. We study six metadata-generation settings for RDF datasets, ranging from simple rewriting to profile-grounded and agentic graph-based generation, and evaluate them jointly for retrieval effectiveness and faithfulness. Unconstrained metadata rewriting delivers the strongest retrieval gains over the original metadata, but it is also the least faithful, showing that search improvements can be driven by unsupported semantic expansion. More grounded settings substantially improve faithfulness, and profile-grounded rewriting provides the most balanced trade-off between retrieval effectiveness and grounding. These findings position synthetic metadata as a system-level IR problem in which effectiveness, provenance, and trust must be evaluated together.