PluRel-to-RDB-PFN: Schema-Guided Synthetic Relational Pretraining

2026-07-31

AI Summary

This work addresses the heavy reliance of existing relational foundation models on synthetic data tightly coupled with their training pipelines and the lack of effective methods to leverage external data sources. The authors propose a decoupled transformation pipeline that adapts databases generated by the general-purpose synthetic relational database generator PluRel into the RDB-PFN training format, augmented with a schema-guided curriculum learning strategy. This study provides the first empirical validation that external synthetic data can effectively support pretraining of relational foundation models, demonstrating that early exposure to real-world schemas significantly outperforms late-stage adaptation. Using only approximately 5,500 databases (33K tasks), the approach achieves an average ROC-AUC of 0.6346 under the 1024-shot setting—87.6% of the original performance—and remarkably reaches 93.8% of the original performance in the 64-shot setting.

Abstract

Relational Foundation Models (RFMs) require large-scale synthetic relational databases for pretraining, but existing approaches tightly couple data generation with the model training pipeline. We study whether PluRel, a general-purpose synthetic relational database generator, can serve as an external data source for RDB-PFN, a relational in-context learner originally pretrained with a 600K-task single-table warm-up followed by an approximately 1.8M-task adaptation stage. We build a conversion pipeline that maps PluRel-generated databases, including externally constructed binary prediction tasks, into the RDB-PFN training format and evaluate three curriculum strategies: SCHEMA-GUIDED FIRST (real-world schema then fully synthetic), FULLY SYNTHETIC (diverse synthetic schemas throughout), and SCHEMA-GUIDED LAST (fully synthetic then real-world schema). Using only approximately 5,500 relational databases (approximately 33K tasks), roughly 55x fewer tasks than the original protocol, and no single-table warm-up, our best curriculum (SCHEMA-GUIDED FIRST) achieves 0.6346 average ROC-AUC across 19 real benchmark tasks at 1024-shot context, recovering 87.6% of the published RDB-PFN performance (0.7245). At 64-shot context, the gap narrows to 93.8% (0.6116 vs. 0.6517). Our results demonstrate that external synthetic generators can provide useful pretraining signals for RFMs when combined with appropriate curriculum design and that exposure to a real-world schema early in training is substantially more effective than late-stage schema adaptation.

Problem

• Relational Foundation Models
• synthetic relational databases
• pretraining
• schema-guided
• in-context learning

Innovation

• relational foundation models
• synthetic data generation
• schema-guided pretraining
• curriculum learning
• in-context learning