Zephon: Elastic Determinism for Online, Stateful Foundation Model Data Loading Pipelines
This study addresses the non-deterministic data loading in online stateful data pipelines caused by topology changes, failure recovery, and backend heterogeneity. It proposes Zephon, a system enabling resilient and deterministic data loading. The core innovations include a novel topology-agnostic stream partitioning scheme and an ordered decision serialization mechanism, which achieve constant-overhead checkpoint recovery by persisting only bounded in-flight states. These techniques are integrated with stateless parallel processing, backend abstraction, and incremental checkpointing to form a comprehensive solution. Experiments demonstrate that Zephon sustains high throughput under both text and multimodal workloads while providing online determinism guarantees unattainable by existing approaches.