From Interpretation to Compilation: A Compilation-Based Execution Engine for Semantic Operator Systems

2026-08-06

AI Summary

This work addresses the high latency and linearly scaling costs of existing semantic operator systems, which rely on interpreted execution with repeated large language model (LLM) invocations. The paper introduces SemBaker, the first framework to apply compilation principles to semantic operator execution: it uses a single LLM call to generate deterministic Python functions that efficiently perform filtering, mapping, and joining operations locally, thereby decoupling LLM invocation from data iteration. SemBaker features a plug-in architecture enabling integration with multiple systems and incorporates a cost-based optimizer that dynamically selects the optimal execution plan. Experimental results across three 200-query question-answering workloads demonstrate that SemBaker achieves average speedups of 4.8–6.3× and cost reductions of 5.4–10.7× while maintaining competitive result quality.

Abstract

Semantic operators extend data processing with natural-language predicates. Existing semantic operator systems commonly execute these operators through interpretation-based execution: for every data item, an LLM interprets the operator predicate and directly produces the corresponding result. Although expressive, this design places expensive model invocations inside the data-processing loop, causing latency and monetary cost to scale with input cardinality. We present SemBaker, a compilation-based execution engine for semantic operator systems. SemBaker acts as an external plugin rather than replacing a backend's native execution. For selected semantic filters, maps, and joins, it invokes an LLM once to generate a deterministic Python function and executes that function locally without per-item LLM calls. A cost-based optimizer routes each operator to native or compiled execution, while compilation overlaps pipeline execution. SemBaker supports Palimpzest, LOTUS, Nirvana, and DocETL through thin adapters. Across three 200-query QA workloads, SemBaker achieves average speedups of 4.8 to 6.3 times and average cost reductions of 5.4 to 10.7 times, with competitive processing quality.

Problem

• semantic operators
• interpretation-based execution
• LLM invocation cost
• data processing latency
• scalability

Innovation

• compilation-based execution
• semantic operators
• LLM optimization
• cost-based optimizer
• deterministic code generation