Multi-Language Probabilistic Programming

2025-02-26

AI Summary

Existing probabilistic programming languages (PPLs) operate in isolation, hindering collaborative execution of heterogeneous probabilistic programs and limiting modeling flexibility and inference scalability. This paper introduces MultiPPL, a multi-language probabilistic programming framework that establishes, for the first time, a formal semantic foundation for cross-PPL interoperability, enabling seamless integration of high-precision discrete inference and approximate importance sampling. Our method comprises three core innovations: (1) a unified syntax and formal semantics for cross-language program composition; (2) soundness-preserving cross-PPL inference algorithms that guarantee correctness under composition; and (3) a hybrid inference scheduling mechanism coupled with a collaborative execution engine. Experimental evaluation demonstrates that MultiPPL significantly enhances modeling expressiveness and inference practicality for complex heterogeneous programs—while rigorously preserving inference correctness.

Technology Category

• Reasoning under Uncertainty: Probabilistic Programming
• Machine Learning: Probabilistic Circuits and Graphical Models
• Multiagent Systems: Multiagent Planning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

There are many different probabilistic programming languages that are specialized to specific kinds of probabilistic programs. From a usability and scalability perspective, this is undesirable: today, probabilistic programmers are forced up-front to decide which language they want to use and cannot mix-and-match different languages for handling heterogeneous programs. To rectify this, we seek a foundation for sound interoperability for probabilistic programming languages: just as today's Python programmers can resort to low-level C programming for performance, we argue that probabilistic programmers should be able to freely mix different languages for meeting the demands of heterogeneous probabilistic programming environments. As a first step towards this goal, we introduce extsc{MultiPPL}, a probabilistic multi-language that enables programmers to interoperate between two different probabilistic programming languages: one that leverages a high-performance exact discrete inference strategy, and one that uses approximate importance sampling. We give a syntax and semantics for extsc{MultiPPL}, prove soundness of its inference algorithm, and provide empirical evidence that it enables programmers to perform inference on complex heterogeneous probabilistic programs and flexibly exploits the strengths and weaknesses of two languages simultaneously.%

Problem

• Interoperability in probabilistic programming languages
• Handling heterogeneous probabilistic programs
• Mixing exact and approximate inference strategies

Innovation

• Multi-language probabilistic programming
• Interoperability between inference strategies
• Syntax and semantics for MultiPPL