differentiable execution

This paper addresses the challenge of end-to-end differentiability in complex programs featuring nontrivial control flow and data structures. To this end, it introduces a probabilistic programming paradigm for differentiation, unifying optimization and probabilistic inference within a differentiable programming framework. Methodologically, it transcends conventional automatic differentiation (AD) by establishing, for the first time, a theoretical link between differentiability of control flow/data structures and uncertainty modeling—integrating AD, graphical models, convex optimization, and Bayesian inference into a cohesive differentiable program modeling framework. Key contributions include: (1) revealing that differentiable programming is fundamentally probabilistic programming—not merely gradient computation; (2) proposing the “program-as-model” design principle; and (3) establishing the first comprehensive knowledge system spanning theory, design, and applications, enabling the development of differentiable software infrastructure for large language models and foundation models.

This work addresses the longstanding divide between program execution and gradient-based optimization, which has hindered the use of general-purpose code as learnable scientific models. The authors propose the Differentiable Meta-Circular Interpreter (DMCI), which compiles a self-hosted subset of Scheme into a differentiable computation graph, enabling exact reverse-mode automatic differentiation of arbitrary recursive, higher-order programs with closures and complex data structures—without requiring recompilation or custom gradient definitions. This approach facilitates joint optimization over both program structure and parameters. Validated on 171 programs for gradient correctness, DMCI significantly outperforms handcrafted models and gradient-free methods in inverse problems such as battery degradation and El Niño prediction, successfully achieving symbolic regression and extrapolation with stateful executable programs.

This work addresses the challenge of designing high-fidelity quantum circuits, which often suffer from limited generality and suboptimal performance. The authors propose modeling the design problem as differentiable logic programming, uniquely integrating continuous logic with quantum unitary evolution. They develop an optimization framework based on T-norm fuzzy logic and geodesic interpolation, augmented with bias-aware initialization to mitigate barren plateaus. The approach enables users to specify custom differentiable logical axioms—such as correctness, simplicity, and robustness—and automatically discovers a 4-qubit quantum Fourier transform (QFT) circuit from a space of 21 candidate gates. Validated on the IBM Torino processor for local routing tasks, the method achieves a 59.3-percentage-point fidelity improvement over baselines and demonstrates resilience to hardware faults.

This work addresses the challenge of implementing reverse-mode automatic differentiation for programs featuring algebraic effects such as finite discrete probabilistic choice. Building upon the Compositional Homomorphic Automatic Differentiation (CHAD) framework, it formulates automatic differentiation as a semantics-preserving program transformation and constructs a backward-pass mechanism tailored to the finite atomic distribution monad. The correctness of this construction is established using logical relations from category theory. This study presents the first systematic extension of reverse-mode automatic differentiation to effectful languages with discrete outputs, introducing a reusable differentiation scheme applicable to a broad class of algebraic effects—including nondeterminism, exceptions, and writer effects. The approach not only enables correct reverse differentiation of programs with finite discrete probabilistic structure but also lays a foundational theoretical groundwork for differentiating more general effectful languages.

Binary program symbolic execution suffers from semantic distortion and implementation errors introduced during intermediate representation (IR) translation. Method: This paper proposes the first instruction-level symbolic execution framework directly grounded in formal ISA semantics (Rock/Sail), bypassing conventional IR abstractions by compiling machine-readable ISA specifications into SMT-solvable symbolic semantic models and integrating them into a binary analysis platform. Contributions/Results: (1) The first end-to-end automated pipeline from formal ISA semantics to symbolic execution; (2) Demonstrated scalability on RISC-V—modeling new instructions requires only a few hours; (3) Discovered five previously unknown ISA semantic implementation bugs in angr; (4) Achieved high-fidelity branch modeling and solving capability. The framework significantly improves the accuracy, trustworthiness, and development efficiency of binary symbolic execution.

This work addresses the opacity of neural network internal states during program execution, which hinders verification of correctness and trustworthiness. The authors propose a symbolic neural CPU architecture that enables interpretable program execution through trajectory supervision, explicitly modeling instruction selection, register reads and writes, memory access, and write-back semantics. This architecture is the first to support verifiable full execution trajectories, preserving symbolic path consistency under quantized simulation and incorporating a fixed-point replay mechanism to eliminate numerical drift. By integrating loop control, differentiable ALU routing, target-masked write-back, behavioral cloning, and Actor-Critic reinforcement learning—while aligning with RV32I instruction set semantics—the system precisely reproduces reference executions at 16-bit precision and maintains symbolic path consistency over thousands of instructions even at 8-bit quantization, demonstrating controllable and interpretable low-precision neural execution.

This work addresses the abstraction gap between differentiable programming and emerging probabilistic hardware by introducing Parameterized Stochastic Circuits (PSCs) as a gate-level intermediate representation that closely mirrors native hardware operations. PSCs unify explicit binary, categorical, and continuous signals with localized stochastic kernels. Building on this foundation, the authors develop torx, an open-source JAX framework that enables, for the first time, direct alignment between differentiable stochastic computation and probabilistic hardware, substantially reducing mapping overhead. Experiments on the X0 subthreshold CMOS probabilistic bit chip demonstrate that PSCs efficiently harness physical randomness in tasks such as graph random walks, discrete diffusion, and stochastic graph networks, achieving results consistent with pseudorandom software baselines while confirming the approach’s validity and hardware energy efficiency.

This work addresses the inefficiency of joint program-and-parameter search in neuro-symbolic learning, where each candidate program requires separate parameter optimization. To overcome this bottleneck, the authors propose the Neural Differentiable Virtual Machine (NDVM), which applies automatic differentiation to the interpreter rather than individual programs. By decoupling symbolic structure from differentiable numeric state, NDVM preserves program dynamics while enabling precise backpropagation through execution traces. The design incorporates dense batched numeric buffers and runtime symbolic environment management, substantially amortizing evaluation overhead. Experiments demonstrate that NDVM achieves approximately 60× amortized speedup per-channel batching over baseline methods, exhibits near-linear multi-core scaling, and accelerates the discovery of high-quality solutions by roughly 24× under a fixed computational budget.

This work proposes a parameterized information-flow framework that unifies confidentiality and integrity through their joint interaction, leveraging the duality between open and closed modalities in modal type theory. Traditional approaches model these security properties separately, leading to redundant reasoning, complex specifications, and degradation mechanisms that often undermine modularity and abstraction. In contrast, the proposed framework naturally supports downgrading operations without requiring additional extensions, while remaining compatible with strong noninterference guarantees and practical declassification needs. It preserves full noninterference and not only reproduces but also strengthens mechanisms such as robust declassification, demonstrating their complete compatibility with modular design and abstraction.