A Lightweight Method for Generating Multi-Tier JIT Compilation Virtual Machine in a Meta-Tracing Compiler Framework

2025-04-24

AI Summary

Virtual machines (VMs) generated by meta-compilation frameworks—such as RPython and Graal/Truffle—typically lack multi-tier just-in-time (JIT) compilation support, hindering the balance between startup latency and peak performance for dynamic languages. Method: This paper proposes a lightweight, backend-agnostic paradigm for constructing multi-tier JIT compilers: by embedding compilation directives directly into the interpreter definition, a first-tier lightweight compiler can be built without modifying the underlying framework backend. Contribution/Results: Our approach achieves the first clean decoupling of compilation tiers from the framework backend. Implemented in the RPython-generated 2SOM VM, it enables cooperative two-tier compilation—combining a threaded-code interpreter with a tracing JIT. Evaluation shows a 15% improvement in warmup performance over the standard RPython VM, with only a 5% degradation in peak performance. This significantly enhances the trade-off between startup latency and sustained execution efficiency in dynamic-language VMs.

Technology Category

• Machine Learning: Hardware-aware ML
• Multiagent Systems: Other Foundations of Multi Agent Systems
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements

Abstract

Meta-compiler frameworks, such as RPython and Graal/Truffle, generate high-performance virtual machines (VMs) from interpreter definitions. Although they generate VMs with high-quality just-in-time (JIT) compilers, they still lack an important feature that dedicated VMs (i.e., VMs that are developed for specific languages) have, namely emph{multi-tier compilation}. Multi-tier compilation uses light-weight compilers at early stages and highly-optimizing compilers at later stages in order to balance between compilation overheads and code quality. We propose a novel approach to enabling multi-tier compilation in the VMs generated by a meta-compiler framework. Instead of extending the JIT compiler backend of the framework, our approach drives an existing (heavyweight) compiler backend in the framework to quickly generate unoptimized native code by merely embedding directives and compile-time operations into interpreter definitions. As a validation of the approach, we developed 2SOM, a Simple Object Machine with a two-tier JIT compiler based on RPython. 2SOM first applies the tier-1 threaded code generator that is generated by our proposed technique, then, to the loops that exceed a threshold, applies the tier-2 tracing JIT compiler that is generated by the original RPython framework. Our performance evaluation that runs a program with a realistic workload showed that 2SOM improved, when compared against an RPython-based VM, warm-up performance by 15%, with merely a 5% reduction in peak performance.

Problem

• Enables multi-tier JIT compilation in meta-compiler VMs
• Balances compilation overhead and code quality
• Improves warm-up performance with minimal peak loss

Innovation

• Lightweight multi-tier JIT compilation method
• Directives in interpreter for unoptimized code
• Two-tier JIT with threaded and tracing compilers