Hercules: A Compiler for Productive Programming of Heterogeneous Systems

2025-03-13

AI Summary

Heterogeneous computing programming suffers from tight coupling between algorithmic logic and hardware-specific details, leading to high cross-platform porting costs. This paper introduces Junio, a domain-specific language, and Hercules, a compiler framework that decouples algorithmic logic from device-specific optimizations—such as data partitioning, inter-device communication, and synchronization—via a novel customized intermediate representation (IR) and a fine-grained imperative scheduling language. The framework enables unified CPU/GPU modeling and cross-device co-optimization. Its core innovation is an imperative scheduling-driven mechanism that automatically generates optimal partitioning and communication code. Evaluation shows Junio achieves 92% of hand-optimized performance, delivering average speedups of 1.25× on CPUs and 1.48× on GPUs over baseline implementations. It significantly outperforms mainstream general-purpose heterogeneous programming models, achieving 9.31× and 16.18× higher performance on CPUs and GPUs, respectively.

Technology Category

• Machine Learning: Hardware-aware ML
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Computer Vision: Learning & Optimization for CV

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

Modern computing systems increasingly rely on composing heterogeneous devices to improve performance and efficiency. Programming these systems is often unproductive: algorithm implementations must be coupled to system-specific logic, including device-specific optimizations, partitioning, and inter-device communication and synchronization, which requires developing different programs for different system configurations. We propose the Juno language, which represents general purpose applications in an imperative form that can be transformed into parallel, optimized, system-specific code using an expressive and granular imperative scheduling language. We also introduce the Hercules compiler, which uses a novel intermediate representation to represent general and device-specific parallel code in a manner that is easy to analyze and manipulate using schedules. Our system achieves competitive performance with hand-optimized device-specific code (geomean speedups of $1.25 imes$ and $1.48 imes$ on the CPU and GPU) and significantly outperforms a prior general purpose heterogeneous programming system (geomean speedups of $9.31 imes$ and $16.18 imes$ on the CPU and GPU).

Problem

• Addresses unproductive programming of heterogeneous computing systems.
• Eliminates need for system-specific logic in algorithm implementations.
• Improves performance with competitive speedups on CPU and GPU.

Innovation

• Juno language enables imperative application transformation.
• Hercules compiler uses novel intermediate representation.
• Achieves competitive performance with hand-optimized code.