Compiler Support for Speculation in Decoupled Access/Execute Architectures

2025-01-23

AI Summary

Irregular code suffers from performance bottlenecks due to memory and communication latency, and conventional decoupled access-execute (DAE) architectures fail to maintain effective decoupling under control dependencies. Method: This paper extends DAE to control-dependent scenarios via a compiler-driven memory speculation mechanism—enabling speculative memory requests and poisoning markers without replay or explicit synchronization—while strictly preserving sequential consistency. The approach integrates control-flow-graph–guided compilation optimizations, speculative address generation, execution-side poisoning detection, and formal verification tailored for reducible control flow. Contribution/Results: Our method overcomes the fundamental limitation of traditional DAE in control-dependent contexts, significantly improving DAE coverage and latency-hiding capability across diverse hardware—including CPU/GPU prefetchers, CGRAs, and domain-specific accelerators. As a result, previously non-decouplable irregular programs achieve substantial performance gains.

Technology Category

• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Machine Learning: Hardware-aware ML
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Security and Privacy: Large-scale security measurements
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization

Abstract

Irregular codes are bottlenecked by memory and communication latency. Decoupled access/execute (DAE) is a common technique to tackle this problem. It relies on the compiler to separate memory address generation from the rest of the program, however, such a separation is not always possible due to control and data dependencies between the access and execute slices, resulting in a loss of decoupling. In this paper, we present compiler support for speculation in DAE architectures that preserves decoupling in the face of control dependencies. We speculate memory requests in the access slice and poison mis-speculations in the execute slice without the need for replays or synchronization. Our transformation works on arbitrary, reducible control flow and is proven to preserve sequential consistency. We show that our approach applies to a wide range of architectural work on CPU/GPU prefetchers, CGRAs, and accelerators, enabling DAE on a wider range of codes than before.

Problem

• Data Transmission Latency
• Irregular Code Processing
• DAE Efficiency

Innovation

• Enhanced DAE Technique
• Complex Code Handling
• Hardware Platform Versatility