SPADE: Split-and-Delay Embeddings for Autoregressive High-Granularity Calorimeter Simulation

2026-06-09

AI Summary

This work addresses the challenge of autoregressive generation of high-dimensional token sequences in fine-grained calorimeter simulation, where each token comprises multiple physical features. The authors propose a novel autoregressive Transformer model that introduces a Split-and-Delay mechanism: individual features of each token are independently embedded, and their respective streams are temporally offset to enable standard self-attention to effectively capture intra-token feature dependencies. This approach facilitates efficient and scalable generation of high-dimensional sequences while remaining compatible with large language model–style pretraining paradigms. Evaluated on photon shower simulation in the ILD detector, the model matches the performance of the current state-of-the-art method AllShowers and substantially outperforms its predecessor, OmniJet-α_C.

Abstract

We introduce SPADE (SPlit And Delay Embeddings), an autoregressive transformer for sequences whose tokens carry multiple features. Rather than embedding these features jointly, SPADE embeds them independently. Delaying each feature stream relative to the previous one allows intra-token correlations to be learned by the standard self-attention mechanism. Applied to point-cloud calorimeter shower generation in the highly granular ILD detector, SPADE is competitive with the state of the art AllShowers model on photon showers, and substantially outperforms its VQ-VAE-based predecessor OmniJet-$α_C$. The mechanism is applicable to any generative task with multi-feature tokens, enabling LLM-style pretraining workflows for higher-dimensional data.

Problem

• autoregressive generation
• multi-feature tokens
• calorimeter simulation
• high-granularity detectors
• point-cloud generation

Innovation

• Split-and-Delay Embeddings
• autoregressive transformer
• multi-feature tokens
• calorimeter simulation
• self-attention mechanism