Cooldown Landmines: Cross-Tenant Interference Attacks on LLM Gateways
This study addresses cross-tenant interference in LLM gateways caused by shared cooldown records, which adversaries can exploit via fault-handling mechanisms to restrict other tenants from accessing available backends. Specifically, this work identifies two cross-tenant attacks leveraging shared cooldown states and proposes source verification alongside tenant-level cooldown mechanisms to isolate fault impacts. To mitigate RPM-based denial attacks, a source verification strategy is introduced, while tenant-level cooldowns are achieved by retaining shared backend failure records, thereby bridging the isolation gap between request admission and fault handling. Experimental results demonstrate that the proposed approach effectively defends against forced fallback and denial-of-service attacks, reducing the post-recovery victim fallback rate from 54/60 to 0/60.