MOMAT: Mixture of Multiple Atlases for Low-Power Jailbreak Defense of Quantized LLMs

2026-10-01

AI Summary

This study addresses the vulnerability of quantized large language models deployed on edge devices to jailbreak attacks, arising from degraded alignment capabilities. To mitigate this, we propose a hardware-enhanced security framework that incorporates a multi-graph hybrid architecture to alleviate semantic sparsity. The approach leverages Computing-in-Memory (CiM) to accelerate structured knowledge retrieval and integrates a lightweight Mixture-of-Experts (MoE) detector to enable domain-specific, localized Retrieval-Augmented Generation (RAG) defense. Experimental results demonstrate that the proposed framework achieves a million-fold improvement in retrieval speed and reduces energy consumption by five orders of magnitude. Furthermore, its defensive performance matches state-of-the-art methods while effectively avoiding the false rejection of benign queries, thereby providing an efficient and reliable security solution for resource-constrained deployment scenarios.

Abstract

Quantized large language models are increasingly deployed on edge devices for their low latency and energy efficiency. However, model quantization weakens alignment safeguards, leaving qLLMs (quantized large language models) highly vulnerable to jailbreak attacks. To address this challenge, we present MOMAT (Mixture of Multiple Atlases), a hardware-enhanced safety framework that combines structured knowledge retrieval with low-power defense acceleration. Each atlas represents a semantic cluster of harmful or benign sample sets and policy templates, enabling domain-localized Retrieval-Augmented Generation guarding that mitigates the curse of dimensionality and the resulting semantic sparsity problem in large, heterogeneous safety databases. MOMAT retrieves top-$k$ similarity features from all atlases for each prompt and evaluates them using a lightweight MoE (Mixture of Experts) detector, while a CiM (Compute-in-Memory)-accelerated similarity engine performs fast, low-power atlas-local retrieval. MOMAT's CiM-based retrieval accelerates a 100-query batch from 15,052.44 ms to 3,207.21 ns (a $4.69 \times 10^6\times$ speedup) and reduces energy from $8.1 \times 10^7$ $μ$J to 3.32 $μ$J, yielding an approximately $2.5 \times 10^5\times$ energy reduction over DRAM-based (Raspberry Pi) baselines. Red-team evaluations across standard benchmarks show that MOMAT matches the defense performance of state-of-the-art methods while avoiding benign overkill and providing substantial efficiency gains, demonstrating that CiM-based modular defenses can make edge-deployed qLLMs both safer and more energy-efficient. We will release the full 223.2k-sample dataset to foster future research.

Problem

• Quantized LLMs
• Jailbreak attacks
• Edge devices
• Safety alignment

Innovation

• Quantized LLMs
• Jailbreak Defense
• Compute-in-Memory (CiM)
• Mixture of Experts (MoE)
• Retrieval-Augmented Generation