Quantization-Aware Collaborative Inference for Large Embodied AI Models

2026-02-13

Technology Category

Machine Learning: Learning on the Edge & Model CompressionIntelligent Robots: Embodied AIConstraint Satisfaction and Optimization: Satisfiability Modulo Theories

Application Category

Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applicationsUser Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendationEconomics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Large artificial intelligence models (LAIMs) are increasingly regarded as a core intelligence engine for embodied AI applications. However, the massive parameter scale and computational demands of LAIMs pose significant challenges for resource-limited embodied agents. To address this issue, we investigate quantization-aware collaborative inference (co-inference) for embodied AI systems. First, we develop a tractable approximation for quantization-induced inference distortion. Based on this approximation, we derive lower and upper bounds on the quantization rate-inference distortion function, characterizing its dependence on LAIM statistics, including the quantization bit-width. Next, we formulate a joint quantization bit-width and computation frequency design problem under delay and energy constraints, aiming to minimize the distortion upper bound while ensuring tightness through the corresponding lower bound. Extensive evaluations validate the proposed distortion approximation, the derived rate-distortion bounds, and the effectiveness of the proposed joint design. Particularly, simulations and real-world testbed experiments demonstrate the effectiveness of the proposed joint design in balancing inference quality, latency, and energy consumption in edge embodied AI systems.

Problem

• Large Artificial Intelligence Models
• Embodied AI
• Quantization
• Collaborative Inference
• Resource Constraints

Innovation

• quantization-aware inference
• collaborative inference
• rate-distortion bounds
• embodied AI
• joint optimization