EdgeCoInfer: Hierarchical Collaborative Inference for On-Device Multimodal Large Models

2026-07-19

AI Summary

This work addresses the dual challenges of task intensity and resource constraints in concurrently executing multimodal large models on edge devices. To this end, the authors propose EdgeCoInfer, a framework that enables adaptive deployment granularity through synergistic optimization of cross-model functional module sharing and intra-model fine-grained partitioning. A novel hybrid evolutionary–hierarchical reinforcement learning (HE-HRL) algorithm is introduced to solve the resulting mixed-integer nonlinear programming problem: it combines genetic algorithms for optimizing discrete model placement with Soft Actor-Critic for continuous resource allocation, augmented by a constructive pruning decoder and a two-stage curriculum strategy to navigate sparse feasible regions. Experiments demonstrate that the approach achieves 100% task completion under high concurrency, reduces system cost by 76% compared to state-of-the-art baselines, and saves 71.88% memory usage.

Abstract

Modern mobile applications predominantly execute concurrent Multimodal Large Language Models (MLLMs) to provide ubiquitous intelligence. However, satisfying this demand within edge environments faces significant challenges due to multi-task concurrency and strictly coupled hard constraints. To address these issues, we propose EdgeCoInfer, a framework enabling granularity-adaptive deployment by co-optimizing inter-model functional module sharing and \textbf{intra-model fine-grained partitioning}. We solve the underlying Mixed-Integer Non-Linear Programming (MINLP) problem via a Hybrid Evolutionary Hierarchical Reinforcement Learning (HE-HRL) paradigm, which synchronizes a Genetic Algorithm (GA) for discrete model placement with a Soft Actor-Critic (SAC) agent for continuous resource allocation. To navigate the sparse feasible region, we introduce a feasibility-guided constructive execution mechanism, integrating a constructive cut-step decoder with pre-act pruning and a two-phase curriculum strategy for stable adaptation. Experimental results demonstrate that EdgeCoInfer ensures a 100\% task completion rate in high-concurrency scenarios, achieving a 76\% reduction in system cost and 71.88\% memory savings compared to state-of-the-art baselines.

Problem

• Multimodal Large Language Models
• Edge Computing
• Model Deployment
• Resource Constraints
• Concurrency

Innovation

• Hierarchical Collaborative Inference
• Granularity-Adaptive Deployment
• Mixed-Integer Non-Linear Programming
• Hybrid Evolutionary Reinforcement Learning
• On-Device Multimodal Large Models