Technology Category
Natural Language Processing: (Large) Language ModelsData Mining & Knowledge Management: Scalability, Parallel & Distributed SystemsMachine Learning: Large Multimodal Models (LMMs)
DeInfer: Efficient Parallel Inferencing for Decomposed Large Language Models
🤖 AI Summary
This work addresses the bottleneck of low parallel inference efficiency that hinders the scalability of existing decomposed large language models. To overcome this limitation, we propose DeInfer, the first high-performance parallel inference system tailored for decomposed large language models. DeInfer systematically enhances inference throughput and scalability through a co-design of multi-level parallelism strategies, communication optimizations, and integration with established acceleration techniques. Experimental results demonstrate that DeInfer consistently outperforms current baselines across diverse configurations, effectively enabling efficient and scalable parallel inference for large-scale decomposed models.
Technology Category
Application Category
📝 Abstract
Existing works on large language model (LLM) decomposition mainly focus on improving performance on downstream tasks, but they ignore the poor parallel inference performance when trying to scale up the model size. To mitigate this important performance issue, this paper introduces DeInfer, a high-performance inference system dedicated to parallel inference of decomposed LLMs. It consists of multiple optimizations to maximize performance and be compatible with state-of-the-art optimization techniques. Extensive experiments are carried out to evaluate DeInfer's performance, where the results demonstrate its superiority, suggesting it can greatly facilitate the parallel inference of decomposed LLMs.
Problem
Research questions and friction points this paper is trying to address.
large language models
model decomposition
parallel inference
inference performance
scalability
Innovation
Methods, ideas, or system contributions that make the work stand out.
parallel inference
model decomposition
large language models
high-performance system
efficient inference
🔎 Similar Papers
No similar papers found.
