Beyond Monoliths: Expert Orchestration for More Capable, Democratic, and Safe Large Language Models

2025-05-28

AI Summary

Current large-model ecosystems are dominated by a few tech giants deploying monolithic general-purpose models, constraining innovation, inflating operational costs, and compromising security and controllability. To address this, we propose the “Expert Collaboration Framework,” which replaces the monolithic paradigm with dynamic orchestration of thousands of open-source specialized models, guided by fine-grained capability assessment. Our contributions are threefold: (1) a novel capability-centric (rather than architecture-centric) model selection paradigm; (2) lightweight, interpretable judge models that quantify model competencies across multiple dimensions; and (3) a decentralized, auditable, and user-controllable collaborative inference ecosystem. Experiments demonstrate that our framework outperforms monolithic models under equivalent computational budgets across diverse benchmarks, while substantially reducing inference cost, enhancing transparency, improving alignment and security, and lowering barriers to participation for small- and medium-sized organizations.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This position paper argues that the prevailing trajectory toward ever larger, more expensive generalist foundation models controlled by a handful of big companies limits innovation and constrains progress. We challenge this approach by advocating for an"Expert Orchestration"framework as a superior alternative that democratizes LLM advancement. Our proposed framework intelligently selects from thousands of existing models based on query requirements and decomposition, focusing on identifying what models do well rather than how they work internally. Independent"judge"models assess various models' capabilities across dimensions that matter to users, while"router"systems direct queries to the most appropriate specialists within an approved set. This approach delivers superior performance by leveraging targeted expertise rather than forcing costly generalist models to address all user requirements. The expert orchestration paradigm represents a significant advancement in LLM capability by enhancing transparency, control, alignment, and safety through model selection while fostering a more democratic ecosystem.

Problem

• Challenges dominance of large generalist models controlled by few companies
• Proposes expert orchestration to democratize LLM advancement
• Enhances transparency, control, and safety via model selection

Innovation

• Expert Orchestration framework democratizes LLM advancement
• Judge models assess capabilities across user dimensions
• Router systems direct queries to specialist models