QONNECT: A QoS-Aware Orchestration System for Distributed Kubernetes Clusters

2025-10-10

AI Summary

Existing Kubernetes schedulers struggle to simultaneously optimize user-defined QoS objectives—such as energy efficiency, cost, and global performance—while lacking automated, declarative orchestration capabilities across heterogeneous cloud-fog-edge clusters. To address this, we propose the first QoS-aware federated orchestration system. Our approach employs a lightweight, Raft-replicated resource agent architecture tightly coupled with a centralized knowledge repository, enabling, for the first time, automatic translation of user-specified YAML-declared multi-dimensional QoS constraints (e.g., latency, energy consumption, cost) into microservice placement and dynamic migration policies. The system integrates Istio service mesh and federated cluster management to support policy-driven scheduling, QoS-compliant rescheduling, and zero-touch failover. Evaluated on a nine-cluster testbed, our system demonstrates both effectiveness and scalability in meeting diverse, cross-layer QoS requirements.

Technology Category

• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Machine Learning: Distributed Machine Learning & Federated Learning

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Modern applications increasingly span across cloud, fog, and edge environments, demanding orchestration systems that can adapt to diverse deployment contexts while meeting Quality-of-Service (QoS) requirements. Standard Kubernetes schedulers do not account for user-defined objectives such as energy efficiency, cost optimization, and global performance, often leaving operators to make manual, cluster-by-cluster placement decisions. To address this need, we present QONNECT, a vendor-agnostic orchestration framework that enables declarative, QoS-driven application deployment across heterogeneous Kubernetes and K3s clusters. QONNECT introduces a distributed architecture composed of a central Knowledge Base, Raft-replicated Resource Lead Agents, and lightweight Resource Agents in each cluster. Through a minimal YAML-based interface, users specify high-level QoS goals, which the system translates into concrete placement and migration actions. Our implementation is evaluated on a federated testbed of up to nine cloud-fog-edge clusters using the Istio Bookinfo microservice application. The system demonstrates dynamic, policy-driven microservice placement, automated failover, QoS-compliant rescheduling, and leader re-election after node failure, all without manual intervention. By bridging the gap between declarative deployment models and operational QoS goals, QONNECT transforms the cloud-edge continuum into a unified, self-optimizing platform.

Problem

• Orchestrating applications across cloud-fog-edge environments
• Addressing QoS requirements like energy efficiency and cost
• Automating placement decisions across distributed Kubernetes clusters

Innovation

• Vendor-agnostic orchestration framework for Kubernetes clusters
• Distributed architecture with central Knowledge Base and agents
• Declarative QoS goals translated into placement actions