Saarthi: An End-to-End Intelligent Platform for Optimising Distributed Serverless Workloads

2025-11-10

AI Summary

FaaS platforms suffer from high cold-start latency, static resource provisioning, and inefficient scheduling in distributed environments, leading to performance volatility and hidden operational costs. To address these challenges, this paper proposes an input-aware intelligent serverless platform: (1) a request-feature-based function resource estimation mechanism; (2) a multi-objective integer linear programming (ILP) model jointly optimizing throughput, cost, and SLO compliance; and (3) a proactive fault-tolerant redundancy strategy enabling request-level dynamic orchestration. Implemented on OpenFaaS, experimental evaluation demonstrates a 1.45× throughput improvement, a 1.84× reduction in per-request cost, 98.3% SLO attainment rate, and tail latency increase bounded by ≤0.2 seconds versus baselines. The core contribution lies in the tight integration of input-aware prediction, multi-objective ILP-driven scheduling, and lightweight redundancy management—enabling, for the first time, self-driving, high-cost-efficiency execution of FaaS workloads.

Technology Category

• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Hardware-aware ML
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

FaaS offers significant advantages with its infrastructure abstraction, on-demand execution, and attractive no idle resource pricing for modern cloud applications. Despite these benefits, challenges such as startup latencies, static configurations, sub-optimal resource allocation and scheduling still exist due to coupled resource offering and workload-agnostic generic scheduling behaviour. These issues often lead to inconsistent function performance and unexpected operational costs for users and service providers. This paper introduces Saarthi, a novel, end-to-end serverless framework that intelligently manages the dynamic resource needs of function workloads, representing a significant step toward self-driving serverless platforms. Unlike platforms that rely on static resource configurations, Saarthi is input-aware, allowing it to intelligently anticipate resource requirements based on the characteristics of an incoming request payload. This input-driven approach reinforces function right-sizing and enables smart request orchestration across available function configurations. Saarthi further integrates a proactive fault-tolerant redundancy mechanism and employs a multi-objective Integer Linear Programming (ILP) model to maintain an optimal function quantity. This optimisation aims to maximise system throughput while simultaneously reducing overall operational costs. We validate the effectiveness of Saarthi by implementing it as a framework atop OpenFaaS. Our results demonstrate Saarthi's ability to achieve up to 1.45x better throughput, 1.84x reduced costs, while maintaining up to 98.3% service level targets with an overhead of up to 0.2 seconds as compared to the baseline OpenFaaS.

Problem

• Optimizing resource allocation for serverless workloads to reduce costs
• Improving function performance by addressing startup latencies and static configurations
• Enhancing throughput while maintaining service level targets in FaaS platforms

Innovation

• Input-aware dynamic resource allocation for serverless workloads
• Proactive fault-tolerant redundancy mechanism integration
• Multi-objective ILP optimization for throughput and cost