Process Faster, Pay Less: Functional Isolation for Stream Processing

2026-03-19

AI Summary

This work addresses the inefficiency of query isolation and the performance interference caused by coarse-grained resource sharing in stream processing systems. To reconcile these challenges, the authors propose FunShare, a system that dynamically groups queries with similar performance characteristics to jointly optimize functional isolation and resource sharing. FunShare introduces an adaptive framework that continuously monitors runtime metrics and dynamically adjusts query groupings and execution strategies. It uniquely integrates dynamic query grouping, estimation of computational overlap, and online reconstruction of execution plans to enhance resource efficiency while preserving per-query quality of service. Experimental results demonstrate that, compared to fully isolated execution, FunShare substantially reduces resource consumption while maintaining or even improving the throughput of all queries.

Technology Category

• Data Mining & Knowledge Management: Intelligent Query Processing
• Search and Optimization: Distributed Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Economics, Online Markets and Human Computation: The sharing economy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Concurrent workloads often extract insights from high-throughput, real-time data streams. Existing stream processing engines isolate each query's resources, ensuring robust performance but incurring high infrastructure costs. In contrast, sharing work reduces the amount of necessary resources but introduces inter-query interference, leading to performance degradation for some queries. We introduce FunShare, a stream-processing system that improves resource efficiency without compromising performance by dynamically grouping queries based on their performance characteristics. FunShare strategically relaxes query interdependencies and minimizes redundant computation while preserving individual query performance. It achieves this by using an adaptive optimization framework that monitors execution metrics, accurately estimates computation overlaps, and reconfigures execution plans on the fly in response to changes in the underlying data streams. Our evaluation demonstrates that FunShare minimizes resource consumption compared to isolated execution while maintaining or improving throughput for all queries.

Problem

• stream processing
• resource efficiency
• query interference
• functional isolation
• performance degradation

Innovation

• functional isolation
• stream processing
• query grouping
• adaptive optimization
• resource efficiency