Handling out-of-order input arrival in CEP engines on the edge combining optimistic, pessimistic and lazy evaluation

2025-07-02

AI Summary

In edge computing, Complex Event Processing (CEP) engines face significant challenges from out-of-order, delayed, and duplicate events—particularly when processing multi-source heterogeneous data on resource-constrained devices, where real-time performance conflicts with accuracy. To address this, we propose LimeCEP, a lightweight hybrid CEP architecture. LimeCEP innovatively integrates lazy evaluation, adaptive buffering, and speculative execution; leverages Kafka for low-overhead message reordering and deduplication; and supports three configurable evaluation strategies—optimistic, pessimistic, and lazy—to dynamically trade off accuracy, latency, and resource consumption. Experimental evaluation demonstrates that, compared to SASE and FlinkCEP, LimeCEP reduces end-to-end latency by up to six orders of magnitude, cuts memory footprint by 10×, and lowers CPU utilization by 6×, while maintaining >99.8% precision and recall.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

In Complex Event Processing, handling out-of-order, late, and duplicate events is critical for real-time analytics, especially on resource-constrained devices that process heterogeneous data from multiple sources. We present LimeCEP, a hybrid CEP approach that combines lazy evaluation, buffering, and speculative processing to efficiently handle data inconsistencies while supporting multi-pattern detection under relaxed semantics. LimeCEP integrates Kafka for efficient message ordering, retention, and duplicate elimination, and offers configurable strategies to trade off between accuracy, latency, and resource consumption. Compared to state-of-the-art systems like SASE and FlinkCEP, LimeCEP achieves up to six orders of magnitude lower latency, with up to 10 times lower memory usage and 6 times lower CPU utilization, while maintaining near-perfect precision and recall under high-disorder input streams, making it well-suited for non-cloud deployments.

Problem

• Handling out-of-order event processing in edge CEP engines
• Optimizing resource use for multi-pattern event detection
• Balancing accuracy and latency in real-time analytics

Innovation

• Hybrid CEP combining lazy and speculative evaluation
• Kafka integration for message ordering and deduplication
• Configurable strategies balancing accuracy and resource usage