An Efficient Variant of One-Class SVM with Lifelong Online Learning Guarantees

2025-12-11

AI Summary

Traditional kernel-based One-Class SVM (OCSVM) suffers from high computational overhead, elevated false negative rates, and poor adaptability to distributional drift in single-pass non-stationary streaming data. Method: We propose SONAR—a computationally efficient online OCSVM solver leveraging stochastic gradient descent (SGD) and strongly convex regularization, designed for lifelong learning. It integrates ensemble learning and change-point detection to enhance robustness against both benign and adversarial non-stationarity. Contribution/Results: SONAR establishes the first tight theoretical bounds on Type I and Type II errors for OCSVM-type methods. Its lifelong learning mechanism enables adaptive model evolution under evolving data distributions. Empirically, SONAR significantly reduces both false positive and false negative rates on synthetic and real-world benchmarks, while achieving 10–100× speedup over kernel OCSVM.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

We study outlier (a.k.a., anomaly) detection for single-pass non-stationary streaming data. In the well-studied offline or batch outlier detection problem, traditional methods such as kernel One-Class SVM (OCSVM) are both computationally heavy and prone to large false-negative (Type II) errors under non-stationarity. To remedy this, we introduce SONAR, an efficient SGD-based OCSVM solver with strongly convex regularization. We show novel theoretical guarantees on the Type I/II errors of SONAR, superior to those known for OCSVM, and further prove that SONAR ensures favorable lifelong learning guarantees under benign distribution shifts. In the more challenging problem of adversarial non-stationary data, we show that SONAR can be used within an ensemble method and equipped with changepoint detection to achieve adaptive guarantees, ensuring small Type I/II errors on each phase of data. We validate our theoretical findings on synthetic and real-world datasets.

Problem

• Detects outliers in non-stationary streaming data efficiently
• Reduces Type I/II errors compared to traditional One-Class SVM
• Ensures adaptive performance under distribution shifts and adversarial changes

Innovation

• SGD-based OCSVM solver with convex regularization
• Lifelong learning guarantees under distribution shifts
• Ensemble method with changepoint detection for adaptation