Online Meta-learning for AutoML in Real-time (OnMAR)

2025-02-27

AI Summary

Existing real-time AutoML approaches struggle to balance model quality and configuration efficiency. This paper introduces the first online meta-learning framework tailored for real-time scenarios, which dynamically extracts task-specific meta-features and jointly employs a meta-learner to predict configuration quality in real time; lightweight genetic algorithm optimization is triggered on-demand to iteratively enhance model performance. The meta-learner integrates k-nearest neighbors, random forests, and XGBoost, ensuring strong cross-task generalization. Evaluated on three real-time tasks—image clustering, CNN hyperparameter search, and video classification—the method matches or exceeds baseline accuracy while reducing average runtime by 37.2%–58.6%. These results significantly improve the practicality and scalability of real-time AutoML systems.

Technology Category

Machine Learning: Auto ML and Hyperparameter TuningSearch and Optimization: Metareasoning and MetaheuristicsHumans and AI: Human-in-the-loop Machine Learning

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingSystems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applicationsUser Modeling, Personalization and Recommendation: ML for personalized search and recommendations

Abstract

Automated machine learning (AutoML) is a research area focusing on using optimisation techniques to design machine learning (ML) algorithms, alleviating the need for a human to perform manual algorithm design. Real-time AutoML enables the design process to happen while the ML algorithm is being applied to a task. Real-time AutoML is an emerging research area, as such existing real-time AutoML techniques need improvement with respect to the quality of designs and time taken to create designs. To address these issues, this study proposes an Online Meta-learning for AutoML in Real-time (OnMAR) approach. Meta-learning gathers information about the optimisation process undertaken by the ML algorithm in the form of meta-features. Meta-features are used in conjunction with a meta-learner to optimise the optimisation process. The OnMAR approach uses a meta-learner to predict the accuracy of an ML design. If the accuracy predicted by the meta-learner is sufficient, the design is used, and if the predicted accuracy is low, an optimisation technique creates a new design. A genetic algorithm (GA) is the optimisation technique used as part of the OnMAR approach. Different meta-learners (k-nearest neighbours, random forest and XGBoost) are tested. The OnMAR approach is model-agnostic (i.e. not specific to a single real-time AutoML application) and therefore evaluated on three different real-time AutoML applications, namely: composing an image clustering algorithm, configuring the hyper-parameters of a convolutional neural network, and configuring a video classification pipeline. The OnMAR approach is effective, matching or outperforming existing real-time AutoML approaches, with the added benefit of a faster runtime.

Problem

• Improves real-time AutoML design quality
• Reduces time for real-time AutoML design
• Enhances optimization using meta-learning techniques

Innovation

• Uses meta-learning for real-time AutoML
• Applies genetic algorithm for optimization
• Tests multiple meta-learners for accuracy prediction