Automation and Feature Selection Enhancement with Reinforcement Learning (RL)

2025-03-15

AI Summary

To address the low efficiency, poor generalization, and weak interpretability of feature selection in high-dimensional machine learning, this paper proposes an automated feature engineering framework based on reinforcement learning (RL). We introduce Monte Carlo Reinforcement Feature Selection (MCRFS), a novel RL algorithm instantiated in three architectures: single-agent, dual-agent cooperative, and cascaded multi-stage RL. A hybrid state representation is designed, combining sequential scanning with convolutional autoencoders; additionally, we integrate bandit-based action selection, early stopping, and hierarchical reward shaping. Extensive experiments across diverse high-dimensional benchmark datasets demonstrate that our method significantly improves feature selection quality and computational efficiency, while enhancing downstream model generalization and interpretability. Empirical results show consistent superiority over conventional feature engineering approaches, including filter, wrapper, and embedded methods.

Technology Category

Machine Learning: Feature Construction/ReformulationSearch and Optimization: Learning to SearchMultiagent Systems: Multiagent Learning

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingResponsible Web: Machine-in-the-loop, human agency and autonomyUser Modeling, Personalization and Recommendation: ML for personalized search and recommendations

Abstract

Effective feature selection, representation and transformation are principal steps in machine learning to improve prediction accuracy, model generalization and computational efficiency. Reinforcement learning provides a new perspective towards balanced exploration of optimal feature subset using multi-agent and single-agent models. Interactive reinforcement learning integrated with decision tree improves feature knowledge, state representation and selection efficiency, while diversified teaching strategies improve both selection quality and efficiency. The state representation can further be enhanced by scanning features sequentially along with the usage of convolutional auto-encoder. Monte Carlo-based reinforced feature selection(MCRFS), a single-agent feature selection method reduces computational burden by incorporating early-stopping and reward-level interactive strategies. A dual-agent RL framework is also introduced that collectively selects features and instances, capturing the interactions between them. This enables the agents to navigate through complex data spaces. To outperform the traditional feature engineering, cascading reinforced agents are used to iteratively improve the feature space, which is a self-optimizing framework. The blend of reinforcement learning, multi-agent systems, and bandit-based approaches offers exciting paths for studying scalable and interpretable machine learning solutions to handle high-dimensional data and challenging predictive tasks.

Problem

• Enhance feature selection and representation using reinforcement learning.
• Improve computational efficiency and model generalization in machine learning.
• Handle high-dimensional data with scalable and interpretable solutions.

Innovation

• Reinforcement learning enhances feature selection efficiency.
• Monte Carlo method reduces computational burden effectively.
• Dual-agent RL framework captures feature-instance interactions.