TopoFE: topology-aware LLM-guided Automated Feature Engineering

2026-07-25

AI Summary

This work addresses the limitations of automatic feature engineering (AutoFE) for tabular data, which stem from the vast search space and the inability of existing large language model (LLM)-based approaches to maintain stateful memory or encourage diverse exploration. To overcome these challenges, the authors propose a topology-aware multi-island evolutionary framework that enhances the diversity, composability, and generalization of feature programs through family-specialized exploration, adaptive prompt memory, and topology-guided knowledge transfer. This approach effectively circumvents the homogenized search and stateless generation bottlenecks inherent in conventional LLM-based feature engineering. Extensive experiments across 29 public tabular datasets demonstrate that the proposed method significantly outperforms current AutoFE techniques in both classification and regression tasks, while also producing feature programs with superior transferability and generality.

Abstract

Automatic feature engineering (AutoFE) for tabular learning can be naturally formulated as a program synthesis problem, where the objective is to discover predictive feature transformations from an exponentially large search space. Recent advances in large language models (LLMs) have expanded the expressiveness of AutoFE by enabling feature program generation beyond predefined operator libraries. However, existing LLM-based approaches remain fundamentally limited by stateless generation and homogeneous search: feature proposals are produced from static prompts without accumulating search experience, while single-population exploration quickly converges to dominant transformation patterns and rarely discovers complementary feature compositions across transformation families. We propose TOPOFE, a topology-aware multi-island evolutionary framework for LLM-guided feature engineering. TOPOFE combines family-specialized exploration, adaptive prompt memory, and topology-guided knowledge transfer to efficiently discover diverse and compositional feature programs. Experiments on 29 public tabular datasets demonstrate consistent improvements over state-of-the-art AutoFE methods across classification and regression tasks. Beyond predictive performance, TOPOFE discovers more diverse and transferable feature programs that generalize across multiple downstream predictors and LLM backbones.

Problem

• Automatic feature engineering
• Large language models
• Program synthesis
• Feature diversity
• Search space exploration

Innovation

• topology-aware
• LLM-guided
• multi-island evolution
• adaptive prompt memory
• feature program synthesis