On the (In)Significance of Feature Selection in High-Dimensional Datasets

2025-08-05

AI Summary

The necessity and efficacy of feature selection (FS) in high-dimensional gene expression classification remain widely assumed but insufficiently validated empirically. Many computational FS studies select genes without experimental verification, raising methodological concerns. Method: We propose a hypothesis-testing framework to systematically compare the classification performance of small, randomly sampled feature subsets (0.02%–1% of total features) against those selected by classical FS algorithms and the full feature set. Contribution/Results: On most benchmark gene expression datasets, classifiers trained on random feature subsets achieve accuracy comparable to—or even exceeding—that of FS-based and full-feature models. These findings challenge the prevailing assumption that FS inherently improves predictive performance. They expose potential methodological risks in computation-driven gene selection practices and provide empirical evidence urging critical reevaluation of FS conventions in biomedical feature engineering.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Search and Optimization: Metareasoning and Metaheuristics
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Extensive research has been done on feature selection (FS) algorithms for high-dimensional datasets aiming to improve model performance, reduce computational cost and identify features of interest. We test the null hypothesis of using randomly selected features to compare against features selected by FS algorithms to validate the performance of the latter. Our results show that FS on high-dimensional datasets (in particular gene expression) in classification tasks is not useful. We find that (1) models trained on small subsets (0.02%-1% of all features) of randomly selected features almost always perform comparably to those trained on all features, and (2) a "typical"- sized random subset provides comparable or superior performance to that of top-k features selected in various published studies. Thus, our work challenges many feature selection results on high dimensional datasets, particularly in computational genomics. It raises serious concerns about studies that propose drug design or targeted interventions based on computationally selected genes, without further validation in a wet lab.

Problem

• Evaluates effectiveness of feature selection in high-dimensional datasets
• Compares random feature subsets to FS algorithm results
• Challenges validity of gene selection in computational genomics

Innovation

• Testing random features vs FS algorithms
• Random subsets match FS performance
• Challenges FS utility in genomics