Improving Performance in Classification Tasks with LCEN and the Weighted Focal Differentiable MCC Loss

2026-04-22

AI Summary

This work extends the LCEN algorithm—originally designed for regression tasks—to classification while preserving its sparsity and feature interpretability, substantially enhancing classification performance. To this end, we introduce a weighted focal differentiable Matthews Correlation Coefficient (diffMCC) loss function as a replacement for conventional weighted cross-entropy. Experimental results across four classification datasets demonstrate that the enhanced LCEN model eliminates an average of 56% of redundant features while outperforming most baseline methods. Moreover, models trained with the diffMCC loss achieve average improvements of 4.9% in macro F₁ score and 8.5% in MCC, underscoring the effectiveness of the proposed approach.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

The LASSO-Clip-EN (LCEN) algorithm was previously introduced for nonlinear, interpretable feature selection and machine learning. However, its design and use was limited to regression tasks. In this work, we create a modified version of the LCEN algorithm that is suitable for classification tasks and maintains its desirable properties, such as interpretability. This modified LCEN algorithm is evaluated on four widely used binary and multiclass classification datasets. In these experiments, LCEN is compared against 10 other model types and consistently reaches high test-set macro F$_1$ score and Matthews correlation coefficient (MCC) metrics, higher than that of the majority of investigated models. LCEN models for classification remain sparse, eliminating an average of 56% of all input features in the experiments performed. Furthermore, LCEN-selected features are used to retrain all models using the same data, leading to statistically significant performance improvements in three of the experiments and insignificant differences in the fourth when compared to using all features or other feature selection methods. Simultaneously, the weighted focal differentiable MCC (diffMCC) loss function is evaluated on the same datasets. Models trained with the diffMCC loss function are always the best-performing methods in these experiments, and reach test-set macro F$_1$ scores that are, on average, 4.9% higher and MCCs that are 8.5% higher than those obtained by models trained with the weighted cross-entropy loss. These results highlight the performance of LCEN as a feature selection and machine learning algorithm also for classification tasks, and how the diffMCC loss function can train very accurate models, surpassing the weighted cross-entropy loss in the tasks investigated.

Problem

• classification
• feature selection
• interpretability
• sparsity
• LCEN

Innovation

• LCEN
• classification
• feature selection
• differentiable MCC loss
• interpretability