Exploring the Design Space of Fair Tree Learning Algorithms

2025-09-03

AI Summary

This study addresses decision tree learning under fairness constraints, aiming to jointly optimize predictive accuracy and group fairness. To overcome limitations in existing modeling paradigms, we propose and empirically validate two novel frameworks: (1) a dual-tree architecture—separately learning a prediction tree and a sensitive-attribute tree; and (2) a backtracking-aware constrained tree construction—integrating weighted information gain optimization, greedy fairness-constrained pruning, and backtracking search. These approaches jointly mitigate the fairness–accuracy trade-off. Extensive experiments on multiple benchmark datasets demonstrate that our methods substantially improve key fairness metrics—including statistical parity and equal opportunity—while preserving competitive classification accuracy. Our work fills a critical gap in the design space of fair decision trees, offering principled, flexible, and empirically effective solutions for fairness-aware tree induction.

Technology Category

• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition
• Machine Learning: Ethics, Bias, and Fairness
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Social Networks and Social Media: Fairness and bias in social network and social media analysis

Abstract

Decision trees have been studied extensively in the context of fairness, aiming to maximize prediction performance while ensuring non-discrimination against different groups. Techniques in this space usually focus on imposing constraints at training time, constraining the search space so that solutions which display unacceptable values of relevant metrics are not considered, discarded, or discouraged. If we assume one target variable y and one sensitive attribute s, the design space of tree learning algorithms can be spanned as follows: (i) One can have one tree T that is built using an objective function that is a function of y, s, and T. For instance, one can build a tree based on the weighted information gain regarding y (maximizing) and s (minimizing). (ii) The second option is to have one tree model T that uses an objective function in y and T and a constraint on s and T. Here, s is no longer part of the objective, but part of a constraint. This can be achieved greedily by aborting a further split as soon as the condition that optimizes the objective in y fails to satisfy the constraint on s. A simple way to explore other splits is to backtrack during tree construction once a fairness constraint is violated. (iii) The third option is to have two trees T_y and T_s, one for y and one for s, such that the tree structure for y and s does not have to be shared. In this way, information regarding y and regarding s can be used independently, without having to constrain the choices in tree construction by the mutual information between the two variables. Quite surprisingly, of the three options, only the first one and the greedy variant of the second have been studied in the literature so far. In this paper, we introduce the above two additional options from that design space and characterize them experimentally on multiple datasets.

Problem

• Design fair decision trees without discrimination
• Explore constrained optimization for fairness metrics
• Develop dual-tree models for independent variable handling

Innovation

• Objective function combines y and s
• Constraint on s during greedy splitting
• Separate trees for y and s