Enforcing Consistency and Fairness in Multi-level Hierarchical Classification with a Mask-based Output Layer

2025-03-19

AI Summary

In multi-level hierarchical classification (MLHC), conventional methods suffer from inconsistent predictions and poor fairness compliance due to independent output layers that ignore inter-class hierarchical relationships. This work proposes a model-agnostic masked output layer, enabling hierarchical consistency and sensitive-attribute fairness enforcement during inference—without modifying the backbone architecture or requiring retraining. Our approach unifies hierarchical constraints and group fairness objectives within a single optimization framework, leveraging a plug-and-play masking mechanism and multi-objective optimization. Evaluated on multiple benchmark datasets, the method significantly improves prediction consistency over baselines, reduces the demographic parity gap (ΔSPD) by 37%, and achieves higher accuracy than both unadjusted models and LLM-based debiasing approaches. The solution is particularly suited for high-reliability applications such as e-commerce and healthcare.

Technology Category

Machine Learning: Hardware-aware MLHumans and AI: Human-in-the-loop Machine LearningComputer Vision: Large Vision Models

Application Category

User Modeling, Personalization and Recommendation: Fairness-aware retrieval and rankingSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingEconomics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Traditional Multi-level Hierarchical Classification (MLHC) classifiers often rely on backbone models with $n$ independent output layers. This structure tends to overlook the hierarchical relationships between classes, leading to inconsistent predictions that violate the underlying taxonomy. Additionally, once a backbone architecture for an MLHC classifier is selected, adapting the model to accommodate new tasks can be challenging. For example, incorporating fairness to protect sensitive attributes within a hierarchical classifier necessitates complex adjustments to maintain the class hierarchy while enforcing fairness constraints. In this paper, we extend this concept to hierarchical classification by introducing a fair, model-agnostic layer designed to enforce taxonomy and optimize specific objectives, including consistency, fairness, and exact match. Our evaluations demonstrate that the proposed layer not only improves the fairness of predictions but also enforces the taxonomy, resulting in consistent predictions and superior performance. Compared to Large Language Models (LLMs) employing in-processing de-biasing techniques and models without any bias correction, our approach achieves better outcomes in both fairness and accuracy, making it particularly valuable in sectors like e-commerce, healthcare, and education, where predictive reliability is crucial.

Problem

• Addresses inconsistency in multi-level hierarchical classification predictions.
• Proposes a fair, model-agnostic layer to enforce taxonomy and fairness.
• Improves fairness and accuracy in hierarchical classification tasks.

Innovation

• Mask-based layer enforces hierarchical classification consistency
• Model-agnostic layer optimizes fairness and taxonomy simultaneously
• Superior fairness and accuracy compared to traditional methods