FairLMs: A Turnkey Library for Fairness in Language Models

2026-09-18

AI Summary

该论文提出FairLMs库，通过统一模型接口和规范声明解决语言模型公平性研究中的工具兼容问题，提供多种度量、缓解方法及适配器。

Abstract

Fairness research on language models involves measuring bias, applying mitigation methods, and examining the evidence on which an evaluation rests. Existing tools offer complementary functionality through different interfaces, so combining them requires reconciling model interfaces, evidence formats, access constraints, and result types before applicability can be checked or methods compared. We introduce \textbf{FairLMs}, a Python library that connects these activities through explicit declarations of model capabilities and input requirements. It provides 33 intrinsic and extrinsic metrics, 14 mitigation components spanning four intervention categories, 14 dataset and scoring-instrument diagnostics, adapters for the three Transformer architectures and supported hosted completion APIs, and benchmark loaders. Declarations are checked before execution and results carry the configuration under which they were obtained, so that compatible components can be combined, methods compared under a common protocol, and workflows extended to new models and datasets. The source code is available at: https://github.com/FairLMs/FairLMs.

Problem

• Fairness
• Language Models
• Bias Measurement
• Mitigation Methods
• Tool Integration

Innovation

• Fairness in Language Models
• Python Library
• Model Capabilities
• Mitigation Components
• Transformer Architectures