A Large-Scale Measurement of AI Bill of Materials Completeness in Hugging Face Models

2026-07-19

AI Summary

This study addresses the widespread incompleteness of machine-readable documentation in current AI model repositories, which significantly undermines transparency and governance across the AI supply chain—particularly concerning provenance, licensing, datasets, and usage restrictions. Focusing on approximately 97.5K public models hosted on Hugging Face, this work presents the first large-scale quantitative assessment of AI Bill of Materials (AIBOM) completeness. Through structured parsing, metadata extraction, and categorical analysis, it systematically evaluates coverage of critical fields such as model identity, licenses, and model cards. The findings reveal that while basic AIBOM structures are largely present, essential content—including responsible use guidelines, environmental impact disclosures, and usage limitations—is frequently missing. To address these gaps, the paper advocates for enhanced model card practices and automated validation mechanisms to advance standardized, comprehensive documentation throughout the AI supply chain.

Abstract

Pretrained machine learning (ML) models help developers build ML-intensive software systems without training models from scratch. However, model repositories often provide incomplete machine-readable documentation about model provenance, licenses, datasets, limitations, and external references, creating transparency and governance gaps across the AI supply chain. Artificial Intelligence Bills of Materials (AIBOMs) address these gaps by documenting AI artifacts, including models, metadata, licenses, datasets, model-card information, and external references. Taking public Hugging Face (HF) model repositories as a case study, this paper empirically investigates AIBOM completeness, defined as the extent to which repositories provide AIBOM-relevant information for machine-readable AI supply-chain documentation. We examine approximately 97.5K AIBOM artifacts to assess the extent to which generated AIBOMs: (i) contain required structural and metadata fields, (ii) represent model identity, license, and external-reference information, (iii) capture model-card documentation such as datasets, limitations, safety-risk assessment, and environmental information, and (iv) vary in documentation coverage across repository and artifact characteristics such as task, license availability, dataset declaration, model family, and paper reference. Results indicate that generated AIBOMs provide complete coverage of required AIBOM structure but limited AI-specific documentation completeness. Required fields are fully represented, but model-card, metadata, responsible-use, environmental, limitation, and meaningful-description fields remain weakly represented or missing across generated artifacts. Our findings motivate improved model-card practices, repository-level traceability, and automated AIBOM validation to advance the generation and adoption of more complete AIBOMs.

Problem

• AI Bill of Materials
• model documentation
• transparency
• governance
• machine learning models

Innovation

• AI Bill of Materials
• model card
• transparency
• machine learning supply chain
• empirical measurement