ES-Trace: Auditing Ethical-Sourcing Disclosure of Code Generation Models Beyond Model Cards

2026-10-06

AI Summary

This study addresses the fragmentation and poor traceability of ethical disclosure evidence in code generation models by proposing the ES-Trace framework. This framework innovatively introduces the Model Documentation Traceability Graph (MDTG), transcending traditional auditing paradigms that rely solely on model cards. By parsing structured metadata to construct a knowledge graph, it enables automated cross-document correlation analysis and consistency verification. Experimental results demonstrate that citation parsing is critical for improving disclosure scores; expanding the audit scope increases the average disclosure score from 1.77 to 2.82 while revealing severe under-documentation regarding social and labor dimensions. Ultimately, this work establishes a reference-aware auditing standard, providing a systematic methodology for quantifying and localizing deficiencies in model ethical disclosures.

Abstract

Code generation models have been increasingly used in software development, but their development raises ethical-sourcing concerns involving intellectual property, privacy, fairness, labour practices, and environmental impact. Although prior work has defined ethical-sourcing criteria for code generation, it remains unclear how much evidence existing models disclose and where that evidence can be found. We introduce ES-Trace, a framework for ethical-sourcing disclosure audits that traces disclosed evidence beyond model cards using the Model Documentation Traceability Graph (MDTG), which represents relationships among models, versions, and documentation artifacts. We apply ES-Trace to 26 models from 10 publishers across 77 documents and 20 ES-CodeGen aspects. Model-card-only auditing yields a mean score of 1.77/5, while resolving the declared references increases it to 2.82/5, with most of the increase arising from documents that the publisher declares in structured metadata. The key findings of our study include: (1) social and labour-related aspects remain poorly documented, even when expanding the audit to the full documentation scope, (2) resolving documentation references substantially increases observed disclosure, raising the mean score from 1.77/5 to 2.82/5, (3) model-card-only audits can mischaracterize release-level disclosure changes, and (4) documentation mismatches can associate evidence with the wrong model or version, highlighting the need for explicit model--version binding and consistency across documentation artifacts. Our study calls for reference-aware ethical-sourcing disclosure audits, explicit model--version binding and consistency across documentation artifacts, and stronger documentation of currently underreported social and labour-related aspects.

Problem

• ethical-sourcing
• code generation models
• disclosure auditing
• model cards
• documentation traceability

Innovation

• Ethical-Sourcing Audit
• Model Documentation Traceability Graph
• Code Generation Models
• Reference-Aware Auditing
• Model-Version Binding