Benchmarking LLM Compliance with China AI Generated Content Regulations

2026-09-17

AI Summary

论文针对中文内容合规性问题，设计了一个包含2303个问题的评估框架，用于测试20个著名语言模型的合规性和拒绝率，以符合中国的AI生成内容法规。

Abstract

The widespread adoption of LLMs has led to escalating content compliance risks. Prior works have contributed to addressing these risks in the English context, downplaying the complexity of Chinese language content. This paper follows China's current AI-Generated content compliance requirements and provides evaluation results on 20 notable LLMs, offering insight into China's regulatory landscape. We design a novel framework to assess the compliance and refusal rates with 2303 questions spanning six distinct dimensions, including 203 self-constructed constitutional questions. The framework employs several judges to generate verdicts independently based on their hierarchical alignment memory. Our findings show that international models also exhibit high levels of compliance despite the use of standard Chinese questions, and the main differences may stem from dimensions closely related to ideological alignment. We establish a regulatory benchmark that enables the global AI community to evaluate both Chinese and non-Chinese LLMs under a unified set of legally grounded compliance requirements.

Problem

• LLM Compliance
• China AI Regulations
• Content Compliance Risks
• Chinese Language Complexity

Innovation

• novel framework
• compliance evaluation
• hierarchical alignment memory
• ideological alignment