Agentic Evaluation of Copyright Law Compliance

2026-07-23

AI Summary

This work addresses the lack of effective frameworks for evaluating copyright compliance of large language model (LLM) agents in commercial tasks. We introduce Copyright-Bench, the first benchmark specifically designed for assessing LLM agents’ adherence to copyright norms, leveraging realistic scenarios such as website development, product design, and fundraising presentations. The benchmark incorporates tasks requiring selection between public-domain and copyrighted materials, while integrating real-world variables like user preferences and time pressure. Through multi-scenario prompt engineering, controlled experiments, and comparison against human baselines, we systematically evaluate both open- and closed-source models. Our findings reveal that prevailing LLM agents frequently select copyrighted content even when viable public-domain alternatives exist, and that open-source models exhibit significantly higher violation rates under certain conditions, highlighting latent copyright risks in complex instruction-following settings.

Abstract

Large language model (LLM) agents increasingly perform commercial tasks that involve retrieving external content such as images and, where appropriate, reproducing that content. LLM agents should comply with the law, including copyright law. Presently, however, we lack adequate frameworks to assess whether they do so in practice. To that end, we introduce \textbf{Copyright-Bench}, a benchmark designed to evaluate \textit{LLM agents' compliance with} \emph{copyright law}. Copyright-Bench is comprised of realistic commercial tasks---website development, merchandise design, and pitch deck production---that involve agents selecting between public-domain content (the use of which is \textit{legal}) and copyrighted content (the use of which is \textit{infringing} in this setting).The evaluation introduces prompt variations that simulate different user preferences, as well as time pressure.Comparing state-of-the-art LLM agents against a human baseline, we find that: (1) agents select copyrighted works despite the availability of public-domain alternatives; and (2) for open-weights models, violation rates increase in response to certain user preferences and simulated time pressure.

Problem

• copyright law compliance
• LLM agents
• evaluation framework
• content usage
• legal compliance

Innovation

• Copyright-Bench
• LLM agents
• copyright compliance
• legal evaluation
• public-domain content