Monet: Measuring the Ecosystem of Open-Source Text-to-Image Models Tailored for Harmful Services

2026-09-21

AI Summary

研究通过构建有害服务分类和跨平台分析，系统测量了开源文本到图像模型生态系统中存在的针对有害服务定制的模型（Monets），揭示了其传播、规避治理及商业化问题。

Abstract

The open-source text-to-image (T2I) ecosystem enables rapid model development and sharing, but also hosts models intentionally tailored for harmful services, which we call Monets. Prior work has examined specific types of harmful T2I models on individual platforms, but a Monet does not exist in isolation. The broader Monet ecosystem, spanning model characteristics, cross-platform propagation, governance evasion, monetization, and downstream deployment, remains poorly understood. In this study, we present the first systematic, ecosystem-level measurement of Monets. Grounded in the policies of real-world model hubs, we construct a taxonomy of ten harmful service categories and identify 23,947 Monets across eight major T2I model hubs, with the most popular exceeding 19 million downloads. While some developers employ anti-theft mechanisms against unauthorized re-uploading, Monets propagate across platforms at scale, with 40.76% mirrored across hubs. Such propagation further enables governance evasion via cross-platform archiving, keeping 11.99% of Monets accessible after bans on their original platforms, alongside other evasion strategies including keyword obfuscation and model-level safeguard circumvention. Monets also anchor coordinated commercial campaigns---one spanning 668 models with 914 completed commissions and another advertising gray-market account-farming service---and reach users through GitHub projects and inference APIs, raising downstream child safety concerns. These findings expose the limitations of platform-siloed defenses and highlight the need for cross-platform threat intelligence, coordinated governance, and technical safeguards.

Problem

• text-to-image models
• harmful services
• ecosystem
• governance evasion
• cross-platform propagation

Innovation

• Ecosystem-level Measurement
• Cross-platform Propagation
• Governance Evasion
• Monetization Strategies