Automated Boilerplate: Prevalence and Quality of Contract Generators in the Context of Swiss Privacy Policies

2025-10-07

AI Summary

Small and medium-sized enterprises (SMEs) face disproportionately high compliance costs and limited capacity in adapting to emerging digital regulations—such as the 2023 Swiss privacy law revision—while empirical evidence on the adoption and efficacy of automated contract generation tools remains scarce. Method: We construct a multilingual legal compliance benchmark dataset and propose the first large-scale, cross-lingual automated compliance assessment framework powered by GPT-5, empirically evaluating privacy policy generation practices across Swiss websites. Contribution/Results: We find that 18% of sampled local websites employ contract generators; their privacy policies achieve up to a 15-percentage-point higher compliance rate than those of non-users. This finding provides novel SME-level evidence supporting the “Brussels Effect” and validates the effectiveness and scalability of large language model–driven legal compliance assessment.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Privacy
• Philosophy and Ethics of AI: AI & Law, Justice, Regulation & Governance

Application Category

• Responsible Web: Consent frameworks and practices on the web
• Security and Privacy: Large-scale security measurements
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content

Abstract

It has become increasingly challenging for firms to comply with a plethora of novel digital regulations. This is especially true for smaller businesses that often lack both the resources and know-how to draft complex legal documents. Instead of seeking costly legal advice from attorneys, firms may turn to cheaper alternative legal service providers such as automated contract generators. While these services have a long-standing presence, there is little empirical evidence on their prevalence and output quality. We address this gap in the context of a 2023 Swiss privacy law revision. To enable a systematic evaluation, we create and annotate a multilingual benchmark dataset that captures key compliance obligations under Swiss and EU privacy law. Using this dataset, we validate a novel GPT-5-based method for large-scale compliance assessment of privacy policies, allowing us to measure the impact of the revision. We observe compliance increases indicating an effect of the revision. Generators, explicitly referenced by 18% of local websites, are associated with substantially higher levels of compliance, with increases of up to 15 percentage points compared to privacy policies without generator use. These findings contribute to three debates: the potential of LLMs for cross-lingual legal analysis, the Brussels Effect of EU regulations, and, crucially, the role of automated tools in improving compliance and contractual quality.

Problem

• Evaluating prevalence and quality of automated contract generators for privacy policies
• Assessing compliance impact of Swiss privacy law revision using novel methods
• Measuring how automated generators affect legal compliance levels in websites

Innovation

• Created multilingual benchmark dataset for compliance evaluation
• Developed GPT-5-based method for privacy policy assessment
• Automated tools substantially increase legal compliance rates