Doing Audits Right? The Role of Sampling and Legal Content Analysis in Systemic Risk Assessments and Independent Audits in the Digital Services Act

2025-05-06

AI Summary

This study addresses the lack of methodological foundations for systemic risk assessment and independent auditing under the EU’s Digital Services Act (DSA). Methodologically, it proposes the first evidence-driven compliance auditing framework, uniquely integrating legal requirements with empirical sampling theory to establish dynamic, risk-category-specific, and platform-sensitive representativeness criteria. At its core lies legally guided content sampling, augmented by stratified and temporal sampling, interdisciplinary representativeness evaluation, systemic risk classification modeling, and empirical review of compliance reporting—yielding a mixed-methods audit pathway combining qualitative and quantitative analysis. The contribution lies in overcoming key limitations of existing auditing approaches—namely, opacity and weak evidentiary chains—by empirically validating the “sampling + legal analysis” approach across diverse systemic risks, including illegal content, fundamental rights violations, democratic interference, and gender-based violence. The framework delivers an actionable, reproducible DSA compliance assessment methodology for regulators and independent auditors.

Technology Category

• Application Domains: Humanities & Computational Social Science
• Philosophy and Ethics of AI: AI & Law, Justice, Regulation & Governance
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Responsible Web: Audits of web technologies, standards, platforms, and applications
• Web Mining and Content Analysis: Web data quality in the era of algorithmically-generated content
• Security and Privacy: Data transparency and provenance

Abstract

A central requirement of the European Union's Digital Services Act (DSA) is that online platforms undergo internal and external audits. A key component of these audits is the assessment of systemic risks, including the dissemination of illegal content, threats to fundamental rights, impacts on democratic processes, and gender-based violence. The DSA Delegated Regulation outlines how such audits should be conducted, setting expectations for both platforms and auditors. This article evaluates the strengths and limitations of different qualitative and quantitative methods for auditing these systemic risks and proposes a mixed-method approach for DSA compliance. We argue that content sampling, combined with legal and empirical analysis, offers a viable method for risk-specific audits. First, we examine relevant legal provisions on sample selection for audit purposes. We then assess sampling techniques and methods suitable for detecting systemic risks, focusing on how representativeness can be understood across disciplines. Finally, we review initial systemic risk assessment reports submitted by platforms, analyzing their testing and sampling methodologies. By proposing a structured, mixed-method approach tailored to specific risk categories and platform characteristics, this article addresses the challenge of evidence-based audits under the DSA. Our contribution emphasizes the need for adaptable, context-sensitive auditing strategies and adds to the emerging field of DSA compliance research.

Problem

• Evaluating methods for auditing systemic risks under DSA
• Proposing mixed-method approach for DSA compliance audits
• Assessing sampling techniques for detecting systemic risks

Innovation

• Mixed-method approach for DSA compliance audits
• Content sampling with legal and empirical analysis
• Tailored strategies for specific risk categories