Enabling Multilingual Privacy Policy Audits: Large-Scale Analysis of Spanish Mobile Apps

2026-07-20

AI Summary

This study addresses the systemic blind spots in existing privacy policy auditing tools, which are predominantly limited to English and thus fail to cover multilingual contexts such as the European Union. The authors propose a cross-lingual analysis approach leveraging large language models without fine-tuning, enabling direct processing of privacy policies in 24 EU languages. They conduct a multidimensional audit of 2,611 Android applications from the Spanish Google Play Store, integrating policy text, privacy labels, and network traffic data. The work demonstrates, for the first time, that large language models can achieve consistently high-accuracy cross-lingual classification under a zero-shot setting, with macro F1 scores ranging from 0.91 to 0.94. Findings reveal that public-sector apps frequently use local languages yet exhibit discrepancies between stated policies and actual behavior, whereas popular commercial apps tend to default to English, underscoring the limitations of current English-centric auditing practices.

Abstract

Automated analyses of privacy policies enable large-scale assessments of transparency in digital ecosystems, yet existing auditing pipelines remain predominantly English-centric. This limits their ability to systematically evaluate multilingual environments, as in the European Union, where many services disclose privacy practices only in local languages. This paper examines whether large language models (LLMs) can extend privacy policy analysis beyond English without requiring language-specific adaptation, thus empowering large-scale auditing in linguistically diverse app ecosystems. We assemble an evaluation corpus spanning all 24 official EU languages from translated versions of two established expert-annotated datasets (OPP-115 and MAPP) and assess translation fidelity through automated metrics and targeted legal-expert review. Our LLM-based classifier for identifying categories of personal data collection achieves stable cross-lingual performance, with macro-F1 scores ranging between 0.91 and 0.94. We then leverage this capability in a large-scale audit of 2,611 Android applications from the Spanish Google Play Store. Combining multilingual privacy policy analysis with the evaluation of corresponding privacy labels and runtime network traffic exposes an important linguistic barrier: public-sector apps predominantly provide privacy policies in Spanish, whereas popular commercial apps mostly provide them in English. We reveal systematic discrepancies between declared and observed practices, especially in public-sector apps. Overall, our results indicate how English-only privacy audits can systematically obfuscate transparency gaps in multilingual environments.

Problem

• multilingual privacy policy
• privacy audit
• cross-lingual analysis
• digital transparency
• language barrier

Innovation

• multilingual privacy auditing
• large language models
• cross-lingual generalization
• privacy policy analysis
• digital transparency