Regulating Artificial Intimacy: From Locks and Blocks to Relational Accountability

2026-04-20

AI Summary

This study addresses the potential risks posed by companion chatbots to vulnerable populations such as children and the exacerbation of power asymmetries between platforms and users. Integrating legal text analysis, regulatory theory, psychology, and information systems perspectives, it pioneers the incorporation of power dynamics inherent in “artificial intimacy” into a regulatory framework. The work proposes a three-dimensional governance model encompassing access control, content moderation, and relational accountability. Central to this approach is an open-ended duty of care that constrains platform power and advances a relational accountability mechanism transcending reactive, case-by-case harm mitigation. By doing so, the study offers policymakers and platform operators a comprehensive regulatory pathway that balances technical feasibility with ethical depth.

Technology Category

• Philosophy and Ethics of AI: AI & Law, Justice, Regulation & Governance
• Natural Language Processing: Safety and Robustness
• Application Domains: Humanities & Computational Social Science

Application Category

• Responsible Web: Technology governance, policy, and regulations
• Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

A series of high-profile tragedies involving companion chatbots has triggered an unusually rapid regulatory response. Several jurisdictions, including Australia, California, and New York, have introduced enforceable regulation, while regulators elsewhere have signaled growing concern about risks posed by companion chatbots, particularly to children. In parallel, leading providers, notably OpenAI, appear to have strengthened their self-regulatory approaches. Drawing on legal textual analysis and insights from regulatory theory, psychology, and information systems research, this paper critically examines these recent interventions. We examine what is regulated and who is regulated, identifying regulatory targets, scope, and modalities. We classify interventions by method and priority, showing how emerging regimes combine "locks and blocks", such as access gating and content moderation, with measures addressing toxic relationship features and process-based accountability requirements. We argue that effective regulation of companion chatbots must integrate all three dimensions. More, however, is required. Current regimes tend to focus on discrete harms, narrow conceptions of vulnerability, or highly specified accountability processes, while failing to confront deeper power asymmetries between providers and users. Providers of companion chatbots increasingly control artificial intimacy at scale, creating unprecedented opportunities for control through intimacy. We suggest that a general, open-ended duty of care would be an important first step toward constraining that power and addressing a fundamental source of chatbot risk. The paper contributes to debates on companion chatbot regulation and is relevant to regulators, platform providers, and scholars concerned with digital intimacy, law and technology, and fairness, accountability, and transparency in sociotechnical systems.

Problem

• companion chatbots
• artificial intimacy
• regulation
• power asymmetry
• duty of care

Innovation

• relational accountability
• artificial intimacy
• duty of care
• companion chatbots
• regulatory asymmetry