Problem
Research questions and friction points this paper is trying to address.
deepfakes
AIG-NCII
non-consensual intimate imagery
epistemic harms
dignity harms
Position: AI/ML Deepfake Research is Misaligned with AI-Generated Non-Consensual Intimate Imagery (AIG-NCII)
🤖 AI Summary
This study addresses the prevailing overemphasis in deepfake research on image authenticity and audience-centered cognitive harms, while systematically neglecting victim-centered dignity harms—particularly in the context of AI-generated non-consensual intimate imagery (AIG-NCII). Through a qualitative landscape analysis of high-impact literature, the paper delineates for the first time between audience-oriented cognitive harms and subject-oriented dignity harms, demonstrating that prevailing technical interventions—such as authenticity detection—not only fail to mitigate but may even exacerbate actual harm to victims. The work advocates for integrating AIG-NCII into the core agenda of AI safety, redefining threat models accordingly, and calls for interdisciplinary safety mechanisms and a paradigm shift in research approaches within high-risk contexts.
📝 Abstract
AI-generated non-consensual intimate imagery (AIG-NCII) is not adequately addressed in AI/ML literature regarding AI-generated media, commonly referred to as"deepfakes". While research on deepfakes currently focuses on its epistemic harms -- or harms relating to truth and authenticity -- this is misaligned with the dominant reality of generative AI abuse involving sexualized imagery. We conduct a landscape analysis of highly-cited works to demonstrate that technical interventions addressing deepfakes almost entirely ignore AIG-NCII, limiting the research ecosystem to authenticity detection tools. In this position paper, we argue that existing interventions address viewer-centric epistemic harms, such as fraud or scams, but ignore subject-centric dignity harms, such as AIG-NCII. We illustrate that knowing an image is synthetic does not mitigate harms to subjects and may, in some cases, even exacerbate them. We conclude by offering recommendations to realign the field, including updating threat models to consider subject-centric harms and addressing AIG-NCII in AI safety research. Finally, we caution that researchers should only engage in this high-risk domain if they implement safety guardrails for both subjects and researchers and establish partnerships with domain experts in sexual violence prevention.
Innovation
Methods, ideas, or system contributions that make the work stand out.
AIG-NCII
deepfakes
subject-centric harms
AI safety
generative AI abuse
🔎 Similar Papers
No similar papers found.
