Is Seeing Believing? Evaluating Human Sensitivity to Synthetic Video

2026-03-14

AI Summary

This study investigates human perceptual sensitivity to audiovisual distortions and artifacts in synthetic videos—such as deepfakes—and their impact on perceived credibility and learning outcomes. Through three between-subjects experiments combining subjective credibility ratings with objective measures of learning performance, the research systematically evaluates how visual and auditory artifacts influence cognitive processing. The findings reveal, for the first time, that multimodal artifacts inherent in deepfakes significantly undermine content credibility and impair learning effectiveness. These results provide novel empirical evidence for theories of synthetic media perception and establish a foundational basis for developing more effective deepfake detection methods and mitigation strategies.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Advances in machine learning have enabled the creation of realistic synthetic videos known as deepfakes. As deepfakes proliferate, concerns about rapid spread of disinformation and manipulation of public perception are mounting. Despite the alarming implications, our understanding of how individuals perceive synthetic media remains limited, obstructing the development of effective mitigation strategies. This paper aims to narrow this gap by investigating human responses to visual and auditory distortions of videos and deepfake-generated visuals and narration. In two between-subjects experiments, we study whether audio-visual distortions affect cognitive processing, such as subjective credibility assessment and objective learning outcomes. A third study reveals that artifacts from deepfakes influence credibility. The three studies show that video distortions and deepfake artifacts can reduce credibility. Our research contributes to the ongoing exploration of the cognitive processes involved in the evaluation and perception of synthetic videos, and underscores the need for further theory development concerning deepfake exposure.

Problem

• deepfakes
• synthetic video
• human perception
• credibility assessment
• disinformation

Innovation

• deepfakes
• synthetic video perception
• audio-visual distortions
• credibility assessment
• cognitive processing