How can the use of different modes of survey data collection introduce bias? A simple introduction to mode effects using directed acyclic graphs (DAGs)

2025-10-01

AI Summary

This study investigates how mode effects (e.g., face-to-face vs. online surveys) and mode selection bias jointly distort epidemiological inference. Using directed acyclic graphs (DAGs), we systematically model their interplay and demonstrate that conventional conditional adjustment may induce collider bias. We integrate DAG-based identifiability analysis, quantitative bias modeling, multiple imputation, and causal sensitivity analysis to characterize the direction and magnitude of resulting biases. Our contributions are threefold: (1) first formal distinction and joint modeling of mode effects and selection bias within mixed-mode survey designs; (2) identification of inherent limitations in standard statistical approaches—particularly naive covariate adjustment—for addressing such biases; and (3) proposal of a DAG-driven bias mitigation framework that enhances causal validity and estimation reliability in multi-mode data integration. (149 words)

Technology Category

• Reasoning under Uncertainty: Causality
• Machine Learning: Causal Learning
• Knowledge Representation and Reasoning: Preferences

Application Category

• Social Networks and Social Media: Fairness and bias in social network and social media analysis
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Survey data are self-reported data collected directly from respondents by a questionnaire or an interview and are commonly used in epidemiology. Such data are traditionally collected via a single mode (e.g. face-to-face interview alone), but use of mixed-mode designs (e.g. offering face-to-face interview or online survey) has become more common. This introduces two key challenges. First, individuals may respond differently to the same question depending on the mode; these differences due to measurement are known as 'mode effects'. Second, different individuals may participate via different modes; these differences in sample composition between modes are known as 'mode selection'. Where recognised, mode effects are often handled by straightforward approaches such as conditioning on survey mode. However, while reducing mode effects, this and other equivalent approaches may introduce collider bias in the presence of mode selection. The existence of mode effects and the consequences of naïve conditioning may be underappreciated in epidemiology. This paper offers a simple introduction to these challenges using directed acyclic graphs by exploring a range of possible data structures. We discuss the potential implications of using conditioning- or imputation-based approaches and outline the advantages of quantitative bias analyses for dealing with mode effects.

Problem

• Investigating bias from mixed-mode survey data collection
• Analyzing mode effects and selection using directed acyclic graphs
• Evaluating collider bias risks in epidemiological survey conditioning

Innovation

• Using directed acyclic graphs to visualize mode effects
• Evaluating conditioning and imputation approaches for bias
• Proposing quantitative bias analyses for survey data