Hypothesizing an effect size by considering individual variation

2026-04-09

AI Summary

This study addresses the common reliance on unrealistic assumptions about average treatment effects in experimental and observational research designs. It proposes a novel paradigm that shifts focus from directly positing average effects to modeling the full distribution of individual treatment effects, from which more plausible assumptions about average effects can be derived. By integrating distributional modeling with cross-disciplinary case studies, the approach demonstrates its validity and utility across diverse fields—including medicine, economics, and psychology—offering researchers a principled, heterogeneity-aware framework for specifying effect sizes grounded in empirical realism rather than idealized assumptions.

Technology Category

Game Theory and Economic Paradigms: Mechanism DesignReasoning under Uncertainty: CausalityMachine Learning: Causal Learning

Application Category

User Modeling, Personalization and Recommendation: Practical large-scale studies of user experienceResponsible Web: Human-perceived consequences of algorithmic deployment on the webEconomics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

When designing and evaluating an experiment or observational study, it is useful to have a realistic hypothesis regarding the average treatment effect. We present an approach to conceptualizing this average by first considering a distribution of effects. We demonstrate with examples in medicine, economics, and psychology.

Problem

• effect size
• individual variation
• average treatment effect
• experimental design
• observational study

Innovation

• effect size
• individual variation
• average treatment effect
• heterogeneous effects
• study design