Improvement in Minimum Detectable Effects in Randomized Control Trials: Comparing User-Based and Geo-Based Randomization
DOI:
https://doi.org/10.15680/IJCTECE.2022.0504005Keywords:
Causal inference, Randomized Control Trials, Digital Advertising, Minimum Detectable EffectAbstract
In digital advertising, randomized control trials (RCTs) are a fundamental method for evaluating campaign effectiveness. The randomization approach, whether user-based or geo-based—can substantially impact the statistical power and precision of detecting treatment effects. This paper explores the influence of these randomization methods on the Minimum Detectable Effect (MDE), focusing on how intra-cluster correlation (ICC) in geo-based randomization inflates variance and increases sample size requirements. Through mathematical modeling and simulation, we demonstrate that user-based randomization is more efficient, particularly for detecting small advertising effects, and offer recommendations for optimizing experimental designs in digital advertising contexts.
References
1. Imbens, G. W., & Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences. Cambridge University Press.
2. Angrist, J. D., & Pischke, J. S. (2009). Mostly Harmless Econometrics: An Empiricist's Companion. Princeton University Press.
3. Dube, J.-P., Chintagunta, P. K., & Chintagunta, P. K. (2010). "Tipping and Targeting in Digital Advertising."
Marketing Science, 29(5), 844-862.
4. Zhang, H., & Choi, J. (2019). "Impact of Randomization Strategy on Estimating Effects in Digital Advertising." Journal of Marketing Analytics, 7(2), 89-101.
5. Gerber, A. S., & Green, D. P. (2012). Field Experiments: Design, Analysis, and Interpretation.
6. Kohavi, R., Tang, D., & Xu, Y. (2020). Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing.
7. Imbens, G. W., & Rubin, D. B. (2015). Causal Inference for Statistics, Social, and Biomedical Sciences.
8. Xu, Y., Chen, N., & Xiong, J. (2020). "Evaluation of Geo-Based Versus User-Based Randomization in Online Advertising Experiments."
9. Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical Linear Models: Applications and Data Analysis Methods.
10. Bakshy, E., Eckles, D., & Yan, R. (2012). "Unbiased Estimation in A/B Tests with Interference."
11. Lewis, R. A., & Rao, J. M. (2015). "The Unfavorable Economics of Measuring the Returns to Advertising."
12. Higgins, J. P. T., & Green, S. (Eds.). (2011). Cochrane Handbook for Systematic Reviews of Interventions.
13. Gelman, A., & Hill, J. (2006). Data Analysis Using Regression and Multilevel/Hierarchical Models.
