The Role of Randomized Control Trials in Online Demand Generation: Exploratory Evidence from Facebook

Proceedings of the International Conference on Information Systems (ICIS) 2021

30 Pages Posted: 22 Mar 2021 Last revised: 27 Apr 2022

See all articles by Julian Runge

Julian Runge

Duke University (Visiting Scholar)

Harikesh Nair

Stanford University - Graduate School of Business

Date Written: September 27, 2021

Abstract

Observing the organically evolving Facebook advertising marketplace, this paper sets out to study how the adoption of randomized control trials (RCTs) for policy evaluation impacts firms’ behavior and performance on the platform. Findings indicate strong positive effects from the adoption of RCTs: After or in coincidence with RCT adoption, firms increase their investment in demand generation, at stable or higher levels of performance, as measured by customer purchases achieved per advertising dollar spent. A human expert provided to firms by the platform is also able to induce increases in investment but smaller ones, and, importantly, performance gains appear to exclusively materialize after the adoption of RCTs for policy evaluation, and not through human expertise alone. By means of the unbiased causal estimates obtained through RCTs, firms may be able to uniquely reduce their uncertainty about the environment and identify effective policies to exploit it with certainty.

Keywords: Experimentation, firm learning, online advertising, randomized control trials, exploration-exploitation

JEL Classification: M31, M37

Suggested Citation

Runge, Julian and Nair, Harikesh, The Role of Randomized Control Trials in Online Demand Generation: Exploratory Evidence from Facebook (September 27, 2021). Proceedings of the International Conference on Information Systems (ICIS) 2021, Available at SSRN: https://ssrn.com/abstract=3794028 or http://dx.doi.org/10.2139/ssrn.3794028

Julian Runge (Contact Author)

Duke University (Visiting Scholar) ( email )

Box 90120
Durham, NC 27708-0120
United States

Harikesh Nair

Stanford University - Graduate School of Business ( email )

655 Knight Way
Stanford, CA 94305-5015
United States
650-736-4256 (Phone)

HOME PAGE: http://faculty-gsb.stanford.edu/nair/index.html

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
102
Abstract Views
504
rank
349,444
PlumX Metrics