Self-fulfilling Bandits: Dynamic Selection in Algorithmic Decision-making
56 Pages Posted: 31 Aug 2021 Last revised: 19 Oct 2021
Date Written: October 19, 2021
This paper identifies and addresses dynamic selection problems that arise in online learning algorithms with endogenous data. In a contextual multi-armed bandit model, we show that a novel bias (self-fulfilling bias) arises because the endogeneity of the data influences the choices of decisions, affecting the distribution of future data to be collected and analyzed. We propose a class of algorithms to correct for the bias by incorporating instrumental variables into leading online learning algorithms. These algorithms lead to the true parameter values and meanwhile attain low (logarithmic-like) regret levels. We further prove a central limit theorem for statistical inference of the parameters of interest. To establish the theoretical properties, we develop a general technique that untangles the interdependence between data and actions.
Keywords: Self-fulfilling Bias, Dynamic Selection, Endogeneity Spillover, Contextual Multi-armed Bandit Model
Suggested Citation: Suggested Citation