Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem

53 Pages Posted: 14 Mar 2019 Last revised: 15 Mar 2020

See all articles by Mochen Yang

Mochen Yang

University of Minnesota - Twin Cities - Carlson School of Management

Edward McFowland

Harvard Business School

Gordon Burtch

Boston University - Questrom School of Business

Gediminas Adomavicius

University of Minnesota - Twin Cities - Carlson School of Management

Date Written: February 22, 2019

Abstract

Combining machine learning with econometric analysis is becoming increasingly prevalent in both research and practice. A common empirical strategy involves the application of predictive modeling techniques to "mine" variables of interest from available data, followed by the inclusion of those variables into an econometric framework, with the objective of estimating causal effects. Recent work highlights that, because the predictions from machine learning models are inevitably imperfect, econometric analyses based on the predicted variables are likely to suffer from bias due to measurement error. We propose a novel approach to mitigate these biases, leveraging the ensemble learning technique known as the random forest. We propose employing random forest not just for prediction, but also for generating instrumental variables to address the measurement error embedded in the prediction. The random forest algorithm performs best when comprised of a set of trees that are individually accurate in their predictions, yet which also make "different" mistakes, i.e., have weakly correlated prediction errors. A key observation is that these properties are closely related to the relevance and exclusion requirements of valid instrumental variables. We design a data-driven procedure to select tuples of individual trees from a random forest, in which one tree serves as the endogenous covariate and the other trees serve as its instruments. Simulation experiments demonstrate the efficacy of the proposed approach in mitigating estimation biases, and its superior performance over an alternative method (simulation-extrapolation), which has been suggested by prior work as a reasonable method of addressing the measurement error problem.

Keywords: machine learning, econometric analysis, instrumental variable, random forest, causal inference

Suggested Citation

Yang, Mochen and McFowland, Edward and Burtch, Gordon and Adomavicius, Gediminas, Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem (February 22, 2019). Kelley School of Business Research Paper No. 19-20, Available at SSRN: https://ssrn.com/abstract=3339983 or http://dx.doi.org/10.2139/ssrn.3339983

Mochen Yang (Contact Author)

University of Minnesota - Twin Cities - Carlson School of Management ( email )

19th Avenue South
Minneapolis, MN 55455
United States

Edward McFowland

Harvard Business School ( email )

Boston, MA 02163
United States

Gordon Burtch

Boston University - Questrom School of Business ( email )

595 Commonwealth Avenue
Boston, MA 02215
United States

Gediminas Adomavicius

University of Minnesota - Twin Cities - Carlson School of Management ( email )

19th Avenue South
Minneapolis, MN 55455
United States

Do you want regular updates from SSRN on Twitter?

Paper statistics

Downloads
199
Abstract Views
1,776
rank
208,051
PlumX Metrics