Explaining Machine Learning by Bootstrapping Partial Dependence Functions and Shapley Values

66 Pages Posted: 18 Nov 2021 Last revised: 22 Nov 2021

See all articles by FRB of Kansas City Submitter

FRB of Kansas City Submitter

Federal Reserve Bank of Kansas City

Thomas R. Cook

Federal Reserve Bank of Kansas City

Greg M. Gupton

Federal Reserve Board of Governors

Zach Modig

Federal Reserve Board of Governors; George Mason University

Nathan Palmer

Board of Governors of the Federal Reserve System

Date Written: October 28, 2021

Abstract

Machine learning and artificial intelligence methods are often referred to as \black boxes" when compared to traditional regression-based approaches. However both traditional and machine learning methods are concerned with modeling the joint distribution between endogenous (target) and exogenous (input) variables. The fitted models are themselves functionals of the data,
about which we can do statistical inference using computationally intensive methods such as the bootstrap. Where linear models describe the fitted relationship between the target and input variables via the slope of that relationship (coefficient estimates), the same fitted relationship can be described rigorously for any machine learning model by first-differencing the partial dependence functions. Bootstrapping these fi rst-differenced functionals provides standard errors and confidence intervals for the estimated relationships. We show that this approach replicates the point estimates of coefficients attained in a linear OLS models, and demonstrate how this generalizes to marginal relationships in ML/AI models. This paper extends the partial dependence plot described in Friedman (2001), and visualizes the marginal distribution used to construct the PDP as described in Goldstein et al. (2015) before applying the steps described above. We further discuss the extension of PDP into a Shapley value decomposition and explore how it can be used to further explain model
outputs. We conclude with a hedonic house pricing example, which illustrates how machine learning methods such as random forests, deep neural net, and support vector regression automatically capture nonlinearities and shed light on inconsistencies revealed by meta-studies of hedonic house pricing.

JEL Classification: C14, C18, C15

Suggested Citation

Submitter, FRB of Kansas City and Cook, Thomas R. and Gupton, Greg M. and Modig, Zachary and Palmer, Nathan, Explaining Machine Learning by Bootstrapping Partial Dependence Functions and Shapley Values (October 28, 2021). Available at SSRN: https://ssrn.com/abstract=3965766 or http://dx.doi.org/10.2139/ssrn.3965766

FRB of Kansas City Submitter

Federal Reserve Bank of Kansas City

1 Memorial Drive
Kansas City, MO 64198
United States

Thomas R. Cook (Contact Author)

Federal Reserve Bank of Kansas City ( email )

1 Memorial Dr.
Kansas City, MO 64198
United States

Greg M. Gupton

Federal Reserve Board of Governors ( email )

20th Street and Constitution Avenue NW
Washington, DC 20551
United States

Zachary Modig

Federal Reserve Board of Governors ( email )

20th Street and Constitution Avenue NW
Washington, DC 20551
United States

George Mason University ( email )

Fairfax, VA
United States

Nathan Palmer

Board of Governors of the Federal Reserve System ( email )

20th Street and Constitution Avenue NW
Washington, DC 20551
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
28
Abstract Views
265
PlumX Metrics