Visualizing The Implicit Model Selection Tradeoff

49 Pages Posted: 25 Oct 2021 Last revised: 9 Mar 2022

See all articles by Zezhen (Dawn) He

Zezhen (Dawn) He

University of Rochester - Simon Business School

Yaron Shaposhnik

University of Rochester - Simon Business School

Date Written: October 21, 2021

Abstract

The recent rise of Machine Learning (ML) has been leveraged by operations management researchers to provide new solutions to operational problems. As other ML applications, these solutions rely on model selection, which is typically done by evaluating certain metrics on models separately and selecting the model whose evaluations (accuracy-related loss and/or some interpretability measures) are optimal. However, empirical evidence suggests that often in practice multiple models attain competitive results. Therefore, while models' overall performance could be similar, the models could operate quite differently. This results in an implicit tradeoff in models' performance throughout the feature space whom resolving requires new model selection tools.

This paper aims to study methods for comparing predictive models in an interpretable manner in order to expose this tradeoff, which characterizes model selection problems. To this end, we propose various methods that synthesize ideas from supervised learning, unsupervised learning, dimensionality reduction, and visualization, and demonstrate how they can be used to inform the model selection process. Using various datasets and a simple Python interface that we developed, we demonstrate how practitioners and researchers could benefit from applying these approaches to better understand the broader impact of their model selection choices.

Keywords: Model selection, Interpretability, Machine learning, Visualization

Suggested Citation

He, Zezhen and Shaposhnik, Yaron, Visualizing The Implicit Model Selection Tradeoff (October 21, 2021). Available at SSRN: https://ssrn.com/abstract=3946701 or http://dx.doi.org/10.2139/ssrn.3946701

Zezhen He (Contact Author)

University of Rochester - Simon Business School ( email )

Rochester, NY 14627
United States

Yaron Shaposhnik

University of Rochester - Simon Business School ( email )

Rochester, NY 14627
United States

Do you have a job opening that you would like to promote on SSRN?

Paper statistics

Downloads
41
Abstract Views
179
PlumX Metrics