Empirically Validating Citation Metrics for Legal Scholars: A Market Approach

34 Pages Posted: 12 Aug 2021 Last revised: 19 Aug 2021

See all articles by Joshua B. Fischman

Joshua B. Fischman

University of Virginia School of Law

Michael A. Livermore

University of Virginia School of Law

Date Written: August 10, 2021

Abstract

Citation counts are a common quantitative metric used by researchers and analysts to assess scholarly output. When U.S. News & World Report announced in 2019 that it was developing a citation-based ranking for law schools, it brought new attention to debates about citations counts in legal scholarship. Supporters of citation metrics argue that they are superior to alternative measures of scholarly reputation, such as surveys. Critics are skeptical that citations serve as a meaningful proxy for scholarly quality and raise concerns that citation metrics could distort the incentives of law professors and faculties. We examine the validity of citation metrics by examining how well they correspond to a “market valuation” of legal scholars in lateral hiring. We consider two outcomes: whether professors make lateral moves and the rank of the institution where they are hired. Using citation counts derived from the HeinOnline database, we find that citation metrics have a weak association with lateral outcomes. Metrics that mitigate the effect of the highly skewed distribution of citations, such as log citations and the h-index, perform slightly better. Article placements are stronger predictors of lateral outcomes. In particular, articles in top law reviews, top peer-reviewed journals, and online law reviews are all associated with moving to higher-ranked faculties, even after controlling for citations. This implies that citation rankings undervalue these kinds of publications. The divergence between citation counts and professors’ market valuation suggest that citation rankings could significantly distort publication and hiring in the legal academy.

Keywords: citation metrics, law school ranking, h-index, u.s. news

JEL Classification: I23

Suggested Citation

Fischman, Joshua B. and Livermore, Michael A., Empirically Validating Citation Metrics for Legal Scholars: A Market Approach (August 10, 2021). Available at SSRN: https://ssrn.com/abstract=3902866 or http://dx.doi.org/10.2139/ssrn.3902866

Joshua B. Fischman (Contact Author)

University of Virginia School of Law ( email )

580 Massie Road
Charlottesville, VA 22903
United States

Michael A. Livermore

University of Virginia School of Law ( email )

Do you want regular updates from SSRN on Twitter?

Paper statistics

Downloads
238
Abstract Views
957
rank
175,902
PlumX Metrics