Evaluating Social Policy by Experimental and Nonexperimental Methods

Abstract

Although it is important to establish causal relationships in social policy evaluation, the effects are difficult to observe due to sample selection. To evaluate the performance of estimators designed to handle sample selection bias, we analyse data from a Norwegian rehabilitation project with a randomised experimental design. The data permit us to compare the performance of different nonexperimental estimators with the experimental results. In our case study we find that nonexperimental evaluation based on sample selection estimators with selection terms that fail to meet conventional levels of statistical significance is highly unreliable. The difference in difference estimator and propensity score matching estimators perform better in our context.

JEL Classification: C51, J24, H43, I12