“Nonexperimental Replications of Social Experiments: A Systematic Review”, 2002-09 (; backlinks; similar):
Controlled experiments, where subjects are randomly assigned to receive interventions, are desirable but frequently perceived to be infeasible or overly burdensome, especially in social settings. Therefore, nonexperimental (also called quasi-experimental) methods are often used instead. Quasi-experimental methods are less intrusive and sometimes less costly than controlled experiments, but their validity rests on particular assumptions that are often difficult to test. It is therefore important to find empirical evidence to assess the likelihood that a given method applied in a given context will yield unbiased estimates.
The current study is a systematic review of validation research to better understand the conditions under which quasiexperimental methods most closely approximate the results that would be found in a well-designed and well-executed experimental study. We collect and summarize a set of earlier studies that each tried, using convenience samples and one or more quasi-experimental methods, to replicate the findings from a social experiment.
Our synthesis aims to give both producers and consumers of social program evaluations a clear understanding of what we know and what we do not know about the performance of quasi-experimental evaluation methods.