“The Iron Law Of Evaluation And Other Metallic Rules”, Peter H. Rossi2012-09-18 (ads, history, insight porn, politics, sociology, scientific bias, causality; backlinks)⁠:

Problems with social experiments and evaluating them, loopholes, causes, and suggestions; non-experimental methods systematically deliver false results, as most interventions fail or have small effects.

“The Iron Law Of Evaluation And Other Metallic Rules” is a classic review paper by American “sociologist Peter Rossi, a dedicated progressive and the nation’s leading expert on social program evaluation from the 1960s through the 1980s”; it discusses the difficulties of creating a useful social program, and proposed some aphoristic summary rules, including most famously:

• The Iron law: “The expected value of any net impact assessment of any large scale social program is zero”

• the Stainless Steel law: “the better designed the impact assessment of a social program, the more likely is the resulting estimate of net impact to be zero.”

It expands an earlier paper by Rossi (“Issues in the evaluation of human services delivery”, Rossi 1978_46ya), where he coined the first, “Iron Law”.

I provide an annotated HTML version with fulltext for all references, as well as a bibliography collating many negative results in social experiments I’ve found since Rossi’s paper was published (see also the closely-related Replication Crisis).

• The Iron Law
  • Introduction
  • Some “Laws” Of Evaluation
  • How Firm Are The Metallic Laws Of Evaluation?
  • Is There Something Wrong With Evaluation Research?
  • Sources Of Program Failures
    • Problem Theory
    • Program Theory
    • Implementation
    • Sources of Theory Failure
  • Problem Theory Failures
  • Program Theory And Implementation Failures
    • Wrong Treatment
    • Right Treatment But Insufficient Dosage
    • Counter-Acting Delivery System
      • Pilot and Production Runs
      • Inadequate Reward System
  • Conclusions
  • References
• See Also
• External Links

Backlinks: