“Interpreting the Evidence: Choosing between Randomized and Non-Randomized Studies”, 1999-07-31 (; backlinks; similar):
Evaluations of healthcare interventions can either randomize subjects to comparison groups, or not. In both designs there are potential threats to validity, which can be external (the extent to which they are generalisable to all potential recipients) or internal (whether differences in observed effects can be attributed to differences in the intervention). Randomization should ensure that comparison groups of sufficient size differ only in their exposure to the intervention concerned. However, some investigators have argued that randomized controlled trials (RCTs) tend to exclude, consciously or otherwise, some types of patient to whom results will subsequently be applied. Furthermore, in unblinded trials the outcome of treatment may be influenced by practitioners’ and patients’ preferences for one or other intervention. Though non-randomized studies are less selective in terms of recruitment, they are subject to selection bias in allocation if treatment is related to initial prognosis.
Summary points:
Treatment effects obtained from randomized and non-randomized studies may differ, but one method does not give a consistently greater effect than the other
Treatment effects measured in each type of study best approximate when the exclusion criteria are the same and where potential prognostic factors are well understood and controlled for in the non-randomized studies
Subjects excluded from randomized controlled trials tend to have a worse prognosis than those included, and this limits generalisability
Subjects participating in randomized controlled trials evaluating treatment of existing conditions tend to be less affluent, educated, and healthy than those who do not; the opposite is true for trials of preventive interventions