To consider the relationships between, therapist variability, therapy modality, therapeutic dose and therapy ending type and assess their effects on the variability of patient outcomes.
Multilevel modeling was used to analyse a large sample of routinely collected data. Model residuals identified more and less effective therapists, controlling for case-mix.
After controlling for case mix, 5.8% of the variance in outcome was due to therapists. More sessions generally improved outcomes, by about half a point on the PHQ-9 for each additional session, while non-completion of therapy reduced the amount of pre-post change by 6 points. Therapy modality had little effect on outcome [dodo bird verdict].
Patient and service outcomes may be improved by greater focus on the variability between therapists and in keeping patients in therapy to completion.
…The resulting dataset comprised n = 4,034 patients [CBT: 1,912 (47.4%); Counseling: 2,122 (52.6%)] seen by k = 61 therapists (28 CBT, 33 counsellors). The mean (SD) age of patients in the study sample was 42.1 (13.77) years, 70.1% were female, 90.0% were white and 33.0% were unemployed.
Our primary measure was the Patient Health Questionnaire-9 (PHQ-9; Kroenke et al 200123ya). The PHQ-9 is a 9 item measure of depression. Each item is rated 0–3. Scores can range 0–27, with higher scores indicating more symptoms of depression. The primary outcome was the pre-post change on the PHQ-9. Therefore, positive values were indicative of patient symptom improvement, whilst negative values indicated that their symptoms had worsened.
…To determine statistically reliable and clinically-significant improvement (ie. ‘recovery’) rates, we adopted the procedures as set out by Jacobson & Truax 1991—that is, the change scores for patients had to be greater than the ‘reliable change index’ in order to take account of measurement error, and the end point score had to move from above the cut-off level to below this predetermined score. For the PHQ-9, we used a cut-off score of 10 and a reliable change index of 6 points (McMillan et al 201014ya).
…Therapist Residuals: Figure 1 illustrates the variability between therapists by ranking and plotting the therapist residuals (u0j) produced by the model with their 95% confidence intervals. The ‘average’ therapist is represented by the dashed horizontal line, where the residual equals zero, Therapists whose confidence intervals do not cross zero are statistically-significantly below average, highlighted on the left of the plot (n = 10), or statistically-significantly above average, highlighted on the right of the plot (n = 8). Most therapists (n = 43) were not statistically-significantly different from the ‘average’ therapist.
…Figure 4 presents the recovery rates (statistically reliable and clinically-significant improvement) for patients seen by the 3 groups of therapists identified in the caterpillar plot (Figure 1), across the number of sessions that patients had attended by the end of therapy (ie. their total dose at discharge). Because of the small number of patients who received more than 16 sessions (4.0%, see Figure 2), recovery rates for patients attending more than 16 sessions are not shown in Figure 4. Only 15 (2.8%) patients seen by below average therapists had more than 16 sessions, of whom 26.7% recovered. For average therapists, 114 (3.9%) had more than 16 sessions of whom 52.6% recovered, while the number of patients attending more than 16 sessions with above average therapists was 24 (4.5%) with 75.0% recovered.
The lines of best fit in Figure 4 show the curvilinear relationship between sessions attended and outcome as indicated by the model. The R2 statistics for each of these lines show they fit the data well, particularly for average and above average therapists. The model also indicated that there is less variability between therapists’ outcomes at fewer sessions, and that the variability increases as the sessions attended increases, the ‘fanning-out’ described by the model. The above average therapists’ recovery rates increase most rapidly as sessions increase 2 → 8 sessions while the increase is more gradual for average and particularly below average therapists. For patients who had 8 sessions, the above average therapists were over twice as effective as below average therapists. After 8 sessions, recovery rates begin to level out for average and above average therapists but decrease for the below average therapists. For patients who had 12 sessions, above average therapists were 3× as effective as below average therapists.
…Therapist Effect: The overall therapist effect found, of 5.8%, although statistically-significant, is towards the lower end of the range of therapist effects found elsewhere (Crits-1991; 2005). However, larger effects were found where patients received more than the average number of sessions or completed therapy. Therapists’ recovery rates ranged 16–76% but the majority of therapists could not be considered statistically-significantly different from the average therapist after controlling for case-mix. However, the 13% of therapists that were statistically-significantly more effective than average had recovery rates that were more than twice those of the 16% of therapists identified as statistically-significantly less effective than average.