TLDR: empirical evidence shows immediate (mechanical) gains in GPA that coincide with increases in number of days absent & large increases in chronic absence. These effects are driven by heterogeneous responses to the policy on the basis of ability.

In particular, high ability students (as measured by 8th grade math scores) drive the entirety of the GPA gains while low ability students drive the entirety of the reduction in engagement, as measured by absences.

In the long-run, these effects compound and eventually result in widening achievement gaps as measured by ACT scores. Detailed thread below:

1/ we first develop a theoretical model to explain how students may respond to a policy that lowers grade standards. We show that this has the capacity to WIDEN or MITIGATE achievement gaps, which depends on the distribution of student ability & the magnitude of the leniency

2/ this is the policy in question. Effectively the state moved from a 7-point grading scale to a 10-point scale for all classes. You can see that earning an A became 3 points “easier” but passing a class became 10 points easier.

3/ the immediate response to the policy is stark, even descriptively. The top panel shows 9th grade course averages in math during the last year of the 7-point scale, and the bottom panel shows the same during the first year of the “lenient” scale

4/ you can see that the unnatural bunching patterns immediately shift in response to the policy. We don’t make a claim about the mechanisms, but this at least suggests that schools (teachers or students or both) are keenly aware of the change and respond to it

5/ we want to understand the effect of the policy, & we want to exploit the timing of it on the first cohort of high schoolers facing it; however, the transition from 8th to 9th grade is the second large point of retention for students in K-12, so there’s selection on entry year

6/ rather than compare the 2 cohorts directly, we instrument for HS entry using a separate kindergarten entry rule used by the state. This addresses issues of non-compliance at the 9th grade entry level, as evident by the figure. We then use a fuzzy RD to estimate effects

7/ we estimate LATEs under a fuzzy RDD with an “honest” RD framework to account for the fact that our running variable is discrete. Overall, we find substantial gains in GPA but potentially larger negative effects on engagement (absences)

8/ when we look at these outcomes for “high” and “low” ability students, we see gains for high ability students only and reductions in engagement for low ability students only. We also conduct a simple simulation that suggests low ability students SHOULD be mechanically

9/ receiving the highest bumps in GPA, but the null effect suggests an equally large, negative effect in effort that counteracts this. The null effect on high ability students’ test scores further points to their gains being purely mechanical.

10/ Across high school, these differences compound and begin to widen, but the differences between ability groups persists - ie, low ability students never experience positive GPA gains in response to this policy

11/ we look at longer-run outcomes and find little movement except on compulsory ACT scores, which have a (-) effect for our low ability group. Our placebo analysis shows no differences in ACT b/w the groups, suggesting the policy caused deficits in the low group’s performance

12/ we conduct a battery of robustness checks and show that our main results are not driven by confounding age effects, as is common when using a birthday RD. We also rule out several other mechanisms like sensitive RD specifications and changes in education inputs.

13/ above all, this paper highlights the importance of policy design and the subsequent response of students. At face value, lowering standards may seem like a good solution to combat achievement gaps, but theoretically and empirically we find evidence on the contrary.