“Revisiting a Natural Experiment: Do Legislators With Daughters Vote More Liberally on Women’s Issues?”, Donald P. Green, Oliver Hyman-Metzger, Gaurav Sood, Michelle A. Zee2023-08 ()⁠:

The pioneering work of Washington in 2008 shows that legislators with daughters cast more liberal roll call votes on women’s issues. Costa et al 2019 find that this pattern subsides in more recent congresses [ie. failure to replicate Washington2008] and speculate that increasing party polarization might diminish the daughter effect.

We investigate patterns of change over time by looking at 8 congresses prior to the 4 studied by Washington and 8 subsequent congresses, including 3 not included by Costa et al 2019.

Contrary to the party polarization hypothesis, we find no daughter effect prior to the period that Washington studied and no effect thereafter…The results show a striking pattern. Like Costa et al 2019, we find little evidence of a daughter effect in the sessions after those studied in Washington2008, but we also find no effect in earlier sessions. Contrary to the party polarization hypothesis, the daughter effect was weak during this earlier period of relative party comity. Further, tracking cohorts of legislators over time, we find little temporal variation in the daughter effect over the stretch of 20 congresses. The cohort studied by displays unusually strong daughter effects; average effects among a broader selection of legislators are close to zero. The concluding section of this essay reflects on the importance of conducting out-of-sample replications of natural experiments.

Figure 1: Tracking the daughter effect over time for the cohort of legislators analyzed by Washington2008 and for all other members.

…The literature on “daughter effects” has 3 recurrent themes. The first is that the applications cover a sprawling assortment of institutions, regions, and historical periods. The second is that studies that report the results of a novel application often find statistically-significant results, at least for a subgroup (eg. fathers whose first child is female). Third, the direction and magnitude of these results vary from one application to the next. When daughters are found to have a liberalizing effect, the explanation is that having daughters impels parents to “protect their daughters from possible gender-based discrimination” (Glynn & Send2015, pg41), to learn about the challenges of sex-based discrimination, or to accede to pro-feminist pressures from within the household. When daughters are found to have a conservative effect, the explanation is that they “increase conservative views of teen sex” (Conley & Raushcer2013, pg704). [ie. the literature on daughter effects is largely unreplicable p-hacked nonsense]

…To the extent that something systematic underlies the gap between the initial and subsequent results, it may be a variant of the file-drawer problem: natural experiments that generate noteworthy findings receive attention, while those that do not are consigned to oblivion. In the context of daughter effects, the number of historical eras, countries, and institutions provides a large set of potential draws from the sampling distribution.4 This interpretation has testable empirical implications: natural experiments, especially those that produce theoretically engaging results, should have subpar performance when subjected to out-of-sample replications.5