Supplementary Materials for

Did Cooperation Among Strangers Decline in the United States?

A Cross-Temporal Meta-Analysis of Social Dilemmas (1956–2017)

Supplementary Materials

Coding of Cooperation

We used the overall cooperation rates reported in social dilemma studies as the measure of cooperation. Specifically, in the continuous Public Goods Dilemma (e.g., Fehr & Fischbacher, 2004) and Prisoner’s Dilemma (e.g., Van Lange & Kuhlman, 1994) in which each player can decide how much of their initial endowments to contribute to a group account or their partner, we calculated cooperation as the percentage of endowment contributed (P_cont = Mean contributions/Endowment). In the dichotomous Prisoner’s Dilemma (e.g., Dawes, 1980) and Public Goods Dilemma (e.g., Shank et al., 2019) in which each participant has two options: cooperation (contributing all endowments to the group account) or defection (contributing nothing), we calculated the proportion of cooperative choices, with higher proportions indicating greater cooperation. If a study only reported the cooperation rates in each subgroup (e.g., communication and no communication, different experimental conditions), then we calculated the overall Mean (M) and pooled Standard Deviation (SD) of cooperation by using the following formulas:

Flowchart of Literature Search and Inclusion

Figure S1

PRISMA Flowchart of Literature Search and Inclusion

Records from Cooperation Databank

(k = 3,026)

Records excluded due to:

• Not Prisoner’s Dilemma or Public Goods Dilemma (k = 591)

• Not reliable cooperation rates (k = 39)^a

• Ineligible period of cooperation measure (k = 510)

• Interactions not with strangers (k = 24)

• Duplicate coding for country (k = 47)^b

• Not American participants (k = 1,114)

• No cooperation rates (k = 1)

• Ineligible sample (age) (k = 57)

• Duplicate coding for sample and multiple endowments (k = 17)^b

• Cooperation in diverse real-life contexts (k = 2)

• Social dilemma using a nonlinear payoff structure (k = 6)

Records after exclusions

(k = 618)

Data splitting for cooperation within studies that manipulated study characteristics

Records excluded due to:

• Interactions not with strangers (k = 1)

• Ineligible sample (age, non-college students) (k = 15)

• Duplicate coding for sample (k = 7)

• K index not applicable (k = 66)

• Indicator of conflict of interests out of interval (0 < K index/MPCR < 1) (k = 35)

• Effect size without variance (k = 2)^c

• Extreme outlier (k = 7)

• Extreme outlier (k = 7)

Records included in meta-analysis

(k = 660, N_studies = 511, N_participants = 63,342)

Note. ^a We excluded records if the second annotator reported unreliable cooperation rates due to several reasons (e.g., insufficient information to code cooperation rates) during the re-checking procedure; ^b We excluded records if the sample was a duplicate of other samples (e.g., between the main study and its sub-studies). In the Cooperation Databank, if a study involved cooperation measures in different dilemmas or countries, sub-studies were coded for describing the sample and study characteristics separately; See Spadaro et al., (in press) for more details about how the data in the Cooperation Databank was coded with regard to studies and sub-studies; ^c We excluded records if there was no sufficient information to compute the variance of the cooperation estimate.

Analysis Based on All Eligible Studies

Year of Data Collection. Across all studies included in the meta-analysis, the year of data collection ranged from 1956 to 2017 (Mdn = 1999; see Figure S2 for the number of effect sizes in each year of data collection).

Figure S2

Histogram of Number of Effect Sizes in Each Year of Data Collection

Note. k = 660

Extreme Outliers Excluded From the Analysis. We also conducted analyses including seven extreme outliers (see Table S1). The results remained the same (see Table S5).

Table S1

Studies With Extreme Outliers in the Dataset

Note. PD = Prisoner’s Dilemma; PGD = Public Goods Dilemma; Yi = logit-transformed cooperation rates; Z_Yi = standardized logit-transformed cooperation rates.

Publication Bias Test. To examine the possibility of publication bias, we first considered the funnel plot where all the cooperation estimates were plotted according to their sample size (see Figure S3). Then, we used a modified Egger’s regression method to test for funnel plot asymmetry (Egger et al., 1997). Specifically, we conducted a multi-level meta-regression of cooperation estimate on sample size (weighted by the inverse of the variance of the cooperation estimate). This modified approach allows us to (a) detect selective reporting in the presence of dependent effect sizes (Rodgers & Pustejovsky, 2021), and (b) improve accuracy in detecting publication bias when using logit transformations of the proportion as outcome measure (Macaskill et al., 2001). Results showed that sample size was not positively associated with cooperation estimates (b = 0.0005, SE = .0003, p = .080). Taken together, these results suggest that there is little publication bias in our data.

Figure S3

Funnel Plots Based on all the Cooperation Estimates

Note. k = 660

Correlations Between Study Characteristics. We calculated the correlations between study characteristics based on original dataset and presented the results in Table S2.

Variance Inflation Factor Analysis. We calculated the generalized variance inflation factors (GVIF, Fox & Monette, 1992) to assess multicollinearity (i.e., the degree to which each independent variable is explained by other independent variables) in our full model with all control variables (i.e., dilemma type, proportion of male participants, repetitions, group size, K index, communication, sanctions, and period of cooperation). In our study, predictors consisted of continuous variables and categorical variables with two or more levels. We followed the approach of Fox and Monette (1992), and computed to make GVIFs comparable across variables with a different number of parameters (e.g., a dummy-coded categorical variable could have multiple coefficients). Then we squared values and followed the rules of thumb for VIF values from previous literature (i.e., the multicollinearity level is acceptable when the VIF is lower than 10; O’Brien, 2007). The results presented in Table S3 suggested that our model was not affected by multicollinearity (i.e., all squared values were smaller than 10).

Table S2

Correlations Between Study Characteristics

Note. ^a 0 = Prisoner’s Dilemma, 1 = Public Goods Dilemma; Repetitions: comparison group = one-shot interaction; Communication: comparison group = no communication; ^b 0 = overall, 1 = first; Sanctions: comparison group = no sanctions. *p < .05.

Table S3

Generalized Variance Inflation Factor (GVIF) Values for Independent Variables in the Full Model

Note. k = 660. ^a 0 = Prisoner’s Dilemma, 1 = Public Goods Dilemma; ^b 0 = one-shot interaction, 0.5 = mixed, 1 = repeated interaction; ^c 0 = no communication, 0.5 = mixed, 1 = communication; ^d 0 = no sanctions, 0.5 = mixed, 1 = sanctions; ^e 0 = overall, 1 = first.

Model Results With Dilemma Type Replaced With Number of Choice Options. The original pre-registration also reported that we coded the number of choice options in the dilemmas. Each social dilemma has at least two choice options: cooperate (contribute all) versus defect (contribute nothing). Some studies ask participants to decide how much of an endowment to contribute to a public good. We coded the number of choice options as a continuous variable. Participants have two choice options (cooperation vs. defection) in dichotomous social dilemmas. For the continuous measure of cooperation, the number of choice options generally equals the maximum amount that participants can contribute plus 1 (i.e., initial endowments + 1, taking into account the option of giving nothing). Studies in which the manipulated initial endowments (i.e., endowments are asymmetric within a group) resulted in multiple choice options but the cooperation rates could not be observed in each treatment were coded using the median value of these numbers (k = 31). In our sample, the number of choice options ranged from 2 to 1001 (M = 22.66, Mode =2, Mdn = 2). Due to the skewed distribution, we log-transformed the number of choice options in our analyses.

The number of choice options was highly correlated with dilemma type (r = .38, p < .001). Generally, participants in the Public Goods Dilemma had more choice options than those in the Prisoner’s Dilemma. To reduce multicollinearity, we only focused on the dilemma type in our main analysis. Actually, the number of choice options did not significantly predict cooperation in a separate meta-regression model (b = 0.03, SE = .02, p = .082). When we included the number of choice options in the full model, we drew the same conclusions (see Table S4).

Table S4

Meta-Regression Models Without Control Variables (Model 1) and Including Control Variables (Model 2)

Note. k = 660. ^a Comparison group = one-shot interaction; ^b Comparison group = no communication; ^c Comparison group = no sanctions; ^d 0 = overall, 1 = first. τ² = tau-squared, the estimate of total amount of heterogeneity; I² = percentage of total variability due to heterogeneity; Level 2 and 3 represent within-study and between-study variance, respectively (hereafter the same). *p < .05.

Model Results Including Extreme Outliers. We also conducted the analyses including 513 studies with seven extreme outliers (see Table S1) in the sample (i.e., 667 unique cooperation estimates involving 64,234 participants). These analyses resulted in the same conclusions as the analyses excluding the outliers (see Table S5).

Table S5

Meta-Regression Models Without Control Variables (Model 1) and Including Control Variables (Model 2)

Note. k = 667. ^a 0 = Prisoner’s Dilemma; 1 = Public Goods Dilemma; ^b Comparison group = one-shot interaction; ^c Comparison group = no communication; ^d Comparison group = no sanctions; ^e 0 = overall, 1 = first. *p < .05.

Spline Model Results. Besides the quadradic model we reported in the main text, we also considered fitting a spline model in order to examine whether the pattern of cooperation over time was curvilinear. We used the ‘restricted cubic spline’ model that consists of a series of piecewise cubic polynomials (Stone & Koo, 1985) to further test for a non-linear trend of cooperation over time. For this spline model, we need to choose the number of 'knots', which are the positions where the piecewise cubic polynomials are connected. Here, we chose three and four ‘knots’ to conduct explorative analyses.

When setting three ‘knots’, we used year of data collection (time) to predict cooperation in the meta-regression model. As shown in Figure S4, cooperation increased from 1968 to 1999 (b = 0.010, SE = .004, p = .031), but did not change between 1999 and 2013 (b = -0.004, SE = .005, p = .343). We also used year of data collection (time) to predict cooperation in the meta-regression model with four ‘knots’. As shown in Figure S5, cooperation did not change from 1966 to 1989 (b = 0.01, SE = .01, p = .223), from 1989 to 2005 (b = -0.003, SE = .02, p = .848), and from 2005 to 2015 (b = -0.01, SE = .08, p = .942).

According to above analyses, the changing pattern of cooperation seems not to follow a curvilinear trend.

Figure S4

Cube Spline Model Showing Historical Changes Over Time in the Mean Cooperation Rates in Social Dilemmas (Three Knots)

Note. Knots are located at 1968, 1999, and 2013 for year of data collection (vertical dotted lines). The solid black line represents average model predictions of cooperation estimates. The shaded gray region indicates 95% prediction intervals based on average model predictions. Data points represent study means and the size of the data point is proportional to study (inverse variance) weighting. Larger dots are equated with means that have a smaller variance.

Figure S5

Cube Spline Model Showing Historical Changes Over Time in the Mean Cooperation Rates in Social Dilemmas (Four Knots)

Note. Knots are located at 1966, 1989, 2005, and 2015 for year of data collection (vertical dotted lines). The solid black line represents average model predictions of cooperation estimates. The shaded gray region indicates 95% prediction intervals based on average model predictions. Data points represent study means and the size of the data point is proportional to study (inverse variance) weighting. Larger dots are equated with means that have a smaller variance.

The Relationship Between Cultural Tightness and Cooperation. Cultural tightness-looseness refers to variance in norms, values, and behavior (Uz, 2015). A tight society tends to have many strongly enforced rules and little tolerance for deviance, whereas a loose society has few strongly enforced rules and greater tolerance for deviance (Harrington & Gelfand, 2014). Here we examined whether state-level tightness-looseness within the U.S. is related to cooperation. We did not preregister this hypothesis and analysis. Moreover, a valid and reliable state-level tightness-looseness index was developed around 2013 (see Harrington & Gelfand, 2014), and we predict cooperation in studies conducted between 1956 and 2017.

We coded the U.S. state in which the study was conducted according to the university where the experiments were conducted or the authors were affiliated to, if the former information was not reported in the paper. We did not estimate the inter-rater agreement for this variable because it was coded at a later stage. There are 630 unique samples that could be coded and associated with a specific state (N = 60,199). Then, we retrieved state-level tightness-looseness values (higher scores indicate greater tightness) across the United States from previous research (see Harrington & Gelfand, 2014).

Finally, we fitted a four-level meta-analytic model that can consider four different variance components distributed over the four levels of the model: sampling variance of the extracted cooperation estimates (level 1), variance between cooperation estimates extracted from the same study (level 2), variance between studies (level 3), and variance between states (level 4). We found that tightness-looseness did not have a statistically significant association with cooperation, when only including this as a predictor variable in a meta-regression model (b = 0.004, SE = .003, p = .230). When we included tightness-looseness in the full model comprising all the study characteristics (i.e., year of data collection, dilemma type, proportion of male participants, repetitions, group size, K index, communication, sanctions and period of cooperation), the association between tightness-looseness and cooperation remained nonsignificant (b = 0.003, SE = .003, p = .207).

Other Study Characteristics and Their Effects on Cooperation. The Cooperation Databank also contains other sample and study characteristics, including student sample, academic discipline, and game payment. These three variables scored a high and/or adequate level of inter-rater agreement (Krippendorff’s α ranged from 0.70 to 0.89, agreement rate ranged from 79.10% to 97.30%). Our original pre-registration plan did not state that we would report the effects of these study characteristics on cooperation. Here, we report whether these study characteristics influence cooperation. Considering that we first coded whether participants were college students or not and then coded the specific discipline of students, and thus these two variables have a hierarchical structure, we did not include student sample and discipline into the same model.

In the studies included in the meta-analysis, most of the participants were college students (k = 626, coded 1, N = 58,976), and with a minority of studies including non-student samples (k = 34, coded 0, N = 4,366). Cooperation does not differ between the college student samples and the non-student samples (b = 0.07, SE = .13, p = .558). When we included the student sample variable in the full model comprising all the study characteristics (i.e., year of data collection, dilemma type, students, payment, proportion of male participants, repetitions, group size, K index, communication, sanctions and period of cooperation), (non)student sample was still not significantly associated with cooperation (b = 0.16, SE = .12, p = .206).

We also coded the specific academic discipline of students. Academic discipline includes economics (k = 23, coded 0), psychology (k = 220, coded 1), other (k = 122, coded 2), and non-students (k = 34, coded 3). Besides, for 261 samples specific academic discipline of students was not reported. Cooperation did not differ between economics students, compared to either psychology students (b = -0.12, SE = .13, p = .350), students from other disciplines (b = -0.12, SE = .14, p = .409), and the non-student samples (b = -0.17, SE = .16, p = .298). When we included academic discipline in the full model including all the study characteristics (i.e., year of data collection, dilemma type, discipline, payment, proportion of male participants, repetitions, group size, K index, communication, sanctions and period of cooperation), academic discipline was still not associated with cooperation (all p-values ≥ .186).

We coded the payoffs determined by participants’ decision in the social dilemma. Studies that involved hypothetical values for participant’s outcomes were coded 0 (k = 47, labeled ‘hypothetical’). Studies that paid participants with money were coded 1 (k = 478, labeled ‘monetary’). Studies that paid participants with non-money resources (e.g., candies, school supplies) were coded 2 (k = 36, labeled ‘non-monetary’). The mixed forms of payment were coded 3 (k = 11, labeled ‘mixed’). Besides, for 88 samples the specific form of payment was not reported. Compared to participants who received hypothetical payments, cooperation rates were not higher or lower among participants who received monetary payments (b = 0.13, SE = .11, p = .232), non-monetary payoffs (b = 0.09, SE = .16, p = .572), or mixed forms of payments (b = -0.04, SE = .24, p = .859). When we included the payment in the full model involving all the study characteristics (i.e., year of data collection, dilemma type, students, payment, proportion of male participants, repetitions, group size, K index, communication, sanctions and period of cooperation), then cooperation still did not vary with the type of payment (all p-values ≥ .180).

In sum, student sample, academic discipline, and game payment did not significantly predict cooperation in our meta-analysis. In addition, when we included student sample (or academic discipline) and payment in the full model including all the study characteristics (i.e., year of data collection, dilemma type, proportion of male participants, repetitions, group size, K index, communication, sanctions and period of cooperation), time continued to have a statistically significant positive relation with cooperation (bs = 0.006 and 0.006, SEs = .002 and .002, p-values = .008 and .004, respectively).

Further Evaluating how Sociocultural Indicators are Associated With Cooperation After Detrending the Variables. It is possible that the correlations (see Table 4 in the main text) observed between the sociocultural indicators and cooperation are spurious because most of the variables have a strong linear trend over time, rather than each variable having a causal effect on cooperation. One approach to control for time trends in the data is that we enter year of data collection, the sociocultural indicators, and study characteristics into a three-level meta-regression model together as predictors of cooperation. However, year of data collection is strongly associated with Gini index (r = .98, p < .001), GDP per capita (r = .98, p < .001), social welfare function (r = .73, p < .001), urbanization (r = .98, p < .001), percentage of people living alone (r = .95, p < .001), violent crime rate (r = .44, p < .001), social trust (r = -.76, p < .001), and materialism (r = .66, p < .001). Year of data collection is not associated with unemployment rate (r = .19, p = .15), Party of President (r = -.15, p = .261), and divorce rate (r = .10, p = .461). These correlations might indicate severe multicollinearity in such models, which would result in unreliable estimates. For example, the squared for many of the sociocultural indicators 10 years prior to cooperation (e.g., GDP per capita and percentage of people living alone) were above (or close to) the acceptable threshold of 10 (O’Brien, 2007), such as Gini index (8.318), GDP per capita (38.576), urbanization (14.705), percentage of people living alone (17.972). Therefore, due to severe multicollinearity, the meta-regression models including both the year and the sociocultural indicators are likely to produce unreliable estimates.

An alternative approach to control for time trends in the data is to use time-series analyses. This approach could be used to test whether the associations between the sociocultural indicators and cooperation are due to each of these variables increasing over time. However, there are severe limitations with applying this approach to our data, including that we are unable to (a) account for the different number of cooperation estimates per year, (b) statistically control for between-study heterogeneity, and (c) weight the effect sizes (and so the following analyses in this section are not proper meta-analyses). Nonetheless, we do report the results of these analyses for interested readers.

In order to perform any time-series analyses, we could only include a single estimate of cooperation at each time point. Therefore, we obtained a single pooled cooperation estimate and associated squared standard errors for each year, by running a set of intercept-only meta-regression models for each year. This generated a dataset that includes a total of 62 data points (i.e., from 1956 to 2017).

Considering that the US President is elected every four years and that the Party of President was coded as a dichotomous variable, this variable does not resemble a significant time trend like the other time-series variables (e.g., Gini index or percentage of people living alone). For this reason, we did not conduct any of the following analyses including the Party of President variable.

After pooling the cooperation estimates, we performed a linear interpolation of missing data for certain years (1957, 1961, 1962, 1983 and 1985) for the time-series of cooperation and its squared standard errors. Then, we narrowed our use of cooperation data to match the time frame of the sociocultural indicators which were not available in 1956. For example, the Gini index was only available from 1967 (see Table 1 in the main text), so we used 1967 as the starting data point of the Gini index and cooperation (see Table S6).

Many time-series analytic methods carry the assumption of stationarity. That is, the data do not have underlying trends that bias analyses of their change over time or their multivariate relations (Caluori et al., 2020). Here, we used two approaches that detrend the time-series variables to ensure data stationarity.

The First Detrending Approach and Corresponding Analyses. To make data stationary, prior to performing any following analyses, we detrended our time-series vectors for the sociocultural indicators and cooperation on the basis of year in order to remove autoregressive trends associated with the passage of time. Specifically, we did so by (1) extracting the residuals of a linear regression model with year predicting the sociocultural indicator, and (2) extracting the residuals of a regression model with year predicting cooperation. We extracted the residuals in step (2) following different methods, including (a) running a linear regression model (function ‘lm’), (b) running a linear regression model in which the squared standard errors are specified through the ‘weights’ argument, and (c) running a meta-regression model that includes the squared standard errors, in which the residuals are extracted through the ‘residuals.rma’ function of the metafor R package (Viechtbauer, 2010).

We then subjected each time series to augmented Dickey-Fuller root tests, which evaluate whether a time series has an underlying trend that renders it nonstationary. A statistically significant result obtained with this test implies the stationarity of these time-series data and that they are suitable for time-series analysis. This test revealed significant results for cooperation, no matter what method we used to extract the residuals of a regression model with year predicting cooperation (all p-values ≤ .003), Gini index (p = .026), social welfare function (p = .071), urbanization (p < .001), unemployment rate (p < .001), and materialism (p = .018). We did not find significant results for GDP per capita (p = .968), divorce rate (p = .169), percentage of people living alone (p = .260), violent crime rate (p = .161), and social trust (p = .219). Nonetheless, we reported the results of the analyses of those non-stationary time-series data for interested readers.

After detrending our time series data, we then performed separate models for each of the sociocultural indicators predicting cooperation. Tables S6, S7, and S8 report the correlations of the sociocultural indicators and Americans’ cooperation. Regressions revealed no significant associations between cooperation and any of the sociocultural indicators. That is, changes in cooperation over time were not associated with any changes that occurred among the sociocultural indicators. However, it is important to note that this approach, contrary to the meta-regression method that led to the findings reported in the main text, does not allow for the weighting of the cooperation estimate by its variance.

Table S6

Relation Between the Sociocultural Indicators and Cooperation After the Variables Were Detrended (Method A)

Note. Method A: Cooperation estimates were detrended through a linear regression model.

Table S7

Relation Between the Sociocultural Indicators and Cooperation After All Variables Were Detrended (Method B)

Note. Method B: Cooperation estimates were detrended through a linear regression model in which the squared standard errors are specified through the ‘weights’ argument.

Table S8

Relation Between the Sociocultural Indicators and Cooperation After All Variables Were Detrended (Method C)

Note. Method C: Cooperation estimates were detrended through a meta-regression model and the residuals are extracted through the ‘residuals.rma’ function of the metafor R package.

We next performed Granger tests of predictive causality, as this test can evaluate whether one time-series variable (x) predicts changes in another time-series variable (y), even when controlling for earlier values of y (Caluori et al., 2020; Grossmann & Varnum, 2015). We performed Granger tests of causality on separate models, one for each sociocultural indicator as a predictor of cooperation, at time lags of t －1, t－5, and t－10 years. A statistically significant result at these time lags could suggest that the sociocultural indicator at time t could predict cooperation 1, 5, and 10 years in the future, even when analyses control for cooperation at time t. The results of Granger test of causality are presented in Tables S9, S10, and S11. Most of these associations were non-significant. We only found that percentage of people living alone at time t could predict cooperation 10 years in the future, and that materialism at time t could predict cooperation 5 years in the future when cooperation estimates were detrended using method c. However, the percentage of people living alone did not satisfy the assumption of stationarity, which could produce inaccurate results.

Table S9

F Statistics From the Granger Test of Predictive Causality at 1- , 5- and 10-Year Lags (Method A)

Note. Method A: Cooperation estimates were detrended through a linear regression model.

Table S10

F Statistics From the Granger Test of Predictive Causality at 1- , 5- and 10-Year Lags (Method B)

Note. Method B: Cooperation estimates were detrended through a linear regression model in which the squared standard errors are specified through the ‘weights’ argument.

Table S11

F Statistics From the Granger Test of Predictive Causality at 1- , 5- and 10-Year Lags (Method C)