“No Evidence That Chinese Playtime Mandates Reduced Heavy Gaming in One Segment of the Video Games Industry”, David Zendle, Catherine Flick, Elena Gordon-Petrovskaya, Nick Ballou, Leon Y. Xiao, Anders Drachen2023-08-10 (sociology of tech)⁠:

[pre-registration] Governments around the world are considering regulatory measures to reduce young people’s time spent on digital devices, particularly video games. This raises the question of whether proposed regulatory measures would be effective. Since the early 2000s, the Chinese government has been enacting regulations to directly restrict young people’s playtime. In November 2019, it limited players aged under 18–1.5 hours of daily playtime and 3 hours on public holidays.

Using telemetry data on over 7 billion hours of playtime provided by a stakeholder from the video games industry [Unity], we found:

no credible evidence for overall reduction in the prevalence of heavy playtime following the implementation of regulations: individual accounts became 1.14× more likely to play heavily in any given week (95% confidence interval 1.139–1.141).

This falls below our preregistered smallest effect size of interest (2.0) and thus is not interpreted as a practically meaningful increase. Results remain robust across a variety of sensitivity analyses, including an analysis of more recent (2021) adjustments to playtime regulation.

This casts doubt on the effectiveness of such state-controlled playtime mandates.

…The most consistent and restrictive governmental regulation of play, however, has been occurring in Mainland China. From 2000 onwards, the Chinese government has variously restricted the production, import and sale of gaming consoles and arcade machines (such practices were initially restricted, restrictions were later repealed and practices were later officially permitted^43,44,45); mandated online game providers to install ‘anti-addiction software’^6,46; and repeatedly paused the government approval process for new video gaming licences, which every game title needs to be legally available⁴⁷. Effective November 2019, the Chinese government enacted a further policy controlling access to gaming among young people. Under new regulations defined in the ‘Notice on the Prevention of Online Gaming Addiction in Juveniles’, online video game providers became obligated to both prevent individuals under the age of 18 from playing for more than 1.5 hours each day (or 3 hours on a public holiday) and prevent these users from playing between the hours of 22:00 and 08:00⁶. These regulations were explicitly aimed to prevent the potential negative impacts of a heavy volume of video game consumption on physical and mental health among youth⁴⁸. China’s 2019 policy attracted substantial controversy, which only intensified after its expansion in September 2021 to limit minors to only a single hour of daily playtime between 20:00 and 21:00 on Fridays, Saturdays, Sundays and public holidays⁴⁶.

…When taken together, the evidence base suggests that the impact of Chinese regulations on playtime may be far from straightforward. It is crucial for behavioral science to provide evidence of the efficacy of such legislation. In this study, we therefore investigate the effectiveness of China’s 2019 playtime regulation in reducing heavy play via two separate preregistered analyses of direct behavioral data: in the first of these, we analyse the prevalence of heavy play among more than 2.4 billion gamer profiles both before and after said regulations; in the second, we conduct a within-participants longitudinal analysis (n = 10,000) to determine whether individual gamers tended to play less heavily after restrictions were brought in.

…Datasets and preprocessing: The telemetry data used in this study span over 1m separate game identifiers, 7.04 billion hours of playtime, and ~2.4 billion gamer profiles collected from Chinese users between 16 August 2019 and 16 January 2020. Access to telemetry data was provided by Unity Technologies, makers of the Unity engine, a development environment for games. Unity estimates that there are ~5 billion downloads of apps developed with Unity every month and that Unity is used by 61% of game developers⁷⁶. The majority of games made with Unity are for mobile platforms. Games made in this engine commonly implement Unity Analytics, a play tracking service that allows developers to understand factors such as the daily playtime associated with individual users. Anonymized Unity Analytics telemetry from desktop and mobile games was the source of data for this study. It is important to note that our data were confined to this 22-week period in order to avoid bias and interference from the beginnings of the COVID-19 pandemic in China in early 2020: playtime has been shown to be heavily variable during the pandemic, with related containment and closure policies (‘lockdowns’) influencing a host of gaming-related variables¹.

The odds of heavy gaming before and after regulation: …An overall mean of 0.77% of gamer profiles engaged in heavy play before regulation and 0.88% after (Dataset 1). Formal analysis of odds ratios (OR) using Fisher’s exact test (two-sided) suggests that play tended to be statistically-significantly more heavy (p < 0.001) after regulation (OR = 1.14). However, this statistic does not reach our preregistered threshold for practical importance (OR = 2.00). A matrix showing the OR of heavy play between each week in our data is presented as Table 1; Figure 2 shows the rate of heavy play for each week in our data (Dataset 1).

…Next, in order to more closely test any possible confounding effect of binarizing our outcome on our results, we treated playtime as a continuous variable. We examined whether the mean weekly playtime for a randomly selected account in a post-regulation week still tended to be higher than a randomly selected account in a pre-regulation week. This was the case: after playtime, accounts numerically played for more hours each week. Before regulation, average playtime for any account during any given week was estimated at 1.64 hours (95% confidence interval (CI) 1.6404–1.6408). After regulation, it was estimated at 1.76 hours (95% CI 1.7582–1.7587). This suggests that the outcomes reported above are not well explained as a confounded product of our binary measurement scheme alone (Figure 4). This was formally supported by the calculation of a partially overlapping t-test⁶⁶. We compared both the mean probability that each account engaged in heavy play during the 11 weeks before regulation against the 11 weeks following regulation; and also the mean playtime for each account from the 11 weeks before regulation against the 11 weeks post-regulation. Results suggested that not only did accounts tend to be more likely to play heavily post-regulation (t = 102, d.f. = 2,321,091,203, p < 0.001) but they also tended to log statistically-significantly more hours of play (t = 267, d.f. = 2,316,728,099, p < 0.001)

…Potential confounds: …A final possible explanation is that true positive effects of regulation on minors are masked by majority adult players in our dataset. Chinese gaming company Tencent Games reports that only 6.4% of playtime in China on their games came from minors in September 2020⁶⁸. It seems probable that a similar small fraction of individuals in our dataset were underage and thus subject to restrictions. In our data, playtime appeared heavier post-regulation, albeit to a degree that did not meet our preregistered effect size threshold for practical importance (an OR of 2.0). However, it is crucial to note that we lack age information for each player in our dataset. This lack of metadata means that we cannot test whether unequal processes may be in operation simultaneously within the population under observation: for example, we cannot falsify the idea that an increase in heavy gaming among adults could be co-occurring with youth simultaneously playing less heavily. This lack of relevant demographic detail is a key limitation of the use of large-scale industry datasets such as the one employed here. We maintain that the result observed here is most plausibly explained by an ineffective policy. Nonetheless, in order to build on this work, future research must focus on generating data infrastructure: technological frameworks that allow privacy-preserving independent access to large-scale behavioral data fused to relevant self-report or demographic indicators^69,70.

[This seems to be a major limitation of this paper: if minors are 6.4% of playtime, then are they well-powered to detect the net decrease from even large decreases in minor playtime consistent with policy efficacy? If minors more than halved—surely a large practical effect—that would still only be a net decrease of −3.2% etc.]

…One possible mechanism explaining this effect would be that pre-existing adult-associated player IDs tended to play more heavily post-regulation, overshadowing reductions in heavy playtime among minor-associated IDs, because adults shared their account login details with minors post-regulation (a known loophole of the regulation)⁷². In this scenario, a single adult-associated account used by several individuals may be involved in more hours of play per day while each individual using this account was continuing to play the same amount. This would account for both a lack of evidence for an increase in account creations and a lack of evidence for an increase in total playtime in China. However, it is important to note that we did not find evidence for an increase in account creation following regulation (Figure 4). Indeed, fewer account creations (mean = 118m) occurred in the weeks following regulation than in the weeks before regulation (mean = 127m).

…A similar, further explanation would be inconsistent regulatory compliance across the games industry. Under the regulations, individual game providers are responsible for both ascertaining the real-life identity of each of their players; recording their age; and restricting their play accordingly. Very large stakeholders (such as the gaming company Tencent) have reported complying with this⁶⁸. However, our data highlight the highly federated nature of the games industry: there are over one million game identifiers in our data, which are plausibly produced by tens of thousands of separate companies. A large portion of the global games industry consists of small ‘independent’ developers, which Unity Technologies is thought to primarily capture⁷⁴. Top-down regulation may be able to secure compliance from large corporations who have the resources to effectively identify and police their player bases and have become prime targets of political intervention in China. It is less clear how compliance is easy to affect and police for thousands of small companies, particularly in light of similar noncompliance to top-down industry regulation in other parts of the globe. This uneven compliance may plausibly lead to either a lack of reduction in heavy playtime within small game companies, or even an increase, as heavy players migrate from now-regulated ‘big’ games by ‘big’, compliant companies to non-regulated ‘small’ games by non-compliant ‘small’ companies. We have seen similar phenomena in internet pornography regulation, where restriction of access to minors in one domain resulted in their displacement to unregulated spaces⁵⁵. Our Unity data chiefly consists of small company games, and such a displacement migration may appear to be consistent with the increased likelihood of heavy gaming post-regulation that we observed. However, we would suggest caution in this interpretation. It is crucial to point out that the observed OR in this study fell well below our preregistered threshold for practical importance.