“Trial by Internet: A Randomized Field Experiment on Wikipedia’s Influence on Judges’ Legal Reasoning”, 2022-07-27 (; backlinks):
[blog; cf. et al 2017, et al 2019] In the common law tradition, legal decisions are supposed to be grounded in both statute and precedent, with legal training guiding practitioners on the most important and relevant touchstones. But actors in the legal system are also human, with the failings and foibles seen throughout society. This may lead them to take methodological shortcuts, even to relying on unknown internet users for determinations of a legal source’s relevance.
In this chapter, we investigate the influence on legal judgments of a pervasive, but unauthoritative source of legal knowledge: Wikipedia.
Using the first randomized field experiment ever undertaken in this area [pairs of articles blocked by year & area of law & previous citation/media count & article-author]—the gold standard for identifying causal effects—we show that:
Wikipedia shapes judicial behavior. [Newly-written] Wikipedia articles on [Irish Supreme Court] decided cases, written by law students, guide both the decisions that judges cite as precedents [+20%] and the textual content of their written opinions. The information and legal analysis offered on Wikipedia led judges to cite the relevant legal cases more often and to talk about them in ways comparable to how the Wikipedia authors had framed them.
Collectively, our study provides clear empirical evidence of a new form of influence on judges’ application of the law—easily accessible, user-generated online content. Because such content is not authoritative, our analysis reveals a policy-gap: if easily-accessible analysis of legal questions is already being relied on, it behooves the legal community to accelerate efforts to ensure that such analysis is both comprehensive and expert.
…To stratify effectively, we combined the citation information from JustisOne [now “vLex Justis”] with other information we gathered on the presence of each case in various forms of media: the number of times a case was referred to on the website of RTÉ93 (Ireland’s national broadcaster), the website of the Irish Times94 (a daily newspaper in Ireland), and in other public media sources.95 As in our hypothetical example, we took the larger set of cases and sub-divided them into pairs. To be eligible to be a pair, cases needed to be decided in the same year and be part of the same type of law. For example, two “asylum, immigration and nationality” law cases from the year 2000. Within the potential pairs that met these first two conditions, we stratified cases based on finding their “nearest neighbor”96 according to: # positive citations, # neutral citations, # negative citations, publication year, whether mentioned in RTÉ, whether referenced more or less than the median number of times in the Irish Times, and whether mentioned more than 10× in other public media.97 For 9 cases, no sufficiently comparable nearest neighbor case could be found and they were excluded from the experiment.
The resultant pairs of cases were then given to the article writers, mostly law students. For each pair, a single student would write both cases. This guaranteed that each author had an exactly equal number of articles in the treatment and control group. Put another way, by implementing this aspect of our experimental design we automatically stratified on the author characteristics, and thus our articles were also balanced in those ways too.
…For this project we created 154 draft Wikipedia articles on Irish Supreme Court cases. The process of creation was done in 3 waves. After each wave, a random half of the articles were added to Wikipedia and the other half held back. The first wave, in early 2019, was a pilot study to understand the article creation and addition process in which law faculty in Maynooth University wrote Wikipedia articles on 14 cases. In the second wave, in Spring 2019, undergraduate law students from Maynooth University created 8 articles as part of the civic engagement stream. These were published in late 2019. In the third wave, in Fall 2019, a cohort of graduate students wrote 132 articles as part of a professional development seminar. These were published in early 2020.
…Our articles received a total of 56,733 views through January 16, 2022…Overall, we find that the addition of a Wikipedia article increases the number of citations received by that case by 0.064 per month…The difference in average monthly citations between the treated and control groups is −4.8% in the pre-treatment period, and 17.0% in the post-treatment period. Hence, we estimate that adding a Wikipedia article increases judicial citations to those cases by 21.8%.
See Also: