“Writing Matters”, 2024 (; similar):
For papers to have scientific impact, they need to impress our peers in their role as referees, journal editors, and members of conference committees.
Does better writing help our papers make it past these gatekeepers? In this study, we estimate the effect of writing quality by comparing how 30 economists judge the quality of papers written by PhD students in economics. Each economist judged 5 papers in their original version and 5 different papers that had been language edited. No economist saw both versions of the same paper.
Our results show that writing matters. Compared to the original versions, economists judge edited versions as higher quality; they are more likely to accept edited versions for a conference; and they believe that edited versions have a better chance of being accepted at a good journal.
[Keywords: academic writing, writing quality, economics, randomized experiment]
…The language editing was done by two professional editors, who aimed to make the papers easier to read and understand. We then asked 18 writing experts and 30 economists to judge some of the original and edited papers. Each of these experts judged 5 papers in their original versions and 5 papers in their edited version, leading to a total estimation sample of 480 paper-expert observations (180 of those from writing experts and 300 from economists). On average, experts spent around 5 min per paper. None of them saw both versions of the same paper. None of the experts knew that some of the papers were edited. The writing experts judged the writing quality and the economists judged the academic quality of the papers. All economists in our sample have PhDs in economics, and their academic positions range from postdoc to full professor; 4 of them are editors of academic journals; and all of them are regularly involved in judging the quality of academic papers as referees or members of conference committees. We estimate the effect of language editing on perceived writing quality and perceived academic paper quality by comparing the average judgement of original and edited papers.
Our results show that writing matters. Writing experts judged the edited papers as 0.6 standard deviations (SD) better written overall (1.22 points on an 11-point scale). They further judged the language-edited papers as allowing the reader to find the key message more easily (0.58 SD), having fewer mistakes (0.67 SD), being easier to read (0.53 SD), and being more concise (0.50 SD). These large improvements in writing quality translated into still substantial effects on economists’ evaluations. Economists evaluated the edited versions as being 0.2 SD better overall (0.4 points on an 11-point scale). They were also 8.4 percentage points more likely to accept the paper for a conference, and were 4.1 percentage points more likely to believe that the paper would get published in a good economics journal.
Our heterogeneity analysis shows that the effects of language editing on writing quality and perceived academic quality are particularly large if the original versions were poorly written. [Implying diminishing returns to editing?]
…only one other paper has investigated how the writing quality of academic papers affects the evaluations of scientists. 1980 altered the writing quality of the conclusion section in 4 management papers. In contrast to our findings, his results suggest that improving the writing causes experts to evaluate papers less favorably. We improve upon Armstrong’s approach by having a larger sample, as well as a more rigorous study design and empirical analysis. For example, our study includes 30 papers (instead of 4) and the language editing was done by professional editors (instead of Armstrong, who is a good writer but not an expert editor). By asking scientists to evaluate whole papers instead of one individual section, the context of our study is also closer to how peer review is conducted in practice. Finally, since Armstrong conducted his study, attitudes towards writing have changed. We have seen the birth of the plain language movement, and several countries now require government and other agencies to write according to plain language principles.
View PDF: