The log-normal distribution (quincunx visualization from Li et al 2010) is a skewed distribution which is the multiplicative counterpart to the normal distribution: where the normal distribution is conceptually applicable to where many independent parts are added, the log-normal is where those parts instead multiply. A common example is latent variables multiplying to give a final output; a concrete example would be multiple successive liability threshold-like steps. Also like the normal, the log-normal enjoys many general properties such as limit theorems or preservation under multiplication/addition.¹

• Power law fits are often suggested for, but also heavily criticized: power law fits are not always carefully compared against log-normals, and sometimes turn out to fit log-normals better or be mechanistically implausible.

“On the Statistics of Individual Variations of Productivity in Research Laboratories”, Shockley1957

Some researchers are orders of magnitude more prolific and successful than others. Under a normal distribution conceptualization of scientific talent, this would be odd & require them to be many standard deviations beyond the norm on some ‘output’ variable. Shockley suggests that this isn’t so surprising if we imagine scientific research as more of a ‘pipeline’: a scientist has ideas, which feeds into background research, which feeds into a series of experiments, which feeds into writing up papers, then getting them published, then influencing other scientists, then back to getting ideas.

Each step is a different skill, which is plausibly normally-distributed, but each step relies on the output of a previous step: you can’t experiment on non-existent ideas, and you can only publish on that which you experimented on, etc. Few people have an impact by simply having a fabulous idea if they can’t be bothered to write it down. (Consider how much more impact Claude Shannon, Euler, Ramanujan, or Gauss would have had if they had published more than they did.) So if one researcher is merely somewhat better than average at each step, they may wind up having a far larger output of important work than a researcher who is exactly average at each step.

Shockley notes that with 8 variables and an advantage of 50%, the output under a log-normal model would be increased by as much as 25×, eg:

simulateLogNormal <- function(advantage, n.variables, iters=100000) {
  regular <- 1
  advantaged <- replicate(iters, Reduce(`*`, rnorm(n.variables, mean=(1+advantage), sd=1), 1))
  ma <- mean(advantaged)
  return(ma)
}
simulateLogNormal(0.5, 8)
# [1] 25.58716574

With more variables, the output difference would be larger still, and is connected to the o-ring theory of productivity. This poses a challenge to those who expect small differences in ability to lead to small output differences, as the log-normal distribution is common in the real world, and also implies that if several stages can be optimized, the remainder will become a severe bottleneck.

“The Best And The Rest: Revisiting The Norm Of Normality Of Individual Performance”, O’Boyle & Aguinis2012

“The Geometric Mean, in Vital and Social Statistics”, Galton 1879_145ya; “Ability and Income: III. The Relation Between the Distribution of Ability and the Distribution of Income”, Burt1943

Bias In Mental Testing, ch 4: §“Distribution of Achievement”, Jensen 1980_44ya; “Giftedness and Genius: Crucial Differences”, Jensen 1996_28ya; Greenberg

“Why is there only one Elon Musk? Why is there so much low-hanging fruit?”, Alexey Guzey 2020

Lotka’s law/Price’s law, Preferential attachment/Matthew effect

O-ring model; Purchase funnel

Drug Development:

• animal study methodological problems & lack of transfer

• “When Quality Beats Quantity: Decision Theory, Drug Discovery, and the Reproducibility Crisis”, Scannell & Bosley2016

• “Is Target-Based Drug Discovery Efficient? Discovery and ‘Off-Target’ Mechanisms of All Drugs”, Sadri2023

• “Artificial intelligence in drug discovery: what is realistic, what are illusions? Part 1: Ways to make an impact, and why we are not there yet: Quality is more important than speed and cost in drug discovery”, Bender & Cortés-Ciriano2021

• Psychiatric Drugs: “The Alzheimer Photo”; “Prescriptions, Paradoxes, and Perversities” (eg. chlorpromazine); “Is Pharma Research Worse Than Chance?” (see also: ketamine, MDMA, LSD, amphetamines, lithium, artificial sweeteners, & off-label drugs in general)

• discovery of supersweet corn

• discovery of GLP-1 agonists like semaglutide

“Dissolving the Fermi Paradox”, Sandberg et al 2018 (the mean estimate of the Drake equation may be high but the distribution is wide and the median is much smaller than the mean, somewhat akin to Jensen’s inequality/inequality of arithmetic and geometric means)

“Prospecting for Gold”, Cotton-Barratt2016; “Counterproductive Altruism: The Other Heavy Tail”, Kokotajlo & Oprea2020

“The Fundamentals of Heavy Tails: Properties, Emergence, & Estimation: Chapter 6: Multiplicative processes”, Nair et al 2021

“Construction of arbitrarily strong amplifiers of natural selection using evolutionary graph theory”, Pavlogiannis et al 2018

“Effectiveness is a Conjunction of Multipliers”

“The Story Construction Tells About America’s Economy Is Disturbing”

See Also: On Development Hell, Multi-Stage Selection