June 2020 Gwern.net newsletter with 3 new pages/essays, and links on CRISPR, population screening, AI scaling, politics, and technological unemployment.
June 2020’s Gwern.net newsletter is now out; previous, May 2020 (archives). This is a collation of links and summary of major changes, overlapping with my Changelog; brought to you by my donors on Patreon.
Writings
Media
Links
Genetics:
-
Everything Is Heritable:
-
“Efficient polygenic risk scores for biobank scale data by exploiting phenotypes from inferred relatives”, et al 2020 (PGSes are not optimal for pure predictive power; BLUP/mixed-models greatly outperform them; see 2015)
-
“Germline mutation rates in young adults predict longevity and reproductive lifespan”, et al 2020
-
Sequence everyone: “Whole-genome sequencing of rare disease patients in a national healthcare system”, et al 2020; “Genomic analyses implicate noncoding de novo variants in congenital heart disease”, et al 2020; “Genetic ancestry analysis on >93,000 individuals undergoing expanded carrier screening reveals limitations of ethnicity-based medical guidelines”, 2020; “An integrated polygenic and clinical risk tool enhances coronary artery disease prediction”, et al 2020
-
-
Recent Evolution:
-
Engineering:
-
Successful treatment of Beta Thalassemia & Sickle Cell Anemia in humans with CRISPR
-
“Preimplantation Genetic Testing for Polygenic Disease Relative Risk Reduction: Evaluation of Genomic Index Performance in 11,883 Adult Sibling Pairs”, et al 2020 (previously: et al 2019 & et al 2019 ; while I have concluded that transhumanists need not be interested in embryo selection because it will not be causing major societal changes in the next decade or two compared to the much more rapid progress in AI & it increasingly looks like more powerful paradigms will arrive too late to matter, it is still happening)
-
“A Year In, 1st Patient To Get CRISPR Gene-Editing For Sickle Cell Disease Is Thriving”
-
AI:
-
“Image Augmentations for GAN Training”, et al 2020b; et al 2020 ; et al 2020 ; et al 2020c 1 (Commentary—correct data augmentation for GANs: surprisingly simple & effective, and invented simultaneously 4 times in the past month; see also et al 2020a .)
-
“SBR: Learning to Play No-Press Diplomacy with Best Response Policy Iteration”, et al 2020 (natural language Diplomacy agents surely can’t be too much more difficult given NLM progress…)
-
“Exploration Strategies in Deep Reinforcement Learning”, Lilian Weng
-
Matters Of Scale:
-
“OpenAI API” (GPT-3-as-a-service: demos; eg. code writing video)
-
“GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding”, et al 2020
Training a 600b-parameter NN translation model for 100 languages; +13.5 BLEU; training 1t-parameter models already tested. Note that while 600b is impressive, as a mixture of experts (MoE), it is more like thousands of much smaller highly-redundant models pasted together—this has great performance advantages in training & querying, but means it is more analogous to a regular NMT model with, say, 60b-parameters rather than 600b.
-
Literally GPT-2 but for 64px images—simply train a big Transformer to predict the next pixel, and it catches up with the fancier semi-supervised methods like SimCLR. This eschews any metric learning or bootstrapping or non-quadratic attention mechanisms like Sparse Transformers, but with enough compute still works… “Attention is all you need”? The new efficient attentions may break the quadratic bottleneck limiting current uses.
-
2018 AI budgets: OpenAI: ~$51m ($31m cloud compute); DeepMind: ~$700m2
-
Politics/religion:
-
Fads: “Accelerating dynamics of collective attention”, Lorenz-et al 2019 (the issue-attention cycle speeds up? See also “The universal decay of collective memory and attention”, et al 2019)
-
“Radical Chic: That Party at Lenny’s”, Tom Wolfe 1970 (previously: “Mau-Mauing the Flak Catchers”)
-
“Agenda Seeding: 1960s Black Protests Moved Elites, Public Opinion and Voting”, 2020
-
“The Cost of Torture: Evidence from the Spanish Inquisition”, 2020
-
“Breast-Feeding of Animals by Women: Its Socio-Cultural Context and Geographic Occurrence”, 1982 (from “Did Breast-Feeding Play A Role In the Evolution of Pets?” via “Animal Ethics and Evolutionary Psychology—10 ideas”)
Psychology/biology:
-
“The Curse of Konzo: In 198143ya, an international group of doctors identified the devastating disease behind a perplexing outbreak of paralysis in northern Mozambique” (the investigation of cyanide poisoning by, among others, Hans Rosling)
-
Scintillating scotoma (‘typical mind fallacy’ examples: 1% of the population has occasional hour-long visual blindness/hallucination for no particular reason)
-
“Why Humans Totally Freak Out When They Get Lost: People really do circle past the same tree over and over again—it doesn’t just happen in movies” (why redirected walking works—we’re not good at this)
-
“Increased weight loading reduces body weight and body fat in obese subjects—A proof of concept randomized clinical trial”, et al 2020 (the rodent ‘gravitostat’ replicates in humans with weight vests? previously: et al 2018 /et al 2018 /2018/et al 2019 . This raises so many questions if it’s really real: which bones? Can we use springs on our legs instead? Do standing desks help? What about acute stresses like squatting, or just holding, say, 140kg on our shoulders for 5 minutes? How simple is the chemical signaling of bone stress? Are the receptors druggable?)
-
“Human postprandial responses to food and potential for precision nutrition”, et al 2020
-
“Fast food outlets, physical activity facilities, and obesity among adults: a nationwide longitudinal study from Sweden”, et al 2020; “Association Between Lottery Prize Size and Self-reported Health Habits in Swedish Lottery Players”, Östling et al 2020 (population registries: where correlations go to die—note that the lottery gets much more precise & near-zero causal effect estimates than previous work, exemplifying the iron & stainless steel laws)
-
“Health Recommendations and Selection in Health Behaviors”, 2020 (self-fulfilling prophecies in nutrition/diet)
-
Johan Bjorksten’s cross-linkage theory of aging research (predecessors to LysoSENS) (“one man’s modus ponens is another man’s modus tollens”)
-
“On the Potential of Silicon as a Building Block for Life”, et al 2020 (generally infeasible except possibly in a sulfuric acid environment)
Technology:
-
“A Group Is Its Own Worst Enemy”, Clay Shirky 200321ya/2005
-
“The Monotype 4-Line System for Setting Mathematics”, 2007 (the Monotype system brought the black art of boiling lead & mathematical typesetting to perfection just in time to be obsoleted by computers—famously spurring Knuth to invent TeX, eventually giving academia the omnipresent look of LaTeX; see also The Printing of Mathematics)
-
“Real-world dynamic programming: seam carving”, Avik Das (explanation of 2007’s seam carving, a fun & simple image editing trick exploiting dynamic programming)
-
“The Incredible Story of the U.S. Army’s Earth-Shaking, Off-Road Land Trains” (the 174-meter-long TC-497 Overland Train Mark II, designed for building the Arctic DEW, could haul 150 tons >640km; its predecessor, the VC-22 Sno-Freighter was designed & built in a month)
-
“When SimCity got serious: the story of Maxis Business Simulations and SimRefinery”
Economics:
Philosophy:
-
“Schrödinger’s Zombie: Adam Brown” (the quantum bomb tester can ‘photograph’ objects & ‘run’ computers to compute counterfactually without any interaction; what if instead you use a simulation or a real brain? Keith Bowden suggests it could test whether you have a quantum soul.)
Books
Nonfiction:
-
The Printing of Mathematics, et al 1954
Music
Misc:
-
“Pumped Up Kicks” (Hildegard Von Blingin’ {2020}) [folk]