February 2020 News
February 2020 Gwern.net newsletter with links on AI scaling and disasters; 1 book, 1 movie, and 2 opera reviews.
February 2020’s Gwern.net newsletter is now out; previous, January 2020 (archives). This is a collation of links and summary of major changes, overlapping with my Changelog; brought to you by my donors on Patreon.
Writings
-
Gwern.net features:
-
added:
-
new local link archive feature (uses SingleFile)
-
“admonitions” & multi-column lists (Markdeep-inspired)
-
auto-smallcaps Pandoc plugin (since removed)
-
-
removed:
<q>
quote syntax highlighting due to maintenance/annotation burden; removed PNG preview popups due to low screenshot quality (the Web is why we can’t have nice things) & reader complaints
-
Media
Links
Genetics
-
Everything Is Heritable:
-
“The behavioral, cellular and immune mediators of HIV-1 acquisition: New insights from population genetics”, et al 2020
-
“The transcriptional legacy of developmental stochasticity”, et al 2019 ( media; whence the nonshared-environment? Identical-quadruplet armadillos allow estimating how much is due to random prenatal transcriptome differences: a lot!)
-
“Childhood Adoption and Mental Health in Adulthood: The Role of Gene-Environment Correlations and Interactions in the UK Biobank”, et al 2019 (genetic confounding in adverse outcomes for adoptees)
-
“Is population structure in the genetic biobank era irrelevant, a challenge, or an opportunity?”, et al 2020
-
-
Recent Evolution:
-
“The Evolution of Sex Ratio Distorter Suppression Affects a 25 cM Genomic Region in the Butterfly Hypolimnas bolina”, et al 2014 (remarkable real-world demonstration of Fisher’s principle: a parasite caused a 100:1 female:male sex ratio, which after a century of stasis, collapsed to 1:1 in 5 years once a resistant mutant immigrated)
-
AI
-
“A Step Toward Quantifying Independently Reproducible Machine Learning Research”, 2019 (blog)
-
Robot startup Covariant.ai releases details: in warehouse deployments, >95% picker success, ~600 items/hour (real-world success of DRL robotics from BAIR alumni & Abbeel, using a relatively exotic mix—imitation learning + sim2real + transfer learning + meta-learning + fleet-learning—for high performance in robot arms picking up many kinds of objects)
-
“Transformers as Soft Reasoners over Language”, et al 2020 (“deep learning can’t reason”)
-
Matters Of Scale:
-
Turing-NLG: a 17b-parameter LM, showing the usual log-linear gains from increasing parameters (1.5× T5 & 2× MegatronLM) trained using ZeRO (user-friendly library for splitting a single model across GPUs & then replicating that across the cluster, scaling to 100b parameters & planning for 1000b; competitor to Mesh-Tensorflow)
-
“A Constructive Prediction of the Generalization Error Across Scales”, et al 2019 (smooth power-law scaling of NN performance with data & model size across many architectures & datasets; see previously et al 2017 /et al 2018 /et al 2018 /et al 2020 ; found via TR on OA, which confirms OA is all-in on the scaling thesis and is working on a multimodal followup to GPT-2, presumably trained with model parallelism on a cluster, like Turing-NLG using ZeRO1)
-
“REALM: Retrieval-Augmented Language Model Pre-Training”, et al 2020 ( blog; learning to query all of WP for question-answering; combo with T5 pending)
-
“Growing Neural Cellular Automata: Differentiable Model of Morphogenesis”, et al 2020 ( HN; training tiny CNNs to coordinate as cellular automata to create complex damage-resilient global patterns, fun)
-
Statistics/Meta-Science
-
“Explorable Explanations” (a gallery of interactive scientific visualizations/simulations, curated by Nicky Case; subreddit)
Politics/religion
-
“How the Horrific 1918106ya Flu Spread Across America: The toll of history’s worst epidemic surpasses all the military deaths in World War I and World War II combined. And it may have begun in the United States” (the Spanish flu & government censorship); “A Sea Story”, William Langewiesche 200420ya (the sinking of the MS Estonia & the psychology of survival among horror)
-
“This Is Why Your Holiday Travel Is Awful: The long, sordid history of New York’s Penn Station shows how progressives have made it too hard for the government to do big things—and why, believe it or not, Robert Caro is to blame” (tragedy of the anticommons in infrastructure)
Psychology/biology
-
“The reality and evolutionary significance of human psychological sex differences”, 2019 (graph of differences); “Greater male than female variability in regional brain structure across the lifespan”, et al 2020
-
“Typologies of Extreme Longevity Myths”, Young et al 201014ya; “Was Jeanne Calment the Oldest Person Who Ever Lived—or a Fraud? Some researchers have cast doubt on the record of the celebrated supercentenarian” (update on the fraud accusations: Novoselov & Zak’s claims appear to not be holding up, but unfortunately, also no progress on blood testing, and so the mystery remains)
-
“A simple procedure for the long-term cultivation of chicken embryos”, Auerbach et al 197450ya; “A Novel Shell-less Culture System for Chick Embryos Using a Plastic Film as Culture Vessels”, 2014 (DIY artificial wombs)
-
“Physical activity and weight following car ownership in Beijing, China: quasi-experimental cross sectional study”, et al 2019 (no effect of car ownership on weight; it’s a pity more regulations aren’t randomized)
-
“Perceptual convergence of multi-component mixtures in olfaction implies an olfactory white”, et al 2012
-
“In the 1970s, the CIA Created a Robot Dragonfly Spy. Now We Know How It Works.” (reflective eyes for remote laser microphone eavesdropping, powered by an engine running on lithium nitrate crystal fumes, and the operator controlled flying using infrared lasers to heat a bimetallic strip & adjust wings)
Technology
-
“‘The intelligence coup of the century’: For decades, the CIA read the encrypted communications of allies and adversaries” (2 leaked internal histories lay bare exactly how the CIA/BND backdoored Crypto AG to achieve global penetration for ~40 years, obfuscating weak designs & subsidizing marketing to gain users)
-
“All the money in the world couldn’t make Kinect happen: For a moment a decade ago, the game industry looked like a very different place” (oral history of Microsoft’s bold but ultimately failed experiment in revolutionizing UI using the Kinect motion-tracking device)
-
“Old CSS, New CSS”, Eevee (HN; why is web programming so screwed up? a history of worse-is-better from 199529ya to now)
-
“Lawrence Bragg’s role in the development of sound-ranging in World War I”, Van der Kloot 200519ya (artillery sound-ranging)
Economics
Philosophy
-
“Anesthetizing the Public Conscience: Lethal Injection and Animal Euthanasia”, Alper 200816ya (see also inverse p-zombies/anesthesia awareness, pain & learning, and Dennett’s 197846ya “Why You Can’t Make A Computer That Feels Pain”)
Fiction
-
“A Martian Sends a Postcard Home”, Craig Raine 197747ya (canonical Martian example of defamiliarization)
Misc
-
“In 196460ya, with Seven Up! Michael Apted stumbled into making what has become the most profound documentary series in the history of cinema. 55 years later, the project is reaching its conclusion.” (regrets & reflections as Apted tries to finish Up, one of the longest-running documentary film projects ever)
-
“The fascinating and ego-killing existence of human wormholes” (on the brevity of contemporary history, especially when compared to histories like ancient Egypt where the millennia roll on… A history of humanity in 10 words: “nothing much happened for a long time; then everything happened.” cf. “Timeghost”)
-
“The Great Buenos Aires Bank Heist: They were an all-star crew. They cooked up the perfect plan. And when they pulled off the caper of the century, it made them more than a fortune—it made them folk heroes” (they tunneled into vault from city sewers, staged an armed robbery to distract police, dammed the sewer, & escaped with the loot on Zodiacs while spreading fake evidence; as often with perfect crimes, the only flaw was a woman scorned)
-
“Visions of the Future” (JPL-sponsored Art Deco poster series advertising travel in the Solar System & to exoplanets)
Books
Nonfiction
-
The Hye Ch’O Diary: Memoir of the Pilgrimage to the Five Regions of India, Hyecho, translated by Yang et al 198440ya (review)
Film/TV
Live-Action
-
Porgy and Bess (Met HD opera; review)
-
GPT-3 did require model parallelism on a cluster, but actually turned out to be text-only: “GPT-3: Language Models are Few-Shot Learners”, et al 2020. ↩︎