AI

• “EfficientZero: Mastering Atari Games with Limited Data”, Ye et al 2021 (beating humans on ALE-100k/​2h by adding self-supervised learning to MuZero-Reanalyze); “Procedural Generalization by Planning with Self-Supervised World Models”, Anand et al 2021 (generalization capabilities of MuZero-Reanalyze: self-supervised learning also leads to new SOTA on ProcGen⁠, w/​implicit meta-learning on MetaWorld)
• “DORA: No-Press Diplomacy from Scratch”, Bakhtin et al 2021 (previously: Gray et al 2020)
• “Monkey Plays Pac-Man with Compositional Strategies and Hierarchical Decision-making”, Yang et al 2021 (videos)
• “Fake It Till You Make It: Face analysis in the wild using synthetic data alone”, Wood et al 2021 (modern ML-aided CGI pipelines are amazing)
• “Acquisition of Chess Knowledge in AlphaZero”, McGrath et al 2021
• “KaraSinger: Score-Free Singing Voice Synthesis with VQ-VAE using Mel-spectrograms”, Liao et al 2021 (I am easily amused by AI papers with sung abstracts; similar to Jukebox)

Matters Of Scale:

• “Open-Ended Learning Leads to Generally Capable Agents”, Open Ended Learning Team et al 2021 (blog)

• “Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model”, Smith et al 2022 (blog⁠; 3× GPT-3 & using The Pile⁠, but undertrained, so only modestly better); “Yuan 1.0: Large-Scale Pre-trained Language Model in Zero-Shot and Few-Shot Learning”, Wu et al 2021 (media⁠; 245.7b-parameter GPT-3-like Chinese text model)

• “China Has Already Reached Exascale—On Two Separate Systems” (anonymous source claims 4.4 exaflops FP16; but kept secret? Concerning arms race dynamics if true)

• “Recursively Summarizing Books with Human Feedback”, Wu et al 2021

• “Exploring the Limits of Large Scale Pre-training”, Abnar et al 2021

• “FLAN: Finetuned Language Models Are Zero-Shot Learners”, Wei et al 2021 (blog)

• “Unsupervised Neural Machine Translation with Generative Language Models Only”, Han et al 2021; “Scaling Laws for Neural Machine Translation”, Ghorbani et al 2021; “Data and Parameter Scaling Laws for Neural Machine Translation”, Gordon et al 2021

  • “Training Verifiers to Solve Math Word Problems”, Cobbe et al 2021 (blog⁠; boosting GPT-3 on math word problems from ~15% to ~60% by self-distilling a critic & best-of = 100 sampling)

• “Program Synthesis with Large Language Models”, Austin et al 2021 (smooth scaling like Codex continues to 137b-parameter LaMDA⁠; enables dialogue, solving math problems); “Show Your Work: Scratchpads for Intermediate Computation with Language Models”, Anonymous et al 2021 (AQUA-RAT might be worth revisiting)

• “On the Opportunities and Risks of Foundation Models”, Bommasani et al 2021

• “A Universal Law of Robustness via Isoperimetry”, Bubeck & Sellke2021 (Twitter⁠, talk⁠; blessings of scale—scaling is also magic pixie dust for adversarial attacks, bitter-lessoning an entire academic field of ever more elaborate (yet failed) defenses…?); “Why Robust Generalization in Deep Learning is Difficult: Perspective of Expressive Power”, Li et al 2022

• “On the Predictability of Pruning Across Scales”, Rosenfeld et al 2020 (scaling laws for sparsity: initially free large size reductions, then power-law worsening, then plateau at tiny but bad models)

Genetics

Everything Is Heritable:

• new GIANT n = 5.4m height GWAS explains full SNP heritability of height: “A Saturated Map of Common Genetic Variants Associated with Human Height from 5.4 Million Individuals of Diverse Ancestries”, Yengo et al 2022 (media⁠; height becomes the first human GWAS to ‘finish’ by reaching the ceiling of SNP heritability; so much for ‘missing heritability’! It took barely a decade to go from the first height GWASes of n = 4k  / ​ 0.3% variance to complete prediction. Exponentials are a helluva drug.)

  • “Improving GWAS discovery and genomic prediction accuracy in Biobank data”, Orliac et al 2021

• “Rare variant contribution to human disease in 281,104 UK Biobank exomes”, Wang et al 2021; “Protein-coding repeat polymorphisms strongly shape diverse human phenotypes”, Mukamel et al 2021; “The sequences of 150,119 genomes in the UK Biobank”, Halldorsson et al 2021

• “Discovery of 42 Genome-Wide Statistically-Significant Loci Associated with Dyslexia”, Doust et al 2021

• “Somatic mutation rates scale with lifespan across mammals”, Cagan et al 2021

• “100,000 Genomes Pilot on Rare-Disease Diagnosis in Health Care—Preliminary Report”, The 100k Genomes Project Pilot Investigators2021 (reminder: we should WGS sequence everyone); “Ultrarapid Nanopore Genome Sequencing in a Critical Care Setting”, Gorzynski et al 2022

Recent Evolution:

• “A selection pressure landscape for 870 human polygenic traits”, Song et al 2021
• “The genetic basis of tail-loss evolution in humans and apes”, Xia et al 2021 (progress towards catgirls, or, Return To Monke? Rudimentary human tails still appear sometimes)

Engineering:

• “How Silicon Valley hatched a plan to turn blood into human eggs: A well-connected startup company [Conception] is trying to rewrite the rules of reproduction”
• “CRISPR Gene-Editing Experiment Partly Restores Vision In Legally Blind Patients”
• “In a First, Surgeons Attached a Pig Kidney to a Human—and It Worked” (and then into the abdomen as well); “Man Receives a Heart From a Genetically Altered Pig” (unsurprisingly, ultimately failed—but progress)
• “A Cure for Type 1 Diabetes? For One Man, It Seems to Have Worked. A new treatment using stem cells that produce insulin has surprised experts and given them hope for the 1.5 million Americans living with the disease”

Statistics/​Meta-Science

• “Nuclear Fission: Reaction to the Discovery in 1939”, Badash et al 1986 (what does it look & feel like to be in the early stages of creating an x-risk like nuclear bombs? eg. Nobel physicist Millikan’s assertion that nuclear energy is a “hobgoblin” & men should “sleep in peace with the consciousness that the Creator has put some foolproof elements into his handiwork and that man is powerless to do it any titanic physical damage”—one striking flaw is the repeated reliance on equilibrium/​anthropic Outside View arguments, that ‘nuclear risks must be impossible because the Earth and natural ore deposits still exist’, which is irrelevant to the question of the dangers of creating new technologies such as with highly-concentrated ores)
• “Serendipity: Towards a taxonomy and a theory”, Yaqub2018
• “Truncating Bar Graphs Persistently Misleads Viewers”, Yang et al 2021

Politics/​Religion

• “Good news on climate change” (emissions progress; tail risks overestimated; cost/​experience curves underestimated); “Advice to Young People, as You Face Annihilation”⁠, Roger Hallam (co-founder of Extinction Rebellion); “Stop Telling Kids They’ll Die From Climate Change: Many young people feel like their future is in peril. To make progress on climate change, we must move past doomsday scenarios”⁠, Hannah Ritchie (Our World in Data)
• “Genes, Ideology, and Sophistication”, Kalmoe & Johnson2021 (many peoples’ politics are too inchoate to have stable causes like genes or upbringing)
• “Long-term Health and Social Outcomes in Children and Adolescents Placed in Out-of-Home Care”, Sariaslan et al 2021 (foster homes may be bad)

Psychology/​Biology

• “A Multisite Preregistered Paradigmatic Test of the Ego-Depletion Effect”, Vohs et al 2021 (large failure to replicate ego-depletion, strong support for null; as predicted)
• “Physical principles for scalable neural recording”, Marblestone et al 2013; “Can One Concurrently Record Electrical Spikes from Every Neuron in a Mammalian Brain?”, Kleinfeld et al 2019; “Tracking neural activity from the same cells during the entire adult life of mice”, Zhao et al 2021; “Imaging intact human organs with local resolution of cellular structures using hierarchical phase-contrast tomography”, Walsh et al 2021
• “Computation in the human cerebral cortex uses less than 0.2 watts yet this great expense is optimal when considering communication costs”, Levy & Calvert2020 (communication 20× energy of computation; 7 bits/​sec/​neuron; similar efficiency to rats—humans aren’t special, we just have big brains)
• “Energy compensation and adiposity in humans”, Careau et al 2021; “The future of weight loss”⁠, Stephan J. Guyenet (on semaglutide); “Anti-obesity drug discovery: advances and challenges”, Müller et al 2021
• Molecular recording: “Time-tagged ticker tapes for intracellular recordings”, Lin et al 2021 (Twitter); “Recording of cellular physiological histories along optically readable self-assembling protein chains”, Linghu et al 2021; “Infinite re-reading of single proteins at single-amino-acid resolution using nanopore sequencing”, Brinkerhoff et al 2021; “Molecular recording of sequential cellular events into DNA”, Loveless et al 2021; “Multiplex genomic recording of enhancer and signal transduction activity in mammalian cells”, Chen et al 2021; “A temporally resolved, multiplex molecular recorder based on sequential genome editing”, Choi et al 2021

Technology

• “Preliminary Study Of The Nuclear Subterrene”, Robinson et al 1971 (background, and outcome)
• “A Brief History of the Index”⁠, I Love Typography
• “Better eats: The kitchen of 2020 looks mostly the same as that of 1960. But what we do in it has changed dramatically, almost entirely for the better—due to a culture of culinary innovation”
• “The YouTube Revolution in Knowledge Transfer”⁠, Samo Burja

Economics

• “Experienced well-being rises with income, even above $75,000 per year”, Killingsworth2021; “The impact of cash transfers on subjective wellbeing and mental health in low-income and middle-income countries: A systematic review and meta-analysis”, McGuire et al 2020
• “Empirical audit and review and an assessment of evidentiary value in research on the psychological consequences of scarcity”, O’Donnell et al 2021
• “Only the Bad Die Young: Restaurant Mortality in the Western US”, Luo & Stark2014 (not as bad as I thought: 17% first-year mortality, median 4.5 years)

Philosophy

• “Why Philosophers Should Care About Computational Complexity”, Aaronson2011
• “Language and thought are not the same thing: evidence from neuroimaging and neurological patients”, Fedorenko & Varley2016
• “Fundamentally Misunderstanding Visual Perception: Adults’ Belief in Visual Emissions”, Winer et al 2002 (most children & many college-educated adults believe you see by shooting beams from your eyes & can feel stares; it is difficult to durably correct their intuition)

Fiction

• “Shovel Knight Drop Caps: Bitmap Graphics Reimagined as Medieval Woodblock Prints”⁠, Jamie Clarke (custom initials for a game⁠; handsome, and perhaps an under-represented idea? Robert Jordan’s Wheel Of Time was memorable in part for the chapter icons which hinted at the topic & helped the reader keep their place in such long novels; wouldn’t web serials like A Practical Guide To Evil benefit from a set of drop caps, which could also depict character progression? A few hundred dollars in commissioned art could add a lot of visual flavor!)

Miscellaneous

• New London School explosion (the worst school disaster in American history)
• The Pink Panthers (New Yorker)