AI

• “Highly accurate protein structure prediction with AlphaFold 2”, Jumper et al 2021 (source code)
• “StarCraft Commander (SCC): an efficient deep reinforcement learning agent mastering the game of StarCraft II”, Wang et al 2021
• “Predictive Coding: a Theoretical and Experimental Review”, Millidge et al 2021
• “ETA Prediction with Graph Neural Networks in Google Maps”, Derrow-Pinion et al 2021 (impressive that NNs can produce such gains on such mature systems—where’s your “AI winter” now?)

Matters Of Scale:

• “From talking to OpenAI, GPT-4 will be about 100 trillion parameters. That won’t be ready for several years.” (Cerebras CEO on new clustering & software capabilities⁠; note that Sam Altman at the 2021-09-05 SSC Q&A described this as misleading, and to paraphrase, that it was highly speculative discussion a while ago & GPT-4 might never exist, total compute is much more important than parameter count, and they are more interested in making more efficient use of existing models by better prompting, continuous training, & human feedback)
• “Jurassic-1: Technical Details And Evaluation”, Lieber et al 2021 (GPT-3-178b model+public API offered by new startup AI21 Labs⁠; demos⁠; this completes the question of GPT-3 time-lags: it took ~11 months for replication by a competitor, and ~14 months for replication+public-API)
• Tesla AI Day2021: “Tesla Dojo’s Unique Packaging and Chip Design Allow An Order Magnitude Advantage Over Competing AI Hardware” (“Each tile has an impressive 9 PFlops of BF16/​CFP8 and 36 TB/​s of off-tile bandwidth. This far surpasses the off-wafer bandwidth of Cerebras, and enables the Tesla system to scale out better than even scale out designs such as the Tenstorrent architecture.”; SemiAnalysis criticism)
• “ERNIE 3.0: Large-scale Knowledge Enhanced Pre-training for Language Understanding and Generation”, Sun et al 2021 (demo⁠; #1 SuperGLUE⁠, exceeding human baseline & Microsoft Research performance)
• “Structured Denoising Diffusion Models (D3PMs) in Discrete State-Spaces”, Austin et al 2021; “Beyond In-Place Corruption: Insertion and Deletion In Denoising Probabilistic Models”, Johnson et al 2021; “Argmax Flows and Multinomial Diffusion: Learning Categorical Distributions”, Hoogeboom et al 2021a; “ARDMs: Autoregressive Diffusion Models”, Hoogeboom et al 2021b; “SUNDAE: Step-unrolled Denoising Autoencoders for Text Generation”, Savinov et al 2021 “Zero-Shot Translation using Diffusion Models”, Nachmani & Dovrat2021; “Diffusion-LM Improves Controllable Text Generation”, Li et al 2022 (DDPMs for text; potentially better global coherency as the entire text is updated repeatedly, in a much more natural way allowing for turning compute into performance & exploring alternatives… I suspect a diffusion text model will be largely immune to the repetition trap. Good for top-down story generation? Or useful for anagram/​palindrome generation? Anagram generators are useless past 1–2 words due to combinatorial explosion; but a DDPM could handle almost arbitrary sized inputs to reconstruct meaningful sentences. Training noise = shuffling letters. To create: take a target string, random shuffle, and denoise. Do this repeatedly for multiple anagrams of the original target. Can use embeddings to guide denoising towards semantically different or identical anagrams of original. Applications in CS, genomics?)

Genetics

Everything Is Heritable:

• “A Global Nucleic Acid Observatory for Biodefense and Planetary Health”, The Nucleic Acid Observatory Consortium2021 (Twitter)
• “Genomic evidence consistent with antagonistic pleiotropy may help explain the evolutionary maintenance of same-sex sexual behaviour in humans”, Zietsch et al 2021
• “Complex feline disease mapping using a dense genotyping array”, Hernandez et al 2021 (a rare cat GWAS!)
• “Using DNA to predict behaviour problems from preschool to adulthood”, Gidziela et al 2021; “Identification of 370 genetic loci for age at first sex and birth linked to externalising behaviour”, Mills et al 2021

Recent Evolution:

• “De novo evolution of macroscopic multicellularity”, Bozdag et al 2021 (Twitter⁠; the multicellularity Long Term Evolution Experiment)
• Virophage (“a flea / Hath smaller fleas that on him prey⁠; / And these have smaller fleas to bite ’em…”)

Engineering:

• “Gene-drive suppression of mosquito populations in large cages as a bridge between lab and field”, Hammond et al 2021
• “Generation of ovarian follicles from mouse pluripotent stem cells”, Yoshino et al 2021
• “RNA demethylation increases the yield and biomass of rice and potato plants in field trials”, Yu et al 2021

Statistics/​Meta-Science/​Math

• “Three Months in Monte Carlo” (interactive explorables of Ising model algorithms & emergent behavior, and the history of Monte Carlo algorithms)
• “John W. Tukey: His Life and Professional Contributions”, Brillinger2002
• “Causal and Associational Linking Language From Observational Research and Health Evaluation Literature in Practice: A systematic language evaluation”, Haber et al 2021 (Twitter⁠; researchers interpret most correlational language as implying causality)
• Solving Archimedes’s Cattle Problem (WP)
• “MegaLMM: Mega-scale linear mixed models for genomic predictions with thousands of traits”, Runcie et al 2021 (cf. variance components)
• Ulam spiral

Politics/​Religion

• “The Dark Side of Private Ordering: An Institutional and Empirical Analysis of Organized Crime”, Milhaupt & West2000
• “‘He who pays the piper calls the tune’: Researcher experiences of funder suppression of health behaviour intervention trial findings”, McCrabb et al 2021 (“Our results, along with those of previous investigations, suggest that government funders interfere with public-good research.”)
• Tama-Re

Psychology/​Biology

• “High-throughput mapping of a whole rhesus monkey brain at micrometer resolution”, Xu et al 2021
• “Principles of Cryopreservation by Vitrification”, Fahy & Wowk2015
• “Intelligence, health and death”, Deary et al 2021
• “Genetic and environmental contributions to IQ in adoptive and biological families with 30-year-old offspring”, Willoughby et al 2021a; “Parent Contributions to the Development of Political Attitudes in Adoptive and Biological Families”, Willoughby et al 2021b
• “Toxoplasmosis: Recent Advances in Understanding the Link Between Infection and Host Behavior”, Johnson & Johnson2021
• “Cognitive Behavior Therapy for Depression From an Evolutionary Perspective”, Hollon et al 2021
• “A Contamination Theory of the Obesity Epidemic”
• “Energetic Aliens”⁠, Stephen Malina
• “Steps of Reasoning in Children and Adolescents”, Brocas & Carrillo2021 (Piagetian levels in strategy & hypothetical reasoning)

Technology

• Space gun (“Bigger NNs will never make AGI, any more than bigger bottle rockets or compressed tubes of gunpowder could ever get you to space—” “In Project HARP⁠, a 1960s joint United States and Canada defence project, a U.S. Navy…gun was used to fire a 180kg projectile at 3,600 m⁄s, reaching an apogee of 180 km, hence performing a suborbital spaceflight.”)
• The Manhattan Project & the Shinkolobwe mine (on the extraordinary bad luck of humanity)
• “XY problem” (when asking questions or for help, solve problems end-to-end⁠, don’t yak-shave)
• Video game speedrunning improvements over time
• “In-place Superscalar Samplesort (IPS⁴o): Engineering In-place (Shared-memory) Sorting Algorithms”, Axtmann et al 2020 (what it takes to write a truly fast sorting algorithm for today’s hardware)
• Forgotten wiki techs: E2’s “soft links” (the E2 wiki used click-tracking to automatically build “see also” sets of links at the bottom of pages, based on the next link readers visited; writers & readers deliberately cultivate these links to create paths through the wiki, poems or short stories, commentary, criticism, voting systems or reactions… I know of no recent wikis with an equivalent.)

Economics

• “Win-win denial: The psychological underpinnings of zero-sum thinking”, Johnson et al 2021
• “The economic value of targeting aging”, Scott et al 2021
• “Two Blades of Grass: The Impact of the Green Revolution”, Gollin et al 2021 (we still underestimate the Green Revolution)
• “TV Advertising Effectiveness and Profitability: Generalizable Results From 288 Brands”, Shapiro et al 2021
• “Flash Boys 2.0: Frontrunning in Decentralized Exchanges, Miner Extractable Value, and Consensus Instability”, Daian et al 2020 (fun & games with Ethereum⁠; see also Polymarket⁠, the Dark Forest⁠, bot trapping⁠, rugpulling liquidity pools⁠, ponzis⁠, sniping best NFTs⁠, hacking nonce reuses)

Philosophy

• “Buddhist Animal Release Practices: Historic, Environmental, Public Health And Economic Concerns”, Shiu & Stokes2008
• “Survey of subjective ‘God encounter experiences’: Comparisons among naturally occurring experiences and those occasioned by the classic psychedelics psilocybin, LSD, ayahuasca, or DMT”, Griffiths et al 2019

Fiction

• “Emotionally Numb: Expertise Dulls Consumer Experience”, Rocklage et al 2021 (collecting & hedonic treadmills?)
• “Don’t Make Me Think”⁠, Zero HP Lovecraft (Twitter)
• “Avaunt”⁠, Amble (dark-comedy: Terry Pratchett meets Rick+Glen Cook programmer-fantasy)

Miscellaneous

• Hyperart Thomasson