AI

• “Highly accurate protein structure prediction with AlphaFold 2”, Jumper et al 2021 (source code)

• “StarCraft Commander (SCC): an efficient deep reinforcement learning agent mastering the game of StarCraft II”, Wang et al 2021

• “Predictive Coding: a Theoretical and Experimental Review”, Millidge et al 2021

• “ETA Prediction with Graph Neural Networks in Google Maps”, Derrow-Pinion et al 2021 (impressive that NNs can produce such gains on such mature systems—where’s your “AI winter” now?)

• SHRDLU didn’t work (Cyc also never worked—you ever hear of any use for it? cf. Schank1991)

Matters Of Scale:

• “Fine-tuned Language Models are Continual Learners”, Scialom et al 2022

• “Building Machine Translation Systems for the Next Thousand Languages”, Bapna et al 2022

• “Scaling Vision Transformers”, Zhai et al 2021 (testing w/n = 0.03–3b & parameters 0.05–2b using an upgraded JFT-300M dataset, ‘JFT-3B’, confirms the scaling laws: larger models continue to perform much better, more sample-efficient, incredible few-shot performance, & weight decay still useful; ImageNet top-1 classification is approaching saturation, like top-5 before it)

• “Scaling Laws for Acoustic Models”, Droppo & Elibol2021 (smooth scaling of Transformer vs LSTM audio models for log-Mel → next-word prediction; Transformers scale better: 33× per halving vs 63×)

• “NaturalProver: Grounded Mathematical Proof Generation with Language Models”, Welleck et al 2022

• “CPM-2: Large-scale Cost-effective Pre-trained Language Models”, Zhang et al 2021 (11b-dense/198b MoE Zh+En; models have been released—largest publicly-available autoregressive English models as of June 2021)

• Tabular DL: “Regularization is all you Need: Simple Neural Nets can Excel on Tabular Data”, Kadra et al 2021 (or if you prefer, Gorishniy et al 2021, Somepalli et al 2021, Liu et al 2021, Schäfl et al 2022—the last ML holdout, tabular, starts to fall to the Bitter Lesson of Transformers & carefully-regularized MLPs—although those heavily invested in tree approaches will doubtless fight to the bitter end, saying this or that hyperparameter optimization or hand-feature-engineering would reestablish some slight superiority, between this and earlier tabular NNs, the writing is on the wall, and reportedly many industry tabular-ML users have moved to DL already for its other benefits)

• “A Low-latency Communication Design for Brain Simulations”, Du et al 2022 (benchmarking 4k GPUs simulating 20b spiking neurons on Kunlun)

• “Jurassic-1: Technical Details And Evaluation”, Lieber et al 2021 (GPT-3-178b model+public API offered by new startup AI21 Labs; demos; this completes the question of GPT-3 time-lags: it took ~11 months for replication by a competitor, and ~14 months for replication+public-API)

• Big if true: Tesla AI Day2021: “Tesla Dojo’s Unique Packaging and Chip Design Allow An Order Magnitude Advantage Over Competing AI Hardware” (“Each tile has an impressive 9 PFlops of BF16/CFP8 and 36 TB/s of off-tile bandwidth. This far surpasses the off-wafer bandwidth of Cerebras, and enables the Tesla system to scale out better than even scale out designs such as the Tenstorrent architecture.”; SemiAnalysis criticism)

Genetics

Everything Is Heritable:

• “A Global Nucleic Acid Observatory for Biodefense and Planetary Health”, The Nucleic Acid Observatory Consortium2021 (Twitter)

• “Genomic evidence consistent with antagonistic pleiotropy may help explain the evolutionary maintenance of same-sex sexual behavior in humans”, Zietsch et al 2021

• “Complex feline disease mapping using a dense genotyping array”, Hernandez et al 2021 (a rare cat GWAS!)

• “Using DNA to predict behavior problems from preschool to adulthood”, Gidziela et al 2021; “Identification of 370 genetic loci for age at first sex and birth linked to externalising behavior”, Mills et al 2021

Recent Evolution:

• “25 years of selection for improved leg health in purebred broiler lines and underlying genetic parameters”, Kapell et al 2012

• “De novo evolution of macroscopic multicellularity”, Bozdag et al 2021 (Twitter; the multicellularity Long Term Evolution Experiment)

• Virophage (“a flea / Hath smaller fleas that on him prey; / And these have smaller fleas to bite ’em…”)

Engineering:

• “Gene-drive suppression of mosquito populations in large cages as a bridge between lab and field”, Hammond et al 2021

• “Generation of ovarian follicles from mouse pluripotent stem cells”, Yoshino et al 2021

• “Successful Serial Recloning in the Mouse over Multiple Generations”, Wakayama et al 2013 (“>500 viable offspring from 1 original donor mouse”)

• “RNA demethylation increases the yield and biomass of rice and potato plants in field trials”, Yu et al 2021

Statistics/Meta-Science/Math

• “Three Months in Monte Carlo” (interactive explorables of Ising model algorithms & emergent behavior, and the history of Monte Carlo algorithms)

• “John W. Tukey: His Life and Professional Contributions”, Brillinger2002

• “Causal and Associational Linking Language From Observational Research and Health Evaluation Literature in Practice: A systematic language evaluation”, Haber et al 2021 (Twitter; researchers interpret most correlational language as implying causality)

• Solving Archimedes’s Cattle Problem (WP)

• “MegaLMM: Mega-scale linear mixed models for genomic predictions with thousands of traits”, Runcie et al 2021 (cf. variance components)

• Ulam spiral

Politics/Religion

• “The Dark Side of Private Ordering: An Institutional and Empirical Analysis of Organized Crime”, Milhaupt & West2000

• “‘He who pays the piper calls the tune’: Researcher experiences of funder suppression of health behavior intervention trial findings”, McCrabb et al 2021 (“Our results, along with those of previous investigations, suggest that government funders interfere with public-good research.”)

• Tama-Re

Psychology/Biology

• “High-throughput mapping of a whole rhesus monkey brain at micrometer resolution”, Xu et al 2021

• “Principles of Cryopreservation by Vitrification”, Fahy & Wowk2015

• “Intelligence, health and death”, Deary et al 2021

• “Genetic and environmental contributions to IQ in adoptive and biological families with 30-year-old offspring”, Willoughby et al 2021a; “Parent Contributions to the Development of Political Attitudes in Adoptive and Biological Families”, Willoughby et al 2021b

• “Toxoplasmosis: Recent Advances in Understanding the Link Between Infection and Host Behavior”, Johnson & Johnson2021

• “Cognitive Behavior Therapy for Depression From an Evolutionary Perspective”, Hollon et al 2021

• “A Contamination Theory of the Obesity Epidemic”

• “Energetic Aliens”, Stephen Malina

• “Steps of Reasoning in Children and Adolescents”, Brocas & Carrillo2021 (Piagetian levels in strategy & hypothetical reasoning)

• “Male Chimpanzees Prefer Mating with Old Females”, Muller et al 2006 (so why don’t humans?)

Technology

• Space gun (“Bigger NNs will never make AGI, any more than bigger bottle rockets or compressed tubes of gunpowder could ever get you to space—” “In Project HARP, a 1960s joint United States and Canada defence project, a U.S. Navy…gun was used to fire a 180kg projectile at 3,600 m⧸s, reaching an apogee of 180 km, hence performing a suborbital spaceflight.”)

• The Manhattan Project & the Shinkolobwe mine (on the extraordinary bad luck of humanity)

• “XY problem” (when asking questions or for help, solve problems end-to-end, don’t yak-shave)

• Video game speedrunning improvements over time

• “In-place Superscalar Samplesort (IPS⁴o): Engineering In-place (Shared-memory) Sorting Algorithms”, Axtmann et al 2020 (what it takes to write a truly fast sorting algorithm for today’s hardware)

• Forgotten wiki techs: E2’s “soft links” (the E2 wiki used click-tracking to automatically build “see also” sets of links at the bottom of pages, based on the next link readers visited; writers & readers deliberately cultivate these links to create paths through the wiki, poems or short stories, commentary, criticism, voting systems or reactions… I know of no recent wikis with an equivalent.)

Economics

• “Win-win denial: The psychological underpinnings of zero-sum thinking”, Johnson et al 2021

• “The economic value of targeting aging”, Scott et al 2021

• “Two Blades of Grass: The Impact of the Green Revolution”, Gollin et al 2021 (we still underestimate the Green Revolution)

• “TV Advertising Effectiveness and Profitability: Generalizable Results From 288 Brands”, Shapiro et al 2021

• “Flash Boys 2.0: Frontrunning in Decentralized Exchanges, Miner Extractable Value, and Consensus Instability”, Daian et al 2020 (fun & games with Ethereum; see also Polymarket, the Dark Forest, bot trapping, rugpulling liquidity pools, ponzis, sniping best NFTs, hacking nonce reuses)

Philosophy

• “Buddhist Animal Release Practices: Historic, Environmental, Public Health And Economic Concerns”, Shiu & Stokes2008

• “Survey of subjective ‘God encounter experiences’: Comparisons among naturally occurring experiences and those occasioned by the classic psychedelics psilocybin, LSD, ayahuasca, or DMT”, Griffiths et al 2019

Fiction

• “Emotionally Numb: Expertise Dulls Consumer Experience”, Rocklage et al 2021 (collecting & hedonic treadmills?)

• “Avaunt”, Amble (dark-comedy: Terry Pratchett meets Rick+Glen Cook programmer-fantasy)

Miscellaneous

• Hyperart Thomasson