AI

• “Defining and Characterizing Reward Hacking”⁠, Skalseet al2022

• “Comparison of Waymo Rider-Only Crash Data to Human Benchmarks at 7.1 Million Miles”⁠, Kusanoet al2023

• RL IRL: ⁠on Google Search use of ranking & preference-learning 2015^–₄2019⁠

• “Explaining grokking through circuit efficiency”⁠, Varmaet al2023

• “PIGEON: Predicting Image Geolocations”⁠, Haaset al2023 (a much better PlaNet⁠—⁠it beat⁠ Rainbolt⁠!)

• “Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero”⁠, Schutet al2023 (human GMs can learn from AZ… but remain far inferior)

• “Large-scale statistical forecasting models reassess the unpredictability of chaotic systems”⁠, Gilpin2023

• ⁠“Random Search Wired Into Animals May Help Them Hunt: The nervous systems of foraging and predatory animals may prompt them to move along a special kind of random path called a Lévy walk to find food efficiently when no clues are available”⁠

• ⁠“Computer Backgammon”, Hans J. Berliner1980⁠ (“BKG 9.8⁠ is the first computer program to defeat a world champion at any board or card game”); “Marvin Minsky’s Vision of the Future”⁠, Bernstein1981

• “The Manga Whisperer: Automatically Generating Transcriptions for Comics”⁠, Sachdeva & Zisserman2024

• ⁠“Making Amulets with LLaMA-2”, Blackle Mori

Matters Of⁠ ⁠Scale⁠:

• “Thousands of AI Authors on the Future of AI”⁠, Graceet al2024 (dropping timelines)

• “Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training”⁠, Hubingeret al2024 (larger models better at hiding backdoors from safety training)

• “Generative Multimodal Models are In-Context Learners”⁠, Sunet al2023

• “Large Language Models Can Teach Themselves to Use Tools”⁠, Schicket al2023 (API/tool-use emerges ~0.8b-parameters)

• ⁠Phi-2: The surprising power of small language models⁠

• “Mamba: Linear-Time Sequence Modeling with Selective State Spaces”⁠, Gu & Dao2023: ⁠Attention-free models scale poorly at in-context recall/induction (Aroraet al2023⁠), which is mostly why Transformers⁠ beat them

• “ByteDance is secretly using OpenAI’s tech to build a competitor”⁠ (why is it so hard for competing LLMs to surpass GPT-3⁠.5/4? maybe because they are lying about being based on it)

• “CommonCanvas: An Open Diffusion Model Trained with Creative-Commons Images”⁠, Gokaslanet al2023

• “Character-Aware Models Improve Visual Text Rendering”⁠, Liuet al2022 (ByT5⁠ vs T5⁠ vs PaLM⁠ demonstrates BPEs are responsible for screwed-up text in images; PaLM’s scale can solve common spelling by memorization, but not generalize—creating the illusion of a solution)

• “What’s Hidden in a Randomly Weighted Neural Network?”⁠, Ramanujanet al2019 (even random nets contain, with increasing probability in size, an accurate sub-net)

Genetics

Everything Is Heritable:

• “Shared genetic architectures of educational attainment in East Asian and European populations”⁠, Chenet al2024

Recent Evolution:

• “Genetic variants underlying human bisexual behavior are reproductively advantageous”⁠, Song & Zhang2024

• “Elevated genetic risk for multiple sclerosis emerged in steppe pastoralist populations”⁠, Barrieet al2024

Engineering:

• “Concordance of whole-genome amplified embryonic DNA with the subsequently born child”⁠, Liet al2024

• “Cloned rhesus monkey lives to adulthood for first time”⁠

• “AAV1-hOTOF gene therapy for autosomal recessive deafness 9: a single-arm trial”⁠, Lvet al2024; “Gene Therapy Allows an 11-Year-Old Boy to Hear for the First Time: The genetic treatment targeted a particular kind of congenital deafness and will soon be tried in children who are younger”⁠

Statistics/Meta-Science

• ⁠“The Usefulness of Useless Knowledge”⁠, Flexner1939

• “Laboratory-acquired infections and pathogen escapes worldwide 2000^–₂₁2021: a scoping review”⁠, Blacksellet al2023

• ⁠“Making the Micropipette”, Metacelsus

• ⁠“Cracking the Code of Linear B”, Theodore Nash

• “Projecting complete redaction of clinical trial protocols (RAPTURE): redacted cross sectional study”⁠, ███████ et al 20██

• ⁠“Behind The Scenes: The Struggle For Each Paper”, Jeff Huang (2 decades of CS/HCI research)

• “Possible Bubbles of Spacetime Curvature in the South Pacific”⁠, Tippett2012

• “Solar Eclipses and the Origins of Critical Thinking and Complexity”⁠, Litina & Fernández2023 (⁠astronomy as the training wheels of Science?⁠)

Politics/Religion

• “Liberal Visions and Boring Machines: The Early History of the Channel Tunnel”⁠

• ⁠“The Cultural Power of Elite Immigrants: A Slaveowner, a Nazi, a Lutheran, a Jew, and a Forty-Eighter Walk Into a Voting Booth”, Cremieux

• “Economic Consequences of Kinship: Evidence From U.S. Bans on Cousin Marriage”⁠, Ghoshet al2023

• ⁠“Notes on the Ivory Coast”⁠, Matt Lakeman

• Fires in Edo⁠

• ⁠“Review: Demons, by Fyodor Dostoevsky”

• “Is white always the standard? Using replication to revisit and extend what we know about the leadership prototype”⁠, Obenauer & Kalsher2023 (a pre-registered⁠ failure to replicate pro-white-leader bias & evidence of anti-white-leader bias)

Psychology/Biology

• “Efficacy and safety of oral orforglipron in patients with type 2 diabetes: a multicenter, randomized, dose-response, phase 2 study”⁠, Friaset al2023

• “A vibrating ingestible bioelectronic stimulator modulates gastric stretch receptors for illusory satiety”⁠, Srinivasanet al2023

• “Randomized Trial of Ketamine Masked by Surgical Anesthesia in Depressed Patients”⁠, Liiet al2023

• “High associative neuron numbers could drive cognitive performance in corvid species”⁠, Ströckenset al2022; “An energy costly architecture of neuromodulators for human brain evolution and cognition”⁠, Castrillonet al2023; “Gene purging and the evolution of Neoave [bird] metabolism and longevity”, Nget al2023; “Tempo and Pattern of Avian Brain Size Evolution”⁠, Ksepkaet al2020

• “The value of information gathering in phage-bacteria warfare”⁠, Dahanet al2024

• “Fetching felines: a survey of cat owners on the diversity of cat (Felis catus) fetching behavior”⁠, Formanet al2023

• ⁠“How This Marine Worm Can Tell Moonglow From Sunbeams”⁠ (a clever solution: have a light-sensitive protein which is extremely fragile)

• ⁠The discovery of mesopelagic fish

• “The Woman Who Spent Five Hundred Days in a Cave: Beatriz Flamini liked to be alone so much that she decided to live underground—and pursue a world record. The experience was gruelling and surreal.”⁠ (contrary to the initial press conference, she suffered severe mental & physical health issues)

• Thomas Cochrane, 10^th Earl of Dundonald⁠

• “Nail Growth: 35 Years of Observation”⁠, Bean1980; “Effects of Sexual Activity on Beard Growth in Man”⁠, Anonymous1970

Technology

• “Ultraviolet light can kill almost all the viruses in a room. Why isn’t it everywhere?”⁠

• ⁠“Building Apollo: A review of Angle of Attack: Harrison Storms and the Race to the Moon” (project management)

• “Programming as Theory Building”⁠, Naur1985

• “Artificial Molecular Machines”⁠, Erbas-Cakmaket al2015

• ⁠“Plants Find Light Using Gaps Between Their Cells”⁠ (unexpected way to compute phototropism)

• “Thanks to This Man, Airplanes Don’t Crash Into Mountains Anymore: Don Bateman’s terrain mapping device has nearly eliminated the largest cause of death in jetliner accidents”⁠ (initially using USSR global terrain maps bought on the Russian blackmarket)

• “The World’s Fastest Road Cars—and the People Who Drive Them”⁠

• “Chaffing and winnowing”⁠: cryptography without encryption

• “Non-detection of the Tooth Fairy at Optical Wavelengths”⁠, Armstrong2012

Economics

• “Lay economic reasoning: An integrative review and call to action”⁠, Bhattacharjee & Dana2024

• “Does alleviating poverty increase cognitive performance? Short-term & long-term evidence from a randomized controlled trial”⁠, Szasziet al2023 (no); “Cognitive Biases: Mistakes or Missing Stakes?”⁠, Enkeet al2023

• ⁠“Can Rock Dust Soak Up Carbon Emissions? A Giant Experiment Is Set to Find Out”⁠

• “The Tremendous Yet Troubled State of Gaming in 2024”, Matthew Ball

Philosophy

Fiction

• “The Names of the Hare”, Anonymous (translated Heaney1982; see ⁠Ross1933⁠ for etymology)

• ⁠“SAURON, Mount Doom, and Elvish Moths: The Influence of Tolkien on Modern Science”⁠, Larsen2007

• The first successful AMV: ⁠Steamboat Willie⁠ (“Turkey in the Straw”⁠+click tracks⁠)

• ⁠“Cannibalism in the Cars”⁠, Mark Twain1868

• ⁠“I Found David Lynch’s Lost Dune II Script”⁠ in the Frank Herbert university archives, Andy Greenberg