---
title: January 2024 News
description: January 2024 Gwern.net newsletter with links on TODO
created: 2024-01-01
status: in progress
confidence: log
css-extension: dropcaps-de-zs
backlink: False
...

January 2024's [Gwern.net](https://gwern.net/newsletter/2024/01) [newsletter](https://gwern.substack.com/ "'Gwern.net newsletter (Substack subscription page)', Gwern 2013") is now out; previous, [December 2023](https://gwern.net/newsletter/2023/12) ([archives](https://gwern.net/doc/newsletter/index)).
This is a collation of links and summary of major changes, overlapping with my [Changelog](https://gwern.net/changelog); brought to you by my donors on [Patreon](https://www.patreon.com/gwern).

# Writings

- ["Dropcap Generation With AI"](https://gwern.net/dropcap); idea: [better SVG generative models via multi-modal LLMs](https://gwern.net/idea#vector-generation)

# Links

## AI

- ["Defining and Characterizing Reward Hacking"](https://arxiv.org/abs/2209.13085), Skalse et al 2022
- ["Comparison of Waymo Rider-Only Crash Data to Human Benchmarks at 7.1 Million Miles"](https://arxiv.org/abs/2312.12675#google), Kusano et al 2023
- RL IRL: [on Google Search use of ranking & preference-learning 2015–2019](https://searchengineland.com/how-google-search-ranking-works-pandu-nayak-435395#h-navboost-system-a-k-a-glue)
- ["Explaining grokking through circuit efficiency"](https://arxiv.org/abs/2309.02390), Varma et al 2023
- ["PIGEON: Predicting Image Geolocations"](https://arxiv.org/abs/2307.05845), Haas et al 2023 (a much better [PlaNet](https://arxiv.org/abs/1602.05314#deepmind "‘PlaNet—Photo Geolocation with Convolutional Neural Networks’, Weyand et al 2016")---[it beat](https://www.npr.org/2023/12/19/1219984002/artificial-intelligence-can-find-your-location-in-photos-worrying-privacy-expert) [Rainbolt](https://en.wikipedia.org/wiki/Trevor_Rainbolt)!); ["Evaluating Precise Geolocation Inference Capabilities of Vision Language Models"](https://arxiv.org/abs/2502.14412), Jay et al 2025
- ["Bridging the Human-AI Knowledge Gap: Concept Discovery and Transfer in AlphaZero"](https://arxiv.org/abs/2310.16410#deepmind), Schut et al 2023 (human GMs can learn from AZ... but remain far inferior)
- ["Large-scale statistical forecasting models reassess the unpredictability of chaotic systems"](https://arxiv.org/abs/2303.08011 "‘Model scale versus domain knowledge in statistical forecasting of chaotic systems’, Gilpin 2023"), Gilpin 2023
- ["Random Search Wired Into Animals May Help Them Hunt: The nervous systems of foraging and predatory animals may prompt them to move along a special kind of random path called a Lévy walk to find food efficiently when no clues are available"](https://www.quantamagazine.org/random-search-wired-into-animals-may-help-them-hunt-20200611/)
- ["Computer Backgammon"](https://www.bkgm.com/articles/Berliner/ComputerBackgammon/index.html), [Hans J. Berliner 1980](https://en.wikipedia.org/wiki/Hans_Berliner) ("[BKG 9.8](https://en.wikipedia.org/wiki/Hans_Berliner#Computer_chess_research) is the first computer program to defeat a world champion at any board or card game"); ["Marvin Minsky's Vision of the Future"](https://www.newyorker.com/magazine/1981/12/14/a-i), Bernstein 1981
- ["The Manga Whisperer: Automatically Generating Transcriptions for Comics"](https://arxiv.org/abs/2401.10224), Sachdeva & Zisserman 2024
- ["Making Amulets with LLaMA-2"](https://suricrasia.online/blog/making-amulets-with-llama/), Blackle Mori

[Matters Of](https://www.reddit.com/r/mlscaling/ "'ML Scaling subreddit', Gwern 2020") [Scale](https://en.wikipedia.org/wiki/Neural_scaling_law):

- ["Thousands of AI Authors on the Future of AI"](https://arxiv.org/abs/2401.02843), Grace et al 2024 (dropping timelines)
- ["Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training"](https://arxiv.org/abs/2401.05566#anthropic), Hubinger et al 2024 (larger models better at hiding backdoors from safety training)
- ["Generative Multimodal Models are In-Context Learners"](https://arxiv.org/abs/2312.13286#baai), Sun et al 2023
- ["Large Language Models Can Teach Themselves to Use Tools"](https://arxiv.org/abs/2302.04761#facebook "‘Toolformer: Language Models Can Teach Themselves to Use Tools’, Schick et al 2023"), Schick et al 2023 (API/tool-use emerges ~0.8b-parameters)
- [Phi-2: The surprising power of small language models](https://www.microsoft.com/en-us/research/blog/phi-2-the-surprising-power-of-small-language-models/)
- ["Mamba: Linear-Time Sequence Modeling with Selective State Spaces"](https://arxiv.org/abs/2312.00752), Gu & Dao 2023: [Attention-free models scale poorly at in-context recall/induction](https://hazyresearch.stanford.edu/blog/2023-12-11-zoology1-analysis) ([Arora et al 2023](https://arxiv.org/abs/2312.04927 "Zoology: Measuring and Improving Recall in Efficient Language Models")), which is mostly why Transformers beat them
- ["ByteDance is secretly using OpenAI’s tech to build a competitor"](https://www.theverge.com/2023/12/15/24003151/bytedance-china-openai-microsoft-competitor-llm) (why is it so hard for competing LLMs to surpass GPT-3.5/4? maybe because they are lying about being based on it)
- ["CommonCanvas: An Open Diffusion Model Trained with Creative-Commons Images"](https://arxiv.org/abs/2310.16825), Gokaslan et al 2023
- ["Character-Aware Models Improve Visual Text Rendering"](https://arxiv.org/abs/2212.10562#google), Liu et al 2022 (ByT5 vs T5 vs PaLM demonstrates BPEs are responsible for screwed-up text in images; PaLM's scale can solve common spelling by memorization, but *not* generalize---creating the illusion of a solution)
- ["What's Hidden in a Randomly Weighted Neural Network?"](https://arxiv.org/abs/1911.13299), Ramanujan et al 2019 (even random nets contain, with increasing probability in size, an accurate sub-net)

## Genetics

Everything Is Heritable:

- ["Shared genetic architectures of educational attainment in East Asian and European populations"](https://www.nature.com/articles/s41562-023-01781-9), Chen et al 2024

Recent Evolution:

- ["Genetic variants underlying human bisexual behavior are reproductively advantageous"](https://www.science.org/doi/10.1126/sciadv.adj6958), Song & Zhang 2024
- ["Elevated genetic risk for multiple sclerosis emerged in steppe pastoralist populations"](https://www.nature.com/articles/s41586-023-06618-z), Barrie et al 2024

Engineering:

- ["Concordance of whole-genome amplified embryonic DNA with the subsequently born child"](https://www.medrxiv.org/content/10.1101/2024.01.12.24301086.full), Li et al 2024
- ["Cloned rhesus monkey lives to adulthood for first time"](https://www.nature.com/articles/d41586-024-00136-2 "‘Cloned rhesus monkey lives to adulthood for first time: A method that provides cloned embryos with a healthy placenta could pave the way for more research involving the primates’, Naddaf 2024")
- ["AAV1-hOTOF gene therapy for autosomal recessive deafness 9: a single-arm trial"](https://gwern.net/doc/genetics/editing/2024-lv.pdf), Lv et al 2024; ["Gene Therapy Allows an 11-Year-Old Boy to Hear for the First Time: The genetic treatment targeted a particular kind of congenital deafness and will soon be tried in children who are younger"](https://www.nytimes.com/2024/01/23/health/deaf-gene-therapy.html "Gina Kolata 2024-01-23")

## Statistics/Meta-Science

- ["The Usefulness of Useless Knowledge"](https://gwern.net/doc/science/1939-flexner.pdf), Flexner 1939
- ["Laboratory-acquired infections and pathogen escapes worldwide 2000--2021: a scoping review"](https://www.thelancet.com/journals/lanmic/article/PIIS2666-5247(23)00319-1/fulltext), Blacksell et al 2023
- ["Making the Micropipette"](https://press.asimov.com/resources/making-the-micropipette), Metacelsus
- ["Cracking the Code of Linear B"](https://antigonejournal.com/2024/01/decipherment-linear-b/), Theodore Nash
- ["Projecting complete redaction of clinical trial protocols (RAPTURE): redacted cross sectional study"](https://www.bmj.com/content/383/bmj-2023-077329), ███████ et al 20██
- ["Behind The Scenes: The Struggle For Each Paper"](https://jeffhuang.com/struggle_for_each_paper/), Jeff Huang (2 decades of CS/HCI research)
- ["Possible Bubbles of Spacetime Curvature in the South Pacific"](https://arxiv.org/abs/1210.8144), Tippett 2012
- ["Solar Eclipses and the Origins of Critical Thinking and Complexity"](https://gwern.net/doc/science/physics/astronomy/2023-litina.pdf), Litina & Fernández 2023 ([astronomy as the training wheels of Science?](https://gwern.net/review/bakewell#astronomy))

## Politics/Religion

- ["Liberal Visions and Boring Machines: The Early History of the Channel Tunnel"](https://publicdomainreview.org/essay/the-early-history-of-the-channel-tunnel/)
- ["The Cultural Power of Elite Immigrants: A Slaveowner, a Nazi, a Lutheran, a Jew, and a Forty-Eighter Walk Into a Voting Booth"](https://www.cremieux.xyz/p/the-cultural-power-of-high-skilled), Cremieux
- ["Economic Consequences of Kinship: Evidence From U.S. Bans on Cousin Marriage"](https://gwern.net/doc/economics/2023-ghosh.pdf), Ghosh et al 2023
- ["Notes on the Ivory Coast"](https://mattlakeman.org/2024/01/01/notes-on-the-ivory-coast/), Matt Lakeman
- [Fires in Edo](https://en.wikipedia.org/wiki/Fires_in_Edo)
- ["Review: _Demons_, by Fyodor Dostoevsky"](https://www.thepsmiths.com/p/review-demons-by-fyodor-dostoevsky)
- ["Is white always the standard? Using replication to revisit and extend what we know about the leadership prototype"](https://gwern.net/doc/psychology/cognitive-bias/2023-obenauer.pdf), Obenauer & Kalsher 2023 (a pre-registered failure to replicate pro-white-leader bias & evidence of anti-white-leader bias)

## Psychology/Biology

- ["Efficacy and safety of oral orforglipron in patients with type 2 diabetes: a multicenter, randomized, dose-response, phase 2 study"](https://gwern.net/doc/longevity/glp/2023-frias-2.pdf), Frias et al 2023
- ["A vibrating ingestible bioelectronic stimulator modulates gastric stretch receptors for illusory satiety"](https://www.science.org/doi/10.1126/sciadv.adj3003), Srinivasan et al 2023
- ["Randomized Trial of Ketamine Masked by Surgical Anesthesia in Depressed Patients"](https://www.medrxiv.org/content/10.1101/2023.04.28.23289210.full), Lii et al 2023
- ["High associative neuron numbers could drive cognitive performance in corvid species"](https://onlinelibrary.wiley.com/doi/10.1002/cne.25298), Ströckens et al 2022; ["An energy costly architecture of neuromodulators for human brain evolution and cognition"](https://www.science.org/doi/10.1126/sciadv.adi7632), Castrillon et al 2023; ["Gene purging and the evolution of Neoave [bird] metabolism and longevity"](https://www.jbc.org/article/S0021-9258(23)02437-7/fulltext), Ng et al 2023; ["Tempo and Pattern of Avian Brain Size Evolution"](https://www.sciencedirect.com/science/article/pii/S0960982220304309), Ksepka et al 2020
- ["The value of information gathering in phage-bacteria warfare"](https://academic.oup.com/pnasnexus/article/3/1/pgad431/7512761), Dahan et al 2024
- ["Fetching felines: a survey of cat owners on the diversity of cat (_Felis catus_) fetching behavior"](https://www.nature.com/articles/s41598-023-47409-w), Forman et al 2023
- ["How This Marine Worm Can Tell Moonglow From Sunbeams"](https://www.quantamagazine.org/how-this-marine-worm-can-tell-moonglow-from-sunbeams-20231219/) (a clever solution: have a light-sensitive protein which is *extremely* fragile)
- [The discovery of mesopelagic fish](https://hakaimagazine.com/features/all-the-fish-we-cannot-see/)
- ["The Woman Who Spent Five Hundred Days in a Cave: Beatriz Flamini liked to be alone so much that she decided to live underground—and pursue a world record. The experience was gruelling and surreal."](https://www.newyorker.com/magazine/2024/01/29/the-woman-who-spent-five-hundred-days-in-a-cave "D. T. Max 2024-01-21") (contrary to the initial press conference, she suffered severe mental & physical health issues)
- [Thomas Cochrane, 10^th^ Earl of Dundonald](https://en.wikipedia.org/wiki/Thomas_Cochrane,_10th_Earl_of_Dundonald)
- ["Nail Growth: 35 Years of Observation"](https://gwern.net/doc/nootropic/quantified-self/nail-growth/1980-bean.pdf), Bean 1980; ["Effects of Sexual Activity on Beard Growth in Man"](https://gwern.net/doc/nootropic/quantified-self/1970-anonymous.pdf), Anonymous 1970

## Technology

- ["Ultraviolet light can kill almost all the viruses in a room. Why isn’t it everywhere?"](https://www.vox.com/the-highlight/23972651/ultraviolet-disinfection-germicide-far-uv)
- ["Building Apollo: A review of _Angle of Attack: Harrison Storms_ and the Race to the Moon"](https://www.construction-physics.com/p/building-apollo) (project management)
- ["Programming as Theory Building"](https://gwern.net/doc/cs/algorithm/1985-naur.pdf), Naur 1985
- ["Artificial Molecular Machines"](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4585175/), Erbas-Cakmak et al 2015
- ["Plants Find Light Using Gaps Between Their Cells"](https://www.quantamagazine.org/plants-find-light-using-gaps-between-their-cells-20240131/) (unexpected way to compute phototropism)
- ["Thanks to This Man, Airplanes Don’t Crash Into Mountains Anymore: Don Bateman’s terrain mapping device has nearly eliminated the largest cause of death in jetliner accidents"](https://www.bloomberg.com/features/2016-bateman-airplane-safety-device/) (initially using USSR global terrain maps bought on the Russian blackmarket)
- ["The World’s Fastest Road Cars—and the People Who Drive Them"](https://www.newyorker.com/magazine/2023/12/25/the-worlds-fastest-road-cars-and-the-people-who-drive-them)
- ["Chaffing and winnowing"](https://en.wikipedia.org/wiki/Chaffing_and_winnowing): cryptography without encryption
- ["Non-detection of the Tooth Fairy at Optical Wavelengths"](https://arxiv.org/abs/1204.0492), Armstrong 2012

## Economics

- ["Lay economic reasoning: An integrative review and call to action"](https://gwern.net/doc/economics/2024-bhattacharjee.pdf), Bhattacharjee & Dana 2024
- ["Does alleviating poverty increase cognitive performance? Short-term & long-term evidence from a randomized controlled trial"](https://gwern.net/doc/iq/2023-szaszi.pdf), Szaszi et al 2023 (no); ["Cognitive Biases: Mistakes or Missing Stakes?"](https://gwern.net/doc/psychology/cognitive-bias/2023-enke.pdf), Enke et al 2023
- ["Can Rock Dust Soak Up Carbon Emissions? A Giant Experiment Is Set to Find Out"](https://www.wired.com/story/rock-dust-soak-up-carbon-emissions-climate-experiment/)
- ["The Tremendous Yet Troubled State of Gaming in 2024"](https://www.matthewball.co/all/gaming2024), Matthew Ball

## Philosophy

## Fiction

- ["The Names of the Hare"](https://ronnowpoetry.com/contents/anonymous/NamesoftheHare.html), Anonymous (translated Heaney 1982; see [Ross 1933](https://gwern.net/doc/fiction/poetry/1933-ross.pdf) for etymology)
- ["SAURON, Mount Doom, and Elvish Moths: The Influence of Tolkien on Modern Science"](https://gwern.net/doc/fiction/fantasy/2007-larsen.pdf), Larsen 2007
- The first successful AMV: [_Steamboat Willie_](https://www.youtube.com/watch?v=I5pG1wbRKOg) (["Turkey in the Straw"](https://en.wikipedia.org/wiki/Turkey_in_the_Straw)+[click tracks](https://en.wikipedia.org/wiki/Click_track))
- ["Cannibalism in the Cars"](https://gwern.net/doc/fiction/humor/1868-twain.pdf), Mark Twain 1868
- ["I Found David Lynch’s Lost _Dune II_ Script"](https://www.wired.com/story/david-lynch-dune-sequel-script-unearthed/) in the Frank Herbert university archives, Andy Greenberg

# Books

**Fiction**:

- [_Bartkira_](http://www.bartkira.com/)

# Film/TV

**Live-action**:

- [_Carmen_](https://en.wikipedia.org/wiki/Carmen)
- [_Coherence_](https://en.wikipedia.org/wiki/Coherence_(film)) (2013; [review](https://gwern.net/review/movie#coherence))
- [_Nabucco_](https://en.wikipedia.org/wiki/Nabucco)
- [_Palm Springs_](https://en.wikipedia.org/wiki/Palm_Springs_(2020_film)) (2020; [review](https://gwern.net/review/movie#palm-springs))

**Animated**:

- [_Isle of Dogs_](https://en.wikipedia.org/wiki/Isle_of_Dogs_(film)) (2018)
- ["CHAIR"](https://www.youtube.com/watch?v=5BFFsZAB2rY), Hiroshi Mori (2000)
- [_Pokémon Concierge_](https://en.wikipedia.org/wiki/Pok%C3%A9mon_Concierge)