AI

• “Magnetic control of tokamak plasmas through deep reinforcement learning”, Degrave et al 2022

Matters Of Scale:

• “Program Synthesis with Large Language Models”, Austin et al 2021 (smooth scaling like Codex continues to 137b-parameter LaMDA; enables dialogue, solving math problems); “Show Your Work: Scratchpads for Intermediate Computation with Language Models”, Anonymous et al 2021 (AQUA-RAT might be worth revisiting)

• “The Effects of Reward Misspecification: Mapping and Mitigating Misaligned Models”, Pan et al 2022 (“phase transitions: capability thresholds at which the agent’s behavior qualitatively shifts”)

• “CoCa: Contrastive Captioners are Image-Text Foundation Models”, Yu et al 2022 (86% ImageNet zero-shot)

• CLIPIT PixelDraw (Colab; high-quality pixel art via CLIP which Just Works™, using the CLIPDraw approach; this is particularly striking given the failure of pixel art generative models to date, like the endless Pokemon sprite attempts—apparently it really did need n = 400m before pixel art levels of visual abstraction could work… See also “Projected GANs Converge Faster”, Sauer et al 2021 which gets its good Pokemon via ImageNet/CLIP)

• “Make Every feature Binary (MEB): A 135b-parameter sparse neural network for massively improved search relevance” (an extremely large embedding for text ranking, now powering 100% of Microsoft Bing search queries)

• “Partial success in closing the gap between human and machine vision”, Geirhos et al 2021 (“We have to admit that we view our results concerning the benefits of increasing dataset size by 1–3 orders of magnitude with mixed feelings. On the one hand, ‘simply’ training standard models on (a lot) more data certainly has an intellectually disappointing element—particularly given many rich ideas in the cognitive science and neuroscience literature…”); “Evaluating Machine Accuracy on ImageNet”, Shankaar et al 2021 (“the latest models from 2020 are on par with our best human labeler”); “When does dough become a bagel? Analyzing the remaining mistakes on ImageNet”, Vasudevan et al 2022 (“CoCa-FT gets 42 of the 68 [remaining hard errors] correct”)

• “What Changes Can Large-scale Language Models Bring? Intensive Study on HyperCLOVA: Billions-scale Korean Generative Pretrained Transformers”, Kim et al 2021 (Naver)

• DeepMind2020 budget reaches $1b ($1.06b, +$0.085b)

• “M6-10T: A Sharing-Delinking Paradigm for Efficient Multi-Trillion Parameter Pretraining”, Anonymous2021 (easy GPT-10t training via weight-tying: pretrain a single module/layer that fits in nodes, then ‘unroll’ it to the fullsize model for finetuning)

• “Persia: A Hybrid System Scaling Deep Learning Based Recommenders up to 100 Trillion Parameters”, Lian et al 2021 (Kuaisho)

Genetics

Everything Is Heritable:

Recent Evolution:

Engineering:

Statistics/Meta-Science

Politics/Religion

Psychology/Biology

• “Adolescent cannabis use and adult psychoticism: A longitudinal co-twin control analysis using data from two cohorts”, Schaefer et al 2021 (“no”)

• Clonally transmissible cancer

• “How Long Does It Take Ordinary People To ‘Get Good’ At Chess?”, Joseph Wong

• “Did we find a copycat? ‘Do as I Do’ in a domestic cat (Felis catus)”, Fugazza et al 2020

• “Speed-accuracy trade-off in plants”, Ceccarini et al 2020

• “Wet mammals shake at tuned frequencies to dry”, Dickerson et al 2021 (Sandberg thread)

• Human echolocation

• Impossible colors (“Hello, yes, I would like a blacker black, also, a ‘self-luminous red’ which is simultaneously red & brighter than white”—real colors, from the demented ravings of madmen; THEY HAVE PLAYED US FOR FOOLS!)

• “Direct voluntary control of pupil constriction and dilation: Exploratory evidence from pupillometry, optometry, skin conductance, perception, and functional MRI”, Eberhardt et al 2021

Technology

• History of Air Guns

• “AIR-FI: Generating Covert Wi-Fi Signals from Air-Gapped Computers”, Guri2020 (some fun earlier side-channels: Guri et al 2015a, Guri et al 2015b)

• “Syntax highlighting is a waste of an information channel”

• “Ditherpunk—The article I wish I had about monochrome image dithering”

• Block writing:

  • Ed Interlock

  • Anita Jürgeleit’s2016 “Hangulatin”: a Hangul-inspired font for English/German word blocks ; Sebastian Moser’s “Sori”

  • Nihon Typeface

  • Xu Bing’s “Square Word Calligraphy” (Omniglot)

  • Square Kufic (eg. ISIS)

    • Calcula (review)

  • Cyrillic Vyaz

Economics

• “What is the Commons Worth? Estimating the Value of Wikimedia Imagery by Observing Downstream Use”, Erickson et al 2018; “Digitization and the Demand for Physical Works: Evidence from the Google Books Project”, Nagaraj & Reimers2019 (copyright deadweight losses)

• “Potterian Economics”, Levy & Snir2017

• “The Theory of Interstellar Trade”, Krugman1978/2010

• “Should subscription-based content creators display their earnings on crowdfunding platforms? Evidence from Patreon”, Crosby & McKenzie2021 (I’m giving disabling my public display a try… Doesn’t seem to be working thus far.)

Philosophy

Fiction

Miscellaneous