Gwern.net newsletter (Substack subscription page)
April 2021 News
‘newsletter’ directory
Changelog
Gwern Branwen Creating Essays on Gwern.net
Choose-Your-Own-Adventure AI Dungeon Games
GPT-2 Preference Learning for Music Generation § Decision Transformers: Preference Learning As Simple As Possible
ML Scaling subreddit
Podracer architectures for scalable Reinforcement Learning
Google details new AI accelerator chips
ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning
GSPMD: General and Scalable Parallelization for ML Computation Graphs
GPipe: Efficient Training of Giant Neural Networks using Pipeline Parallelism
GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding
DeepSpeed: Extreme-scale model training for everyone
High-performance, Distributed Training of Large-scale Deep Learning Recommendation Models (DLRMs)
DynamicEmbedding: Extending TensorFlow for Colossal-Scale Applications
RecPipe: Co-designing Models and Hardware to Jointly Optimize Recommendation Quality and Performance
From Motor Control to Team Play in Simulated Humanoid Football
Grounded Language Learning Fast and Slow
Unsupervised Speech Recognition
Anthropic raises $124 million to build more reliable, general AI systems
Cooperative AI Foundation (CAIF)
MLP-Mixer: An all-MLP Architecture for Vision
‘MLP NN’ directory
The Bitter Lesson
Data-Efficient Language-Supervised Zero-Shot Learning with Self-Distillation
Distilling the Knowledge in a Neural Network
Conceptual Captions: A Cleaned, Hypernymed, Image Alt-text Dataset For Automatic Image Captioning
Image Super-Resolution via Iterative Refinement
Diffusion Models Beat GANs on Image Synthesis
Denoising Diffusion Probabilistic Models
Generative Modeling by Estimating Gradients of the Data Distribution
Large Scale GAN Training for High Fidelity Natural Image Synthesis
Improved Denoising Diffusion Probabilistic Models
CDM: Cascaded Diffusion Models for High Fidelity Image Generation
Learning to summarize from human feedback
Grokking: Generalization Beyond Overfitting On Small Algorithmic Datasets
https://www.reddit.com/r/mlscaling/comments/n78584/grokking_generalization_beyond_overfitting_on/
2021-power-poster.png#openai
Knowledge distillation: A good teacher is patient and consistent
Scaling End-to-End Models for Large-Scale Multilingual ASR
The Shape of Learning Curves: a Review
Reward is enough
Inverse-Scaling/prize: A Prize for Finding Tasks That Cause Large Language Models to Show Inverse Scaling
Lazy, a tool for running things in idle time
GPT-J-6B: 6B JAX-Based Transformer
Aggregating Nested Transformers
Less is More: Pay Less Attention in Vision Transformers
ByT5: Towards a token-free future with pre-trained byte-to-byte models
Machine learning attacks against the Asirra CAPTCHA
Deep Big Multilayer Perceptrons for Digit Recognition
Bi-ancestral depression GWAS in the Million Veteran Program and meta-analysis in >1.2 million individuals highlight new therapeutic directions
The complete sequence of a human genome
A complete human genome sequence is close: how scientists filled in the gaps. Researchers added 200 million DNA base pairs and 115 protein-coding genes—but they’ve yet to entirely sequence the Y chromosome
A complete reference genome improves analysis of human genetic variation
Using DNA to predict intelligence
Long read sequencing of 3,622 Icelanders provides insight into the role of structural variants in human diseases and other traits
Rapid Sequencing-Based Diagnosis of Thiamine Metabolism Dysfunction Syndrome
Sense codon reassignment enables viral resistance and encoded polymer synthesis
In vivo CRISPR base editing of PCSK9 durably lowers cholesterol in primates
Partial recovery of visual function in a blind patient after optogenetic therapy
With engineered proteins, scientists use optogenetics for the first time to help a blind patient see again
Wireless multilateral devices for optogenetic studies of individual and social behaviors
Scientists Drove Mice to Bond by Zapping Their Brains With Light: The study, a tour de force in bioengineering, comes after two decades of research on brain-to-brain synchrony in people
High-throughput functional variant screens via in vivo production of single-stranded DNA
First genetically modified mosquitoes released in the United States: Biotech firm Oxitec launches controversial field test of its insects in Florida after years of push-back from residents and regulatory complications
Genomic characterization of world’s longest selection experiment in mouse reveals the complexity of polygenic traits
Surrogate broodstock to enhance biotechnology research and applications in aquaculture
Utility of polygenic embryo screening for disease depends on the selection strategy
Limit on lab-grown human embryos dropped by stem-cell body: The International Society for Stem Cell Research relaxed the famous 14-day rule on culturing human embryos in its latest research guidelines
Useful Mutants, Bred With Radiation
Correlated Failures
How a Publicity Blitz Created The Myth of Subliminal Advertising
Clarifying the Structure and Nature of Left-Wing Authoritarianism (LWA)
Book Review: The Decline and Fall of the Roman Empire
Highlights from The Decline and Fall of the Roman Empire
A connectomic study of a petascale fragment of human cerebral cortex
Saturated Reconstruction of a Volume of Neocortex
Accelerating progress in brain recording tech
The Parieto-Frontal Integration Theory (P-FIT) of intelligence: Converging neuroimaging evidence
Neuroimaging evidence for a network sampling theory of individual differences in human intelligence test performance
The neural basis of intelligence in fine-grained cortical topographies
Predicting intelligence from brain gray matter volume
‘NN sparsity’ directory
backstop#deep-bayes
[Transclude the forward-link's
context]
Do Wide and Deep Networks Learn the Same Things? Uncovering How Neural Network Representations Vary with Width and Depth
MDMA-assisted therapy for severe PTSD: a randomized, double-blind, placebo-controlled phase 3 study
Effects of Psilocybin-Assisted Therapy on Major Depressive Disorder: A Randomized Clinical Trial
https://www.newyorker.com/magazine/2021/04/05/why-animals-dont-get-lost
In The Future Of Collecting, Is Anyone Having Fun?
Linking Brain Biology to Intellectual Endowment: A Review on the Associations of Human Intelligence With Neuroimaging Data
The Best And The Rest: Revisiting The Norm Of Normality Of Individual Performance
‘leaky pipelines’ directory
A conserved strategy for inducing appendage regeneration
https://www.astralcodexten.com/p/know-your-amphetamines
Feeling Small: Exploring the Tactile Perception Limits
One Writer Enters International Competition to Play the World-Conquering Game That Redefines What It Means to Be a Geek (And a Person)
How Counting Neutrons Explains Nuclear Waste
Making Is Show Business Now
Shop Class As Soulcraft: The Case for the Manual Trades
https://www.kickstarter.com/projects/upperstory/spintronics-build-mechanical-circuits
RCTs to Scale: Comprehensive Evidence from Two Nudge Units
No causal associations between childhood family income and subsequent psychiatric disorders, substance misuse and violent crime arrests: a nationwide Finnish study of >650 000 individuals and their siblings
Parental income and mental disorders in children and adolescents: prospective register-based study
Everything You Might Want to Know about Whaling
‘Nash eXchange’ directory
Love Is the Plan the Plan Is Death
The Strange Story of Dagobert, the ‘DuckTales’ Bandit
Nordic Quack: Sweden’s bizarre tradition of watching Donald Duck cartoons on Christmas Eve