Bibliography (92):

  1. Gwern.net newsletter (Substack subscription page)

  2. August 2020 News

  3. ‘newsletter’ directory

  4. Changelog

  5. Gwern Branwen Creating Essays on Gwern.net

  6. Instant.page: Make your site’s pages instant in 1 minute and improve your conversion rate noticeably.

  7. asterism-triplewhitestar.svg

  8. Lorem Ipsum

  9. ReBeL: Combining Deep Reinforcement Learning and Search for Imperfect-Information Games

  10. Assessing Game Balance with AlphaZero: Exploring Alternative Rule Sets in Chess

  11. A digital biomarker of diabetes from smartphone-based vascular signals

  12. Prediction of cardiovascular risk factors from retinal fundus photographs via deep learning

  13. Tasks, stability, architecture, and compute: Training more effective learned optimizers, and using them to train themselves

  14. Gradient Descent: The Ultimate Optimizer

  15. Reviewer #2 Doubted That Our Hyper-Hyperparameter Optimizer (Https://arxiv.org/abs/1909.13371) Would Work for Larger Examples. Of Course It Does, but Thanks for Asking. (Erik Meijer)

  16. X-LXMERT: Paint, Caption and Answer Questions with Multi-Modal Transformers

  17. Text-To-Image Generation: Text-To-Image Generation Is the Task of Generating an Image Conditioned on the Input Text. Try It for Yourself: 1. Choose a Caption

  18. StyleGAN network blending

  19. Toonify: Resolution Dependent GAN Interpolation for Controllable Image Synthesis Between Domains

  20. Optimal Peanut Butter and Banana Sandwiches

  21. DeepSpeed: Extreme-scale model training for everyone

  22. L2L: Training Large Neural Networks with Constant Memory using a New Execution Algorithm

  23. DeepSpeed Sparse Attention

  24. Generative Language Modeling for Automated Theorem Proving

  25. The Examples Are Indeed Extremely Simple on Purpose (Otherwise It’s Hard to Communicate Efficiently What’s Happening to Non-Metamath Experts). That Being Said, We’re Still Pretty Far Away from IMOs; but This Is Definitely a Goal for Us, and One We’re Actively Working Towards!

  26. MMLU: Measuring Massive Multitask Language Understanding

  27. 2020-hendrycks-figure1b-gpt3-qascaling.png

  28. DynamicEmbedding: Extending TensorFlow for Colossal-Scale Applications

  29. Why Tool AIs Want to Be Agent AIs

  30. A Time Leap Challenge for SAT Solving

  31. Computer Optimization: Your Computer Is Faster Than You Think

  32. Algorithmic Progress in Six Domains

  33. Efficient Transformers: A Survey

  34. Efficient Attention: Breaking The Quadratic Transformer Bottleneck

  35. A general dimension of genetic sharing across diverse cognitive traits inferred from molecular data

  36. Mapping genomic loci prioritises genes and implicates synaptic biology in schizophrenia

  37. Extended Data Figure 2: GWAS Progress over Time. The Relationship of GWAS Associations to Sample-Size Is Shown in This Plot With Selected SCZ GWAS Meta-Analyses of the past 11 Years. The X-Axis Shows Number of Cases. The Y-Axis Shows the Number of Independent Loci Discovered With at Least One Genome-Wide Statistically-Significant Index SNP in the Discovery Meta-Analysis (Eg. without Replication Data)…The Slope of ~4 Newly Discovered Loci per 1,000 Cases 2013–2019 Increased to a Slope of ~6 With the Latest Sample-Size Increase.

  38. Exome sequencing identifies rare coding variants in 10 genes which confer substantial risk for schizophrenia

  39. Figure 6: The Contributions of Ultra-Rare PTVs [Protein-Truncating Variants] to Schizophrenia Risk. A: Genetic Architecture of Schizophrenia. Statistically-Significant Genetic Associations for Schizophrenia from the Most Recent GWAS, CNV, and Sequencing Studies Are Displayed. The In-Sample Odds Ratio Is Plotted against the Minor Allele Frequency in the General Population. The Color of Each Dot Corresponds to the Source of the Association, and the Size of the Dot to the Odds Ratio. The Shaded Area Represented the LOESS-Smoothed Lines of the Upper and Lower Bounds of the Point Estimates…Because Schizophrenia As a Trait Is under Strong Selection38–40, We Expect That URVs of Large Effect to Be Frequently de Novo or of Very Recent Origin and Contribute to Risk in Only a Fraction of Diagnosed Patients.

  40. Genome-wide association study of over 40,000 bipolar disorder cases provides novel biological insights

  41. Genetic Fortune: Winning or Losing Education, Income, and Health

  42. GWAS of Over 427,000 Individuals Establishes GABAergic and Synaptic Molecular Pathways as Key for Cognitive Executive Functions

  43. Shared heritability of face and brain shape distinct from cognitive traits

  44. Rapid Evolution of Plastic-degrading Enzymes Prevalent in the Global Ocean

  45. The Przewalski’s Horse Project

  46. Human Embryo Gene Editing Gets a Road Map—Not a Green Light: After the 2018 ‘Crispr baby’ scandal, a global commission assessed the technology and set strict criteria for moving it toward clinical trials

  47. Heritable Human Genome Editing

  48. CRISPR-engineered human brown-like adipocytes prevent diet-induced obesity and ameliorate metabolic syndrome in mice

  49. CRISPR-enhanced human adipocyte ‘browning’ as cell therapy for metabolic disease

  50. Blueberry Earth

  51. America in Decay: The Sources of Political Dysfunction

  52. Constitutional Hardball

  53. The Empty Chamber: Just how broken is the Senate?

  54. ‘Dwarf Pride’ Was Hard Won. Will a Growth Drug Undermine It?: An experimental medication that increases height in children with the most common form of dwarfism has raised hope that it can help them lead easier lives. But some say the condition is not a problem in need of a cure.

  55. FDA Approves First Drug for Genetic Cause of Dwarfism

  56. Once-daily, subcutaneous vosoritide therapy in children with achondroplasia: a randomized, double-blind, phase 3, placebo-controlled, multicenter trial

  57. An expressed fgf4 retrogene is associated with breed-defining chondrodysplasia in domestic dogs.

  58. Laws of Xmas [Have you ever wondered what Xmas would be like if it were a Jewish Holiday?…]

  59. Self-Supervised Natural Image Reconstruction and Rich Semantic Classification from Brain Activity

  60. WBE & DRL: a Middle Way of imitation learning on brains

  61. Catching Cowpox: The Early Spread of Smallpox Vaccination, 1798–1810

  62. Creativity and Intelligence: An Investigation of the Threshold Hypothesis

  63. Water on Mars: discovery of three buried lakes intrigues scientists: Researchers have detected a group of lakes hidden under the red planet’s icy surface.

  64. M51-ULS-1b: The First Candidate for a Planet in an External Galaxy

  65. Unraveling the JPEG: JPEG images are everywhere in our digital lives, but behind the veil of familiarity lie algorithms that remove details that are imperceptible to the human eye. This produces the highest visual quality with the smallest file size—but what does that look like? Let’s see what our eyes can’t see!

  66. The Recursive Universe

  67. OTCA metapixel

  68. The First Roman Fonts

  69. Salary Negotiation: Make More Money, Be More Valued

  70. A Deep Dive into K-pop

  71. Wikipedia Matters