Bibliography (130):

1. Gwern.net newsletter (Substack subscription page)

2. ​ December 2020 News

3. ​ ‘newsletter’ directory

4. ​ Changelog

5. Gwern Branwen Creating Essays on Gwern.net

6. ​ Danbooru2020 Is a Large-Scale Anime Image Database With 4.2m+ Images Annotated With 130m+ Tags; It Can Be Useful for Machine Learning Purposes such as Image Recognition and Generation.

   [Transclude the forward-link's context]

7. This Anime Does Not Exist.ai (TADNE)

8. ​ face#extended-stylegan2-danbooru2019-aydao

   [Transclude the forward-link's context]

9. ML Scaling subreddit

10. DALL·E 1: Creating Images from Text: We’ve trained a neural network called DALL·E that creates images from text captions for a wide range of concepts expressible in natural language

11. Generating Diverse High-Fidelity Images with VQ-VAE-2

12. CLIP: Connecting Text and Images: We’re introducing a neural network called CLIP which efficiently learns visual concepts from natural language supervision. CLIP can be applied to any visual classification benchmark by simply providing the names of the visual categories to be recognized, similar to the ‘zero-shot’ capabilities of GPT-2 and GPT-3

13. CLIP: Learning Transferable Visual Models From Natural Language Supervision

14. ​ scaling-hypothesis#blessings-of-scale

    [Transclude the forward-link's context]

15. Contrastive Representation Learning: A Framework and Review

16. Vision Transformer: An Image is Worth 16×16 Words: Transformers for Image Recognition at Scale

17. Large Scale GAN Training for High Fidelity Natural Image Synthesis

18. I Added #CLIP to My Image Labeling Tool and Have Now Full Text Search over My Various Collections. Here Are Potato

19. Deep Residual Learning for Image Recognition

20. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

21. GPT-3: Language Models are Few-Shot Learners

22. Long Short-Term Memory

23. ImageNet Classification with Deep Convolutional Neural Networks

24. Attention Is All You Need

25. Better Language Models and Their Implications

26. A Simple Framework for Contrastive Learning of Visual Representations

27. When will computer hardware match the human brain?

28. Meta Pseudo Labels

29. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

30. GShard: Scaling Giant Models with Conditional Computation and Automatic Sharding

31. Training data-efficient image transformers & distillation through attention

32. Bottleneck Transformers for Visual Recognition

33. Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

34. not-so-BigGAN: Generating High-Fidelity Images on Small Compute with Wavelet-based Super-Resolution

35. VQ-GAN: Taming Transformers for High-Resolution Image Synthesis

36. 2019-lecun-isscctalk-cake.png

37. ZeRO-Offload: Democratizing Billion-Scale Model Training

38. Prefix-Tuning: Optimizing Continuous Prompts for Generation

39. It’s Not Just Size That Matters: Small Language Models Are Also Few-Shot Learners

40. Scaling down Deep Learning

41. ​ Computer Optimization: Your Computer Is Faster Than You Think

42. NNUE: The neural network of the Stockfish chess engine

43. Transformers in Vision: A Survey

44. Organizational Update from OpenAI

45. The Elves Leave Middle Earth: Sodas Are No Longer Free

46. 2020 AI Alignment Literature Review and Charity Comparison

47. Grounded Language Learning Fast and Slow

48. DeBERTa: Decoding-enhanced BERT with Disentangled Attention

49. SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems

50. SuperGLUE Benchmark

51. Solving Mixed Integer Programs Using Neural Networks

52. Learning a Large Neighborhood Search Algorithm for Mixed Integer Programs

53. Towards Fully Automated Manga Translation

54. UPDeT: Universal Multi-agent Reinforcement Learning via Policy Decoupling with Transformers

55. A Framework for Efficient Robotic Manipulation

56. XMC-GAN: Cross-Modal Contrastive Learning for Text-to-Image Generation

57. Nurture might be nature: cautionary tales and proposed solutions

58. A genetic perspective on the association between exercise and mental health in the era of genome-wide association studies

59. Evidence for shared genetics between physical activity, sedentary behavior and adiposity-related traits

60. Antidepressant Response in Major Depressive Disorder: A Genome-wide Association Study

61. Estimating the effect-size of gene dosage on cognitive ability across the coding genome

62. GWAS of 3 molecular traits highlights core genes and pathways alongside a highly polygenic background

63. Genome-scale sequencing and analysis of human, wolf and bison DNA from 25,000 year-old sediment

64. Disentangling sex differences in the shared genetic architecture of post-traumatic stress disorder, traumatic experiences, and social support with body size and composition

65. Distinguishing genetic correlation from causation across 52 diseases and complex traits

66. African genetic diversity and adaptation inform a precision medicine agenda

67. The influence of evolutionary history on human health and disease

68. Local adaptation and archaic introgression shape global diversity at human structural variant loci

69. Genome scans of dog behavior implicate a gene network underlying psychopathology in mammals, including humans

70. Natural Selection in Contemporary Humans is Linked to Income and Substitution Effects

71. The diversity and function of sourdough starter microbiomes

72. In vivo base editing rescues Hutchinson-Gilford progeria syndrome in mice

73. From Genotype to Phenotype: polygenic prediction of complex human traits

74. The Quantum Field Theory on Which the Everyday World Supervenes

75. Torsion balance experiments: A low-energy frontier of particle physics

76. How accurate are citations of frequently cited papers in biomedical literature?

77. Energy-Efficient Algorithms

78. The Optimizer’s Curse: Skepticism and Postdecision Surprise in Decision Analysis

79. ​ Regression To The Mean Fallacies

80. The Mechanisms of Cult Production: An Overview

81. ​ Abandoned Footnotes

82. When Prophecy Fails and Faith Persists: A Theoretical Overview

83. Why We Fight Over Fiction

84. Still Alive

85. The Temporal Dynamics of Opportunity Costs: A Normative Account of Cognitive Fatigue and Boredom

86. A unified framework for association and prediction from vertex-wise grey-matter structure

87. ​ Variance Components Beyond Genetics

88. Sounds from seeing silent motion: Who hears them, and what looks loudest?

89. The sound of change: visually-induced auditory synaesthesia

90. A deafening flash! Visual interference of auditory signal detection

91. Predicting Mental Health From Followed Accounts on Twitter

92. ​ Everything Is Correlated

93. No evidence for general intelligence in a fish

94. Regionalism in Disney Animation: Pink Elephants and Dumbo

95. Microbiome connections with host metabolism and habitual diet from 1,098 deeply phenotyped individuals

96. Universal DNA methylation age across mammalian tissues

97. Whole-body senescent cell clearance alleviates age-related brain inflammation and cognitive impairment in mice

98. BENDR: using transformers and a contrastive self-supervised learning task to learn from massive amounts of EEG data

99. Comment by Peter Norvig on "Being good at programming competitions correlates negatively with being good on the job"

100. Baffles and Bastions: The Universal Features of Fortifications

101. Footnote 36: Redisturbed: In This Issue We’re Focusing on the Redisturbed Typeface For The New Decade [Redisturbed Is a Fresh Look at Our Original Disturbance Typeface from 1993. Looking Deeper at the Concept of a Unicase Alphabet and Designing It for Expanded Use Today. More Weights, Optical Sizes, Language Support and OpenType Features.]

102. Businesses Aim to Pull Greenhouse Gases From the Air. It’s a Gamble. A surge of corporate money could soon transform carbon removal from science fiction to reality. But there are risks: The very idea could offer industry an excuse to maintain dangerous habits.

103. Does Advertising Actually Work? (Part 1: TV)

104. Does Advertising Actually Work? (Part 2: Digital) (Ep. 441)

105. Digital Paywall Design: Implications for Content Demand and Subscriptions

106. ​ Banner Ads Considered Harmful

107. Who Gains and Who Loses from Credit Card Payments? Theory and Calibrations

108. Squeezing the bears: cornering risk and limits on arbitrage during the ‘British bicycle mania’, 1896–1898

109. A Long-Lost Space Age Satire about What It Means to Be a Jew from One of Science Fiction’s Greatest Humorists

110. St Martin’s Four Wishes

111. Wikipedia Bibliography:

     1. Reversible computing

     2. Stack (abstract data type)  :

        https://en.wikipedia.org/wiki/Stack_(abstract_data_type)

     3. Dynamic array  :

        https://en.wikipedia.org/wiki/Dynamic_array

     4. AVL tree  :

        https://en.wikipedia.org/wiki/AVL_tree

     5. Comparison sort  :

        https://en.wikipedia.org/wiki/Comparison_sort

     6. Graph traversal  :

        https://en.wikipedia.org/wiki/Graph_traversal

     7. All-Woman Supreme Court  :

        https://en.wikipedia.org/wiki/All-Woman_Supreme_Court

     8. Preregistration (science) § Registered reports  :

        https://en.wikipedia.org/wiki/Preregistration_(science)#Registered_reports

     9. Delirium tremens  :

        https://en.wikipedia.org/wiki/Delirium_tremens

     10. Parker–Hulme murder case  :

         https://en.wikipedia.org/wiki/Parker%E2%80%93Hulme_murder_case

     11. Slender Man stabbing  :

         https://en.wikipedia.org/wiki/Slender_Man_stabbing

     12. Paracosm

     13. Natural nuclear fission reactor  :

         https://en.wikipedia.org/wiki/Natural_nuclear_fission_reactor

     14. Corrupted Blood incident  :

         https://en.wikipedia.org/wiki/Corrupted_Blood_incident

     15. Debit card  :

         https://en.wikipedia.org/wiki/Debit_card

     16. William Tenn  :

         https://en.wikipedia.org/wiki/William_Tenn

     17. Fabliau  :

         https://en.wikipedia.org/wiki/Fabliau

     18. Anglo-Japanese style  :

         https://en.wikipedia.org/wiki/Anglo-Japanese_style

     19. Stalag Luft III  :

         https://en.wikipedia.org/wiki/Stalag_Luft_III

     20. Graham Island (Mediterranean Sea)  :

         https://en.wikipedia.org/wiki/Graham_Island_(Mediterranean_Sea)