--- title: October 2019 News description: October 2019 Gwern.net newsletter with links on DL scaling, SDAM, and public goods; 1 book and 2 opera reviews. created: 2019-09-30 status: finished previous: /newsletter/2019/09 next: /newsletter/2019/11 confidence: log cssExtension: dropcaps-de-zs backlink: False ... October 2019's [Gwern.net](/newsletter/2019/10 "'October 2019 News', Branwen 2019") [newsletter](https://gwern.substack.com/ "'Gwern.net newsletter (Substack subscription page)', Branwen 2013") is now out; previous, [September 2019](/newsletter/2019/09 "'September 2019 News', Branwen 2019") ([archives](/doc/newsletter/index)). This is a collation of links and summary of major changes, overlapping with my [Changelog](/changelog); brought to you by my donors on [Patreon](https://www.patreon.com/gwern). # Writings - Nothing completed # Media ## Links **Genetics**: - [Everything Is Heritable]{.smallcaps}: - ["Germline burden of rare damaging variants negatively affects human healthspan and lifespan"](https://www.biorxiv.org/content/10.1101/802082.full), Shindyapina et al 2019 - ["Associations of autozygosity with a broad range of human phenotypes"](https://www.nature.com/articles/s41467-019-12283-6), Clark et al 2019 - [Engineering]{.smallcaps}: - ["Technological challenges and milestones for writing genomes: Synthetic genomics requires improved technologies"](/doc/genetics/editing/2019-ostrov.pdf), Ostrov et al 2019 **AI**: - [Matters Of Scale]{.smallcaps}: - ["Grandmaster level in StarCraft II using multi-agent reinforcement learning"](/doc/reinforcement-learning/model-free/alphastar/2019-vinyals.pdf#deepmind), Vinyals et al 2019 ([blog](https://deepmind.google/discover/blog/alphastar-grandmaster-level-in-starcraft-ii-using-multi-agent-reinforcement-learning/); [discussion](https://www.reddit.com/r/reinforcementlearning/comments/dpbfwx/alphastar_grandmaster_level_in_starcraft_ii_using/)) - ["Solving Rubik's Cube With A Robot Hand"](https://arxiv.org/abs/1910.07113#openai "'Solving Rubik’s Cube with a Robot Hand', OpenAI et al 2019"), Akkaya et al 2019 ([blog](https://openai.com/research/solving-rubiks-cube "We’ve trained a pair of neural networks to solve the Rubik’s Cube with a human-like robot hand. The neural networks are trained entirely in simulation, using the same reinforcement learning code as OpenAI Five paired with a new technique called Automatic Domain Randomization (ADR). The system can handle situations it never saw during training, such as being prodded by a stuffed giraffe. This shows that reinforcement learning isn’t just a tool for virtual tasks, but can solve physical-world problems requiring unprecedented dexterity."); [video](https://www.youtube.com/watch?v=QyJGXc9WeNo&list=PLOXw6I10VTv9HODt7TFEL72K3Q6C4itG6&index=3): [Dactyl](https://openai.com/research/learning-dexterity) followup with improved curriculum-learning of machine-generated domain randomizations in a simulation; the diversity of simulations leads to emergent meta-learning, and zero-shot sim2real transfer & robustness to untrained interference. An example of harder tasks leading to better learning (see previously [Clune 2019](https://arxiv.org/abs/1905.10985#uber "AI-GAs: AI-generating algorithms, an alternate paradigm for producing general artificial intelligence")), because neural nets are lazy!) - ["T5: Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"](https://arxiv.org/abs/1910.10683#google "‘Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer’, Raffel et al 2019"), Raffel et al 2019 ([summary](https://threadreaderapp.com/thread/1187161460033458177.html); [finetuning Colab](https://colab.research.google.com/drive/1-ROO7L09EupLFLQM-TWgDHa5-FIOdLLh); [SuperGLUE](https://arxiv.org/abs/1905.00537 "'SuperGLUE: A Stickier Benchmark for General-Purpose Language Understanding Systems', Wang et al 2019")---introduced this year to challenge NN LMs---already almost solved by 11b-parameter Transformer trained on <750 GB text) - ["Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges"](https://arxiv.org/abs/1907.05019#google), Arivazhagan et al 2019; ["Simple, Scalable Adaptation for Neural Machine Translation"](https://arxiv.org/abs/1909.08478#google), Bapna et al 2019 ([blog](https://blog.research.google/2019/10/exploring-massively-multilingual.html "Exploring Massively Multilingual, Massive Neural Machine Translation")) - ["Billion-scale semi-supervised learning for image classification"](https://arxiv.org/abs/1905.00546#facebook), Yalniz et al 2019 ([blog](https://ai.facebook.com/blog/billion-scale-semi-supervised-learning/ "Billion-scale semi-supervised learning for state-of-the-art image and video classification")) - [Artbreeder: now with the anime portrait StyleGAN](https://www.artbreeder.com/ "'Artbreeder', Simon 2019") (upgrade over [TWDNE](/twdne "'This Waifu Does Not Exist', Branwen 2019"): random generation, exploration, image attribute editing, saving to a gallery, and crossbreeding portraits) - ["Animating gAnime with StyleGAN: Part 1---Introducing a tool for interacting with generative models"](https://towardsdatascience.com/animating-ganime-with-stylegan-part-1-4cf764578e)/[part 2](https://towardsdatascience.com/animating-ganime-with-stylegan-the-tool-c5a2c31379d "Animating gAnime with StyleGAN: The Tool - In-depth tutorial for an open-source GAN research tool - I go into more detail about the tool and how it can be used to create animations. I also share a compiled version of the tool and a trained model to interact with, as the process to do that with just the source code is complicated at the moment."), Nolan Kent ([re-implementing StyleGAN](https://github.com/nolan-dev/stylegan_reimplementation) for improved character generation with rectangular convolutions & feature map visualizations, and [interactive manipulation](https://github.com/nolan-dev/GANInterface)) **Statistics/Meta-Science**: - [How often do researchers not read the papers they cite?](/leprechaun#citogenesis-how-often-do-researchers-not-read-the-papers-they-cite "'Leprechaun Hunting & Citogenesis § Citogenesis: How Often Do Researchers Not Read The Papers They Cite?', Branwen 2014") **Politics/religion**: - ["The small effects of political advertising are small regardless of context, message, sender, or receiver: Evidence from 59 real-time randomized experiments"](https://www.science.org/doi/10.1126/sciadv.abc4046), Coppock et al 2020 - ["The Corporate Governance of Benedictine Abbeys: What can Stock Corporations Learn from Monasteries?"](/doc/economics/2010-rost.pdf "'The corporate governance of Benedictine abbeys', Rost et al 2010"), Rost et al 2010 - ["How Airbnb Is Silently Changing Himalayan Villages"](https://veridici.com/how-airbnb-is-silently-changing-himalayan-villages/), Shanu Athiparambath ("All fixed, fast-frozen relations, with their train of ancient and venerable prejudices and opinions, are swept away, all new-formed ones become antiquated before they can ossify. All that is solid melts into air...") **Psychology/biology**: - ["In A Perpetual Present: The Strange Case of the Woman Who Can’t Remember Her Past—Or Imagine Her Future"](https://www.wired.com/2016/04/susie-mckinnon-autobiographical-memory-sdam/) (on having no autobiographical memory but not being amnesiac either: "Severely Deficient Autobiographical Memory" (SDAM); an extreme form of [aphantasia](https://en.wikipedia.org/wiki/Aphantasia)? See also: [/r/SDAM](https://www.reddit.com/r/SDAM/); [Palombo et al 2015](https://www.sciencedirect.com/science/article/pii/S002839321500158X "'Severely deficient autobiographical memory (SDAM) in healthy adults: A new mnemonic syndrome', Palombo et al 2015"); [Palombo et al 2018](https://files.osf.io/v1/resources/qr4f9/providers/osfstorage/5ae7c0b591de99000c1a4f19?format=pdf&action=download&direct&version=1 "Individual differences in autobiographical memory"); [Zeman et al 2015](https://pure.hw.ac.uk/ws/portalfiles/portal/8873489/Lives_without_imagery_1.pdf "Lives without imagery - Congenital aphantasia"); [Wilkins & Clayton 2019](/doc/psychology/neuroscience/2019-wilkins.pdf "Reflections on the spoon test")) - [On why Henry Darger, an elderly, solitary dishwasher, wrote and illustrated a 15,000+ page unpublished fantasy novel](https://nostalgebraist.livejournal.com/68532.html "'About Henry Darger', Nostalgebraist 2011") (on [Henry Darger](https://en.wikipedia.org/wiki/Henry_Darger); [Geschwind syndrome](https://en.wikipedia.org/wiki/Geschwind_syndrome)?) - ["Chess Masters' Hypothesis Testing"](/doc/psychology/chess/2004-cowley.pdf), Cowley & Bryne 2004 **Technology**: - ["Roads and Bridges: The Unseen Labor Behind Our Digital Infrastructure"](https://www.fordfoundation.org/wp-content/uploads/2016/07/roads-and-bridges-the-unseen-labor-behind-our-digital-infrastructure.pdf), Nadia Eghbal 2016 ([background on report](https://nadiaeghbal.com/phd "Reimagining the PhD"); the still-unsolved funding problem of open source software: we all use it, it's incredibly valuable, but no one is paying for it, and volunteerism & [commoditize-your-complement](/complement "'Laws of Tech: Commoditize Your Complement', Branwen 2018") dynamics only go so far. Do we need new funding models? Perhaps something like a [Public Lending Right](https://en.wikipedia.org/wiki/Public_Lending_Right) general tax with allocation based on [quadratic funding](/doc/economics/mechanism-design/quadratic-voting/2018-buterin.pdf "'Liberal Radicalism: A Flexible Design For Philanthropic Matching Funds', Buterin et al 2018") like [Gitcoin's](https://www.gitcoin.co/blog/experiments-with-liberal-radicalism "Experiments With Liberal Radicalism: A crowdfund matching mechanism for public goods, like open source"){#singh-1} [CLR](https://www.gitcoin.co/blog/gitcoin-grants-clr-matching "Gitcoin Grants: CLR Matching - Matching contributions with up to $25,000 in funding, in ETH"){#singh-2} [experiment](https://www.gitcoin.co/blog/radical-results-gitcoins-25k-match "Radical Results: Gitcoin’s $25K Match - Results and lessons learned from our first $25K in matching"){#singh-3}?) - ["How can we develop transformative tools for thought?"](https://numinous.productions/ttft/ "'How Can We Develop Transformative Tools For Thought?', Matuschak & Nielsen 2019"), [Andy Matuschak](https://andymatuschak.org/) & [Michael Nielsen](https://michaelnielsen.org/) 2019 (preliminary report on [_Quantum Country_'s](https://quantum.country/qcvc) integration of [SRS](/spaced-repetition "'Spaced Repetition for Efficient Learning', Branwen 2009") into [QC](https://en.wikipedia.org/wiki/Quantum_computing) physics tutorial as new education approach; [HN](https://news.ycombinator.com/item?id=21152542)) - [Yahoo is deleting the Yahoo Groups email groups & newsletters 2019-12-14 (>2.1b emails)](https://help.yahoo.com/kb/groups/SLN31010.html) ([ArchiveTeam](https://wiki.archiveteam.org/index.php?title=Yahoo!_Groups); [HN](https://news.ycombinator.com/item?id=21269614); [archiving example](https://www.lesswrong.com/posts/YR4PsLavhd5Dx5L6v/archiving-yahoo-groups)) - ["The (Mostly) True Story of Helvetica and the New York City Subway"](https://www.aiga.org/the-mostly-true-story-of-helvetica-and-the-new-york-city-subway) (the colored stripes of the MTA for regional transport derive from a printing shop mistake) - ["The Chudnovsky Brothers and the Mountains of Pi"](https://www.newyorker.com/magazine/1992/03/02/the-mountains-of-pi) (on the [Chudnovsky brothers](!W)' record calculations of π by building a [Beowulf cluster](https://en.wikipedia.org/wiki/Beowulf_cluster) in their bedroom to run their parallelizable [Chudnovsky algorithm](https://en.wikipedia.org/wiki/Chudnovsky_algorithm); see also [their work on the](https://www.newyorker.com/magazine/2005/04/11/capturing-the-unicorn "Capturing the Unicorn: How two mathematicians came to the aid of the Met") [Unicorn tapestries](https://en.wikipedia.org/wiki/The_Hunt_of_the_Unicorn)) - ["Getting started with security keys: How to stay safe online and prevent phishing with FIDO2, WebAuthn and security keys"](https://paulstamatiou.com/getting-started-with-security-keys/), Paul Stamatiou (now that Firefox supports [WebAuthn](!W) and [simjacking](https://en.wikipedia.org/wiki/SIM_swap_scam) is epidemic, it's time to get serious about *real* [2FA](https://en.wikipedia.org/wiki/Multi-factor_authentication)) - ["Evaluating the Design of the R Language: Objects and Functions for Data Analysis"](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.694.7528&rep=rep1&type=pdf), Morandat et al 2012 (parsing CRAN to see what from the strange set of R features is used in the real world---not [laziness](https://en.wikipedia.org/wiki/Lazy_evaluation) or its weirdo context-dependent [scoping](https://en.wikipedia.org/wiki/Scope_\(computer_science\)), turns out) - [Updated Internet Archive service: 'Save Page Now' lets you archive pages & external links and save to a personal bookmarks](https://web.archive.org/save) **Economics**: - ["End-To-End Arguments In System Design"](https://web.mit.edu/Saltzer/www/publications/endtoend/endtoend.pdf), Saltzer et al 1984; ["Observations on Errors, Corrections, & Trust of Dependent Systems"](https://perspectives.mvdirona.com/2012/02/observations-on-errors-corrections-trust-of-dependent-systems/), James Hamilton (the [end-to-end principle](/doc/cs/end-to-end-principle/index)) - ["Anthropological invariants in travel behavior"](https://core.ac.uk/download/pdf/33895541.pdf#page=6), Marchetti 1994 - ["Be Cautious with the Precautionary Principle: Evidence from Fukushima Daiichi Nuclear Accident"](https://docs.iza.org/dp12687.pdf), Neidell 2019 (the seen and the unseen) - ["When Delivery Is Free, Will Ownership Survive?"](https://perspicacity.xyz/2017/08/10/when-delivery-is-free-will-ownership-survive/), Seth Miller (see also [Posner & Weyl 2016](https://www.law.nyu.edu/sites/default/files/upload_documents/Property%20Monopoly.pdf "Property Is Another Name for Monopoly: Facilitating Efficient Bargaining with Partial Common Ownership of Spectrum, Corporations, and Land"), ["Status as a Service"](https://www.eugenewei.com/blog/2019/2/19/status-as-a-service), ["Welcome to the 'Airbnb for Everything' Age"](https://www.wired.com/story/airbnb-for-everything/ "In the sublet economy, you can turn anything into extra cash: your house, your car, your boat, or your backyard.")) - ["On the Resilience of the Dark Net Market Ecosystem to Law Enforcement Intervention"](https://discovery.ucl.ac.uk/id/eprint/10080409/8/Bradley_10080409_thesis.pdf), Bradley 2019 (uses seized SR2 server data; also includes best literature review of DNM research to date) - ["Optimal boarding method for airline passengers"](https://arxiv.org/abs/0802.0733), Steffen 2008 - ["Pricing niche products: Why sell a mechanical keyboard kit for \$1,668?"](/doc/psychology/collecting/2019-08-22-kevinlynagh-pricingnicheproducts.html) (running [Vickrey auctions](https://en.wikipedia.org/wiki/Vickrey_auction) for optimal presales & learning [keyboard](https://en.wikipedia.org/wiki/Keyboard_technology) collector demand curves; [HN discusses more recherché keyboards](https://news.ycombinator.com/item?id=21157404)) **Fiction**: - ["_Neon Genesis Evangelion_: Graphic designer Peiran Tan plumbs the typographic psyche of the celebrated anime franchise"](https://fontsinuse.com/uses/28760/neon-genesis-evangelion "'Neon Genesis Evangelion: Graphic designer Peiran Tan plumbs the typographic psyche of the celebrated anime franchise', Tan 2019") (a look into the signature typefaces of _Evangelion_: Matisse EB, mechanical compression, and [title cards](https://en.wikipedia.org/wiki/Intertitle)) - [_Roar_ (1981 film)](https://en.wikipedia.org/wiki/Roar_\(1981_film\)) ("_Roar_ used real lions during filming, resulting in at least 70 cast and crew being injured. A crew member was scalped, another had his throat bitten out, and much of the footage of attacks was used in the final cut of the film. The blood seen in the movie is real.") - [_The Beginning Was the End_, Maerth 1971](https://en.wikipedia.org/wiki/The_Beginning_Was_the_End) ("eating brains produces an aphrodisiac effect & this caused apes to become addicted, organizing brain hunts wherein the males of another tribe were eaten & females raped in a frenzy of brain-induced sex & violence...The book contains no references whatsoever, based alternately on alleged conversations with present-day cannibals, the eating of ape brain by the author and direct insight from deep meditation.") **Misc**: - [The Rotten Library archives](/doc/rotten.com/library/index.html) ([Rotten.com](https://en.wikipedia.org/wiki/Rotten.com)) - [_Four Organs_](https://en.wikipedia.org/wiki/Four_Organs) ## Books **Nonfiction**: - [_Envisioning Information_](https://www.edwardtufte.com/tufte/books_ei), [Edward Tufte](https://en.wikipedia.org/wiki/Edward_Tufte) (not a sustained presentation but rather a set of clippings of particularly nice visualizations Tufte has come across, roughly grouped by a theme, such as use of color, highlighting [examples like Byrne's _Euclid_](/doc/design/typography/rubrication/1990-tufte-envisioninginformation-ch5-byrneseuclid.pdf "_Envisioning Information_, Tufte 1990: chapter 5, 'Color and Information', pg83-86") or the [use of rubrication](/red "'Rubrication Design Examples', Branwen 2019")) - [_Web Typography_](https://book.webtypography.net/), Rutter 2017 ([review](/review/book#web-typography-rutter-2017 "‘Book Reviews § Web Typography, Rutter 2017’, Branwen 2013")) ## Film/TV **Live-action**: - [_Manon_ opera](https://en.wikipedia.org/wiki/Manon) ([review](/review/opera#manon "'Movie Reviews § Manon', Branwen 2014")) - [_Turandot_ opera](https://en.wikipedia.org/wiki/Turandot) ([review](/review/opera#turandot "'Movie Reviews § Turandot', Branwen 2014"))