2019 News

Gwern

2019 News

Annual summary of 2019 Gwern.net newsletters, selecting my best writings, the best 2019 links by topic, and the best books/movies/anime I saw in 2019, with some general discussion of the year and the 2010s, and an intellectual autobiography of the past decade.

in progress certainty: log importance: 0 bibliography

Writings
- Overview
Media
- Links
- Books
- TV/movies

This is the 2019 summary edition of the Gwern.net newsletter (archives), summarizing the best of the monthly 2019 newsletters:

January
February
March
April
May
June
July
August
September
October
November
December
end of year summary (here)

Previous annual newsletters: 2018, 2017, 2016, 2015.

Writings

2019 went well, with much interesting news and several stimulating trips. My 2019 writings included:

I’m particularly proud of the technical improvements to the Gwern.net site design this year: along with a host of minor typographic improvements & performance optimizations, Inflation.hs enables automatic updates of currencies (a feature I’ve long felt would make documents far less misleading), the link annotations/popups (popups.js) are a major usability enhancement few sites have, sidenotes.js eliminates the frustration of footnotes by providing sidenotes, collapsible sections help tame long writings by avoiding the need for hiding code or relegating material to appendices, and link icons & dropcaps & epigraphs are just pretty. While changes are never unanimously received, we have received many compliments on the overall design, and are pleased with it.

Site traffic (more detailed breakdown) was again up as compared with the year before: 2019 saw 1,361,195 pageviews by 671,774 unique visitors (lifetime totals: 7,988,362 pageviews by 3,808,776 users). I benefited primarily from TWDNE, although the numbers are somewhat inflated by hosting a number of popular archived pages from DeepDotWeb/OKCupid/Rotten.com, which I put Google Analytics on to keep track of referrals.

Overview

2019 was a fun year.

AI: 2019 was a great year for hobbyists and fun generative projects like mine, thanks to spinoffs and especially pretrained models. How much more boring it would have been without the GPT-2 or StyleGAN models! (There was irritatingly little meaningful news about self-driving cars.) More seriously, the theme of 2019 was scaling. Whether GPT-2 or StyleGAN 1+2, or the scaling papers, or AlphaStar, or MuZero, 2019 demonstrated the blessings of scale in scaling up models, compute, data, and tasks; it is no accident that the most extensively discussed editorial on DL/DRL was Rich Sutton’s “The Bitter Lesson”. For all the critics’ carping and goalpost-moving, scaling is working, especially as we go far past the regimes where they assured us years ago that mere size and compute would break down and we would have to use more elegant and intelligent methods like Bayesian program synthesis. Instead, every year it looks increasingly like the strong connectionist thesis is correct: much like humans & evolution, AGI can be reached by training an extremely large number of relatively simple units end-to-end for a long time on a wide variety of multimodal tasks, and it will recursively self-improve meta-learning efficient internal structures & algorithms optimal for the real world which learns how to generalize, reason, self-modify with internal learned reward proxies & optimization algorithms, and do zero/few-shot learning bootstrapped purely from the ultimate reward signals—without requiring extensive hand-engineering, hardwired specialized modules designed to support symbolic reasoning, completely new paradigms of computing hardware etc. (eg. Clune 2019). Self-driving cars remain a bitter mystery (although it was nice to see in 2019 Elon Musk & Tesla snatch victory from the jaws of snatching-defeat-from-the-jaws-of-victory).

2019 for genetics saw more progress on genetic-engineering topics than GWASes; the GWASes that did come out were largely confirmatory—no one really needed more SES GWASes from Hill et al, or confirmation that the IQ GWASes work and that brain size is in fact causal for intelligence, and while the recovery of full height/BMI trait heritability from WGS is a strong endorsement of the long-term value of WGS, the transition from limited SNP data to WGS is foreordained (especially since WGS costs appear to finally be dropping again after their long stagnation). Even embryo selection saw greater mainstream acceptance, with a paper in Cell concluding (for the crudest possible simple embryo selection methods) that, fortunately, the glass was half-empty and need not be feared overmuch. (I am pleased to see that as of 2019, human geneticists have managed to reinvent Fisher 1918/Lush 1943’s breeder’s equation; with any luck, in a few years they may progress as far as reinventing Hazel & Lush 1943.) More interesting were the notable events along all axis of post-simple-embryo-selection strategies: Genomic Prediction claimed to have done the first embryo selection on multiple PGSes, genome synthesis saw E. coli achieved, multiple promising post-CRISPR or mass CRISPR editing methods were announced, gene drive progressed to mammals, gametogenesis saw progress (including at least two human fertility startups I know of), serious proposals for human germline CRISPR editing are being made by a Russian (among others), and while He Jiankui was imprisoned by a secret court there otherwise do not appear to have been serious repercussions such as reports of the 3 CRISPR babies being harmed or an ‘indefinite moratorium’ (ie. ban). Thus, we saw good progress towards the enabling technologies for massive embryo selection (breaking the egg bottleneck by allowing generation of hundreds or thousands of embryos and thus multiple-SD gains from selection), IES (Iterated Embryo Selection), massive embryo editing (CRISPR or derivatives), and genome synthesis.

VR’s 2019 launch of Oculus Quest proved quite successful, selling out occasionally well after launch, and appears to appeal to normal people, with even hardcore VR fans acknowledging how much they appreciate the convenience of a single integrated unit. Unfortunately… it is not successful enough. There is no VR wave. Selling out may have as much to do with Facebook not investing too much into manufacturing. Worse, there is still no killer app beyond Beat Saber. The hardware is adequate to the job, the price is nugatory, the experience unparalleled, but there is no stampede into VR. So it seems VR is doomed to the long slow multi-decade adoption slog like that of PCs: it’s too new, too different, and we’re still not sure what to do with it. One day, it would not be surprising if most people have a VR headset, but that day is a long way away.

Bitcoin: little of note. Darknet markets proved unusually interesting: Dream Market, the longest-lived DNM ever, finally expired; Reddit betrayed its users by wholesale purging of subreddits, including /r/DarkNetMarkets, causing me a great deal of grief; and most shockingly, DeepDotWeb was raided by the FBI over affiliate commissions it received from DNMs (apparently into the tens of millions of dollars—you’d’ve thought they’d taken down those hideous ads all over DDW if the affiliate links were so profitable…)