November 2021 News
November 2021 Gwern.net newsletter with links on TODO
November 2021’s Gwern.net newsletter is now out; previous, October 2021 (archives). This is a collation of links and summary of major changes, overlapping with my Changelog; brought to you by my donors on Patreon.
Writings
Links
AI
-
“2021 AI Alignment Literature Review and Charity Comparison”, Larks
-
“Player of Games”, et al 2021 (PoG generalizes AlphaZero to imperfect information games like poker or Scotland Yard, lifting one of the biggest limits on the domains AlphaZero can be used in; an agent which successfully combined MuZero+Perceiver-IO+PoG would be of remarkable generality)
-
“Optimal Policies Tend to Seek Power”, et al 2021 (formally proving one of the convergent goals of AIs in general: empowerment)
-
“A Neural Network Solves and Generates Mathematics Problems by Program Synthesis: Calculus, Differential Equations, Linear Algebra, and More”, et al 2021 ( Codex)
-
“What is the point of computers? A question for pure mathematicians”, 2021 (the rising tide of theorem provers+libraries+ML?)
-
“QNRs: Toward Language for Intelligent Machines”, 2021 (embeddings are a form of writing & knowledge without precedent in human history; compare 1992 on regular writing—perhaps words are just a poor brain’s thought-vectors…)
-
Image Generation:
-
“Muse: Text-To-Image Generation via Masked Generative Transformers”, et al 2023
-
“eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers”, et al 2022
-
“Parti: Pathways Autoregressive Text-to-Image model”, Google (20b-param DALL-E-1-esque)
-
“Imagen: Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding”, et al 2022 ( site; better than DALL·E 2, can be combined with Parti); Imagen 2 (higher-quality, hands, text)
-
“GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models”, et al 2021/ DALL·E 2 (OpenAI’s DALL·E successors: censored GLIDE is a 5b-parameter diffusion model + noise-aware CLIP; if you thought DALL·E was good, these are better; eg. it can draw “a horse riding on back of an astronaut”, with extended exercises including 1111101000 Robots, “A Bunnies Tale”, Behind The Throne, Book of Imaginary Beings, “We Asked 100 Humans”; The Bees, Toot, imitation game, murals; many more samples at /r/DALL·E 2; problems are fixable IMO)
-
“Make-A-Scene: Scene-Based Text-to-Image Generation with Human Priors”, et al 2022 ( 5 years of progress in AI: 2022 vs 2017 (8 years))
-
“ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation”, et al 2021
-
“CogView2: Faster and Better Text-to-Image Generation via Hierarchical Transformers”, et al 2022
-
“KNN-Diffusion: Image Generation via Large-Scale Retrieval”, et al 2022 (high-quality cartoon stickers by retrieval + diffusion model)
-
“
compviz
: High-Resolution Image Synthesis with Latent Diffusion Models” (best available FLOSS image-generation model as of March 2022?)
-
Genetics
Everything Is Heritable:
Engineering:
-
“Biologic and Clinical Efficacy of LentiGlobin for Sickle Cell Disease”, et al 2021 ( sickle cell disease has been cured)
-
“Evolution of sociability by artificial selection”, et al 2021 (increasing/decreasing complex traits in fruit flies)
Statistics/Meta-Science
-
“The Science of Visual Data Communication: What Works”, et al 2021; “Fooled by beautiful data: Visualization aesthetics bias trust in science, news, and social media”, 2022 (recall et al 2014 )
-
“More treatment but no less depression: The treatment-prevalence paradox”, et al 2021
-
“Jeep problem” (older desert-related variants on the rocket equation)
Politics/Religion
Psychology/Biology
-
“Testing the structure of human cognitive ability using evidence obtained from the impact of brain lesions over abilities”, 2021 (no causal effects of any brain lesions on g—bifactor models best after all?)
-
“On the Working Memory of Humans and Great Apes: Strikingly Similar or Remarkably Different?”, et al 2021 (humans as scaled-up primate brain)
-
“A connectome of the Drosophila central complex reveals network motifs suitable for flexible navigation and context-dependent action selection”, et al 2021 ( media)
-
“Volitional Control of Individual Neurons in the Human Brain”, et al 2020
-
Microdosing still doesn’t work:
-
“Microdosing psychedelics and its effect on creativity: Lessons learned from 3 double-blind placebo controlled longitudinal trials”, et al 2021 (no effects, 2⁄3 domains; placebo effects even stronger in psychedelics users)
-
“Effects of psilocybin microdosing on awe and aesthetic experiences: a preregistered field and lab-based study”, Van et al 2021
-
“Repeated low doses of LSD in healthy adults: A placebo-controlled, dose-response study”, de et al 2022 (some stimulation at the macrodose)
-
-
“Osmosis is not driven by water dilution”, Kramer & Myers 201311ya (is my whole life a lie?)
Technology
-
“A deep dive into an NSO zero-click iMessage exploit: Remote Code Execution” (an incredible weird machine: exploiting a memory error by inserting 70,000 JBIG2 image-pixel boolean operations, defining a programmable ‘VM’ which then runs a ‘program’ to attack the user)
-
“The Random Walk of Cars and Their Collision Probabilities with Planets”, et al 2018
-
“Entanglement between superconducting qubits and a tardigrade”, et al 2021 ( tardigrades apparently lack the necessary soul to count as Copenhagen observers)
Economics
-
“99 Good News Stories You Probably Didn’t Hear About in 2021”
-
““What’s Wrong With The Way I Talk?” The Effect Of Sound Motion Pictures On Actor Careers”, 2019
-
“The signal quality of earnings announcements: evidence from an informed trading cartel”, 2020 (efficient markets are hard to beat, even when you read the news in advance)
-
“What Makes a Good Image? Airbnb Demand Analytics Leveraging Interpretable Image Features”, et al 2017
Philosophy
Miscellaneous
-
Yoshie Shiratori (inspiration for escape artist in Golden Kamuy)
Film/TV
Animated:
-
Post-apocalypse iyashikei: 2 cute young girls flee their USSR-esque home village, which has collapsed in infighting, to travel a desolate megalopolis, investigating ruins and looking for their next food-ration/fuel-resupply, with nothing much in mind. Girls’ Last Tour (GLT) blends contemporary architecture like the Vietname Veterans Memorial with BLAME!’s posthuman architecture & the Miyazaki critique of technology used for war (with a homage to Giant God Warriors from Nausicaa & Castle in the Sky) and other odds and ends (eg. Kettenkrads for the military otaku1).
GLT is not long nor does it move fast or invest overmuch in animation; it exists to convey a specific mood: mono no aware, by way of sitting in a comfy onsen after cutting your wrists. The worldbuilding is initially nonsensical (how could there be no one anywhere? where’s all the junk from billions of people? who built all these Big Dumb Objects when the girls have such crude equipment? how could such foolish naive girls survive more than a few weeks before their Kettenkrad breaks down permanently?), but slowly justified: this is not the apocalypse, nor even the post-apocalypse, but the post-post-apocalypse in AD 3230—humanity has risen, fallen, risen again, and fallen. For the last time. The girls aren’t going to find a budding civilization, having learned from the errors of the past. This is it, the true apocalypse.
The core zombie apocalypse/survivalist fantasy is imagining that there will be a bright line anywhere where you go ‘ah, now is when I go into the bunker and close the door’—that there will be a fire alarm, a moment where Life tells you that there is a before/after dividing line, that before you lived in Civilization and now you live in Barbarism, that now you wake up from your sleepy life, and become the hero of your narrative as you Survive And Rebuild Civilization.
But most disasters aren’t like that. Most disasters are depressingly mundane, a slow endless grind, like a Venezuela or Soviet Russia, where the bureaucracy and daily life ticks on, as things just slowly get worse and worse, with no clear dividing line, until one day a burglar breaks in and shoots you for your remaining rice or you are hauled away as a counter-revolutionary. This moment will only execute a death warrant that was signed long before. There was a moment where you could have prevented this, but that moment was unknowably long ago.
The real apocalypse is like the Roman aristocrat Symmachus: resplendent in a millennium of Greco-Roman learning and imperial wealth, pleading for tolerance from the rising Christians, noting the decay of the roads or the Roman evacuation from England, but sure that this is a passing crisis and the Roman Empire will revive as it always has—unaware that it is in irreversible decline to its final fall, that the lights are going out throughout Europe, that a millennium hence Englishmen will be incapable of Roman construction and will be writing elegaic poems about their ruins and telling myths of how giants built them.
It’s all over, and it is only a question of how one spends the remaining time. During the anime, they meet 3 other people: the Last Man, the Last Woman, and the Last Fish. The Last Man is on a futile quest to map the ever-shifting and (perhaps) growing city; he once traveled with a woman on his motorbike, but both are gone now. He guides them to an elevator to the next level of the megalopolis, but loses all his maps in an accident. He bequeaths them his camera (which they will use throughout), and sets off on foot alone into the north. The Last Woman occupies an airplane repair depot, filled with reference materials and a robotic food factory that is almost worn out, and has almost finished creating a plane of her own design just in time for the season, which she hopes to fly to the megalopolis on the horizon, which may be inhabited; the girls help her in exchange for repairs & food, and she takes off successfully—only for the wings to snap, sending her parachuting to the lowest city level, where she is never seen again. Finally, the Last Fish is the last surviving fish (the girls having previously enjoyed the only other one), in the car of a small semi-intelligent caretaker robot in what used to be a large fish-farming facility; the caretaker robot discusses the possibility of evolution operating in machines as well as biological organisms, but then the maintenance robot begins rebuilding the sector, dooming the fish. At the caretaker robot’s request, they destroy it with explosives. The facility is damaged permanently (the caretaker robot possesses only minor repair functions), but the fish survives.
The girls resolve to travel to the top of the city (perhaps there are people there), and this, we learn at the end, is as futile as the rest, because they are the last humans in the world. The anime cuts off at a big reveal focusing on the “cats” (artificial lifeforms engineered to clean up all the weapons & waste, and turn off the lights), but the manga goes on, grinding down the girls as they travel to the top, sacrificing their old friend the Kettenkrad, all their food and supplies and the travel journal they’d kept, making it to the top—only to discover it is bare concrete, nothing but a stone block & snow.
This is the capstone to the depiction of futility for the previous encounters: the Last Man’s mission is futile, as there is no one to share the maps with and they would quickly date; lacking transportation and heading north, he surely perishes soon. The Last Woman’s mission is likewise futile, as we learn all cities are empty; stranded far from home in the most picked-over level, she would not last long either, not that there is any food left at her home anyway. The Last Fish is a short-lived dead end, unable to reproduce; with the fish-farm facility’s repair robot destroyed, the fish-farm will soon break down, taking the fish & caretaker robot with it. The robots may be capable of limited evolution, but it is clearly too little too late, as they all are breaking down along with the megalopolis and failing to replace or repair themselves. And the girls’ mission was equally futile, with death perhaps not the ending of the story, but not far either. (This is required by the parallels, but one understands why a 1-cours anime would not try to rush to the manga ending.)
The pathos of the girls is their sincere enjoyment of what they can where they find it. A few bottles of beer here while dancing tipsily in the moonlight, a makeshift onsen using steam-water leaking from a pipe there, a long quiet sit bathed in the sound of rain on a roof over here in episode 5 (in an outstanding scene playing with acoustics, surprisingly from the manga), and puzzling over such novelties as MRE “chocolate”… Such small things are all they have. At one point, they sit in an empty room and fantasize about what it would be like to have a ‘home’, with bright lights, and where they could put the ‘pantry’ or ‘bunk beds’ (though they are not entirely sure what these things are, as only one of them reads, and the few books don’t have photos). Almost all of human civilization is denied them.
At one point, the girls ride through a vast plain of drawers in tall black slabs, containing small objects, like a radio (which they take). Eventually they realize: it’s a columbarium, or perhaps cenotaph would be more accurate, for the dead. Each object is a memory, deeply meaningful to the deceased. But there is no one left who remembers, or to grieve. At another, they discover a vast glass lake filled with lotuses and fish, under the benevolent gaze of a triune god; it had been constructed with the wish that those who sorrow could take comfort in a depiction of Heaven.
GLT clarifies why human extinction and death are bad, actually.
{{citation needed}}
It is true that you did not exist for billions of years, and nonexistence returns you to that; if you are not upset that you didn’t exist for those eons, why be upset about missing eons on the other end, so to speak? But the situations are not the same. Humans can’t live for billions of years (much less in the hot plasma of the early universe), and so that was never a hypothetical choice. (If somehow you could have existed for all that time, if Adam & Eve could’ve had you but procrastinated shamelessly and only got around to having you a few decades ago, then you certainly would be entitled to be furious—you missed out on most of human history! All of those people, events, times, places, and the pleasures of ordinary life, all denied to you by their laziness.) Instead, the history of the universe is a history of the necessary processes unfolding to someday yield you. The plasma had to cool, stars had to be born, live, and die in supernovae spewing heavy elements which could someday clump together in planets, which somewhere could cool, and have water, and nurse the spark of life which could begin the long ascent from the sea… All of this took time, and was necessary to create you. There is no cause for complaint that you were born as late in the universe as you were. All things considered, you may be early (at least according to anthropic arguments), indeed, perhaps too early, if you arrive too soon before humanity manages to cure aging. But even then, at least you have the consolation of knowing your sufferings will be justified, in however small measure, by contributing to such a future for other people.The difference between a story where things start well and end badly, and one where things start badly and then end well, even if the amount of on-screen goodness/badness is identical for each, is that the latter is one of progress in a world with a future where people overcome their problems and the bad starting point we are dealt in life, while the former is a story of failure and decay in a universe where we will never emerge again and there is no cosmic producer greenlighting sequels. Once we are gone, that’s that, it’s over.
The future holds up half the sky.
-
Hozuki no Reitetsu 2 (a disappointment. The art still looks lively and nicely mixes Buddhist hell tropes with sumi-e art style, but somehow, the writing and plotting is extremely bland compared to season 1; almost none of the new characters or parts of hell left an impression or were used well, and Hozuki himself is rendered boring. I am left filled with terror: was season 1 not nearly as good as I remember it as being…? Like whether the hells exist, this is a question best put off answering as long as possible.)
-
Tsukumizu mentions that it was inspired by Saving Private Ryan.↩︎