A discussion of why I don’t intend to turn Gwern.net into a book, and how trying to write a book can harm writers.
Along with asking about doing podcasts, people sometimes ask me whether I am going to write a book or turn Gwern.net into a book. To their surprise, I have no such intention. Writing a book can be harmful; for many writers, writing a book is about the last thing they should do (unless they feel a book bursting out of them, much like a facehugger).
But how can I not want to write a book? And I get it: writing a book is the ultimate achievement for Western intellectuals—better than being arrested in a protest (because you don’t have to get sweaty), better than a PhD (because not so devalued), and better even than going to Harvard (because that mostly means you got lucky in admissions). It’s something I’ve definitely aspired to since I became a bookworm, to join the pantheon of authors shelved in my local library, to be able to hold my book in my hands, and carp about how ‘the publisher chose the cover’.
Bad Reasons For Books
Yet, that’s the problem: I am in love with the idea of having published a book, but not with publishing a specific book. I want to have published a book and have the identity of ‘author’1, in the same way one might want to have ‘learned to read Mandarin’ or ‘become a bodybuilder’, but not to actually sweat through memorizing (and then forgetting) endless arbitrary characters or hours in the gym (followed by gluttony as cruel as the starvation).
Costs of Books
It is easy for someone like a book agent or editor to suggest “hey, you should publish a book!”, since they aren’t the one writing it, and they have plenty on their plate. But what book, exactly, would I publish? “Gwern.net”? Leaving aside all the fun hypertext features, what would such a book be? It would have to be some sort of anthology, I suppose; but what a mongrel of an anthology it would be, if I look over my past few essays: a review of a children’s movie about unicorns, then the moral philosophy of self-consistent time travel, then there’s the detailed technical tutorial about generating fonts with 2023 generative AI models… (The anthology would not be ordered chronologically, safe to say.)
I don’t feel enthused about it; but to make a book I would not be ashamed of, I would have to spend many hours revising, rewriting, stripping out hypertext features and deciding what should stay, reformatting for some arcane publisher workflow like EPUB (or worse). This is true whether it’s a big professional publication or a small boutique style, like the LessWrong anthologies—which are nice indeed, but I recall Oliver Habryka ballparking the necessary total effort at in the hundreds of man-hours. (For already-written LW posts!) And while I was in the middle of that, if I wanted to write something else, well, now I would get to feel extra-guilty about procrastinating on The Book™. I definitely won’t have time or energy to follow up on weird anomalies or interesting new rabbit-holes, and even if my output appears unaffected by The Book™, I am in reality eating my intellectual seed-corn and gradually falling out of date, as I neglect the ordinary work of reading & discussion that may flower years later. (Authors who write books have a distinct tendency to ‘go stale’.)
I finish all this, and then what? Now it has to spend months working its way through the pipeline, still a distraction, and then—mirabile dictu!—it is done.
Uh oh—that’s just the start, isn’t it? Now I have to go out and shill for pre-orders (as we are endlessly assured by apologetic authors that pre-orders are about the only sales number that matters these days and oh god if you ever liked my writing please pre-order and don’t wait until afterwards just think of it as a donation if you have to but pre-order now). If you opt for a low-key launch, skipping the pre-publication publicity campaign, well, you’re still going to have to publicize it once it’s out (free copy to the first person to retweet this!), and deal with reviews (soliciting them, specifically—you should be so lucky anyone actually wrote a review, good or bad, unsolicited) etc.
And of course, the double-edged blade of the physical immutability of books as artifacts begins to cut both ways—now that your thoughts are committed to a book, you can never take them back or change them or tweak them. You are now in some respects frozen and reduced to a caricature, to a much greater degree watching your caricature self go around and about (“Borges and I”).
So, a book is a lot of work for a writer, even if it is mostly already-written writing, which crowds out new writing or exploration, and which tends to freeze them in place. But it gets worse.
A book commits you to a single task, one which will devour your time for years to come, cutting you off from readers and from opportunity; in the time that you are laboring over the book, which usually you can’t talk much about with readers or enjoy the feedback, you may be driving yourself into depression.
Case Studies
PhD attrition rates are well-known, and much of it happens after qualifications, while writing the thesis; those who pay attention to biographies of writers will recall many more instances attested, if only in those odd lacuna to which biographers dismiss with a line entire years in which nothing seems to happen but a discarded manuscript.
To these, I’ll add two case-studies that I often think about.
Wait But Why?
The long-form essayist/blogger Tim Urban of “Wait But Why” (WBW) made a name for himself in the mid-2010s by writing long enthusiastic essays, leavened with crude but effective comics, about various rabbit-hole topics like cryonics, the rise of deep learning, willpower/procrastination etc; at his peak he became a favored mouthpiece of Elon Musk. He was often compared to Scott Alexander (then blogging at Slate Star Codex); Scott Alexander has gone from strength to strength and is more widely read than ever (at Astral Codex Ten). WBW… not so much. The reader will have one of two reactions: “who’s that?” and “oh yeah, whatever happened to him?”
The answer is simple: on 2016-06-18, disgusted by the 2016 US Presidential election and American political trends (especially the Great Awokening), as he explains in his post-mortem, he decided to begin writing what would be a book about politics, What’s Our Problem?: A Self-Help Book for Societies (2023). This was a mistake.
Re-emerging over half a decade later (or 2,440 days) to break his silence, he explains at length how it was a mistake: the scope kept sprawling, he kept thinking he was almost done only to redo more, and sunk cost fallacy kept him from killing it, while he kept procrastinating and trying to do it all, to the extent that:
People in my life were worried about me. They tried encouraging me, shaming me, setting deadlines for me, reminding me that one post really shouldn’t take multiple years. Nothing seemed to help.
|
Year |
n |
|---|---|
|
2014 |
15 |
|
2015 |
11 |
|
2016 |
13 |
|
2017 |
1 |
|
2018 |
1 |
|
2020 |
4 |
|
2021 |
1 |
|
2023 |
2 |
|
2024 |
2 |
Eventually, the woman he proposed to & married, several years later, got pregnant. The thought of having a kid while still writing the book terrified him into finishing it before the pregnancy did.
Was the book any good? I couldn’t tell you. I’ve never read it or seen anyone mention it; when I brought it up in a chat, only one person had read it (and they didn’t like it). I suspect many more people read his post about writing the book than will ever read the book.
And one can see the impact looking at his archives. You do not need advanced statistics to see a change ~2016–2017.2
Another way to put the opportunity cost is to note that if WBW had kept up its lowest pre-book annual rate (2015’s 11 posts), then instead of a total of 11 posts to show for 2017–72024 (ie. less than an average year), it would have 55 more (66 total) posts.3
Anonymous
In another case, the author has not written a post-mortem, but I think the same thing happened.
Around roughly the same time as WBW, there was another popular Internet blogger and writer, who gave well-received talks; he too decided to write a book to sum it all up and popularize his views, and went silent.
About 2–3 years later, I went to dinner with him; I tried to avoid bringing up the book at all, and felt I could sense his constant strain and anxiety that I would ask about it, which made conversation a little difficult. This depression was also clearly visible in their minimal online activity. I wondered if they would ever publish the book, or if the moment had already passed.
And then 4 years after that, somewhat to my surprise, the book finally came out—and it was good! It was positively reviewed and I still see it mentioned (which is more than >99% of books can say), and people still ask occasionally “whatever happened to him?” (If you google his name, “what happened to X” is the first suggested search query.)
So, the reward was real but modest. What was the cost? Unfortunately, he has not returned to his previous activities; so in his case, the opportunity cost appears to have been… everything. Everything he would’ve written if he hadn’t been writing the book. As far as I can tell, several years later, he has not begun writing another book nor taken up any major non-writing activities; he just exists—burned out by the trauma of writing the book. And you can’t fix burnout by doing the same kind of thing, I think, only totally different things.
This is regrettable.
As a third case, one could mention Scott Alexander again—at the same time as WBW he was writing, and finishing (on 2017-05-17), his online fantasy novel Unsong, which I enjoyed. He has since been editing and polishing it up for publication, according to occasional mentions on SSC and later on ACX up to the present. After 7 years of ‘editing’ and negotiations for publication, during which “various things went wrong and I procrastinated”, a fan volunteered to help get it self-published on Amazon. It’s nice that a book now exists, technically, but it feels like the mountain gave birth to a mouse, and one wonders if Scott Alexander would’ve been even more prolific 2017–72024 had he not had Unsong hanging from his neck…?
(And a fourth case would be Wildbow’s Worm, supposedly undergoing intense editing—admittedly, quite necessary in Worm’s case—since 201311ya for ebook & print publication. Last I knew, it was still being edited; meanwhile Wildbow has written at least 3 other web serials, of vast length, with no sign of Worm publication imminent.)
Keeping The Book Inside Them
Now of course, there is a second kind of author: authors who write books with little discernible impact on their newsletter or blog. They sit down for an hour (statistically, in the morning), pound out their quota, and turn to the next task. The book does not weigh on their minds, and they are confident that they will have it out, as usual, in 12 months. More power to them—but I also think that most would-be book authors already know if they are one of that latter kind of writers, or if they are the former kind I’ve been discussing.
If they are the former kind, then they should seriously doubt any plan to write a book: why do they think they will finish? Or that it will be worthwhile? Is there a clear professional reason that the writing has to be a book? (Unless they are named “J. K. Rowling”, they can usually rule out “sales” as an adequate professional reason.)
More importantly, what will the effect on them be? And what is the opportunity cost of writing a book: if they spend the next 5 years devoting all their energies to a book, what will be excluded? Would they happily throw everything they wrote in the previous 5 years into a fireplace in exchange for a finished manuscript of the book they are fantasizing about? Would they willing to wager everything they might ever write on the book going well, rather than becoming an albatross and burning them out? If I had decided in 2019 to write up “a book about AI” in the wake of my GPT-2 experiments, say, and only published it in 2024, and sacrificed writing like “The Scaling Hypothesis” (visionary in May 2020, hopelessly obsolete in August 2024), would I be better off than I am now?4
I suspect that if many writers took seriously writing a book as a real project, rather than an aspirational cultural ideal of ‘being an author’ along the lines of ‘being a foreign language speaker’ or ‘being a PhD’, they would realize that it is a serious risk which can backfire. Everyone may have a book inside them, but some should keep it inside them.
And for now, should a book come out of me despite my best efforts, I will take off and nuke it from orbit. (It’s the only way to be sure.)
-
Tanner Greer describes this mentality in characterizing Washington DC elites:
↩︎In Washington people never read books—they just write them…Washington intellectuals are masters of small mountains. Some of their peaks are more difficult to summit than others. Many smaller slopes are nonetheless jagged and foreboding; climbing these are a mark of true intellectual achievement. But whether the way is smoothly paved or roughly made, the destinations are the same: small heights, little occupied. Those who reach these heights can rest secure. Out of humanity’s many billions there are only a handful of individuals who know their chosen domain as well as they do. They have mastered their mountain: they know its every crag, they have walked its every gully. But it is a small mountain. At its summit their field of view is limited to the narrow range of their own expertise.
The social function of such a book is entirely unrelated to its erudition, elegance, or analytical clarity. It is only partially related to the actual ideas or policy recommendations inside it. In this world of small mountains, books and reports are a sort of proof, a sign of achievement that can be seen by climbers of other peaks. An author has mastered her mountain. The wonk thirsts for authority: once she has written a book, other wonks will give it to her.
-
And this is neglecting that the later WBW posts are often shorter or more insubstantial; counting words & illustrations would probably show a larger loss.↩︎
-
This is a good place to note the “equal-odds rule”: that for creative outputs under normal conditions, each output has a surprisingly equal chance of being a hit (without a clear quantity-quality tradeoff); so each of those 55 foregone WBW posts could have been his biggest hit yet. But they were all sacrificed for a single project, and one without particularly high odds of success.↩︎
-
This is not a completely hypothetical example. Consider NYT AI journalist Cade Metz’s book The Genius Makers, written mostly 2017–32020, and published 2021-03-16. It was obsolete on publication, and its main interest now is “how could Metz have missed the DL scaling revolution so completely even while interviewing so many of the principals as it was happening?” (The short answer seems to be a curious lack of curiosity & pursuit of a “Narrative”.)↩︎