In December 2024 during a visit to San Francisco, I was lucky enough to be invited at the last minute to a party that could only have happened there: a celebration of the 50^th anniversary of hypermedia visionary Ted Nelson’s 1974_52ya manifesto, Computer Lib/Dream Machines, extolling the vision of Project Xanadu hypertext. I’ve contributed to English Wikipedia for 20 years now, and I’ve been working on Gwern.net on and off for 15 years now, so I could not possibly miss an entire party of people with strong opinions on hypertext.

Our host, James, had arranged for a surprisingly extensive collection of Nelson memorabilia: not just copies of that book (far larger and more impressive in person than I had realized, similar to a Compact OED in requiring a magnifying glass, also provided), or Nelson’s1997 book The Future of Information, but also copies of his Swarthmore College mimeograph magazines, and most impressive of all—several vintage computers running copies of various Xanadu implementations.

Ted Nelson was still alive at age 87, but unfortunately could not attend. Fortunately, one of the attendees was one of the former Xanadu programmers from the Autodesk era (c. 1988^–₅1993_33ya), and we could listen to some of his stories.

They were a reminder of how, while we romanticize earlier eras of computing, the hardware constraints were really quite severe, and made productive development difficult. I think some disappointments in past software systems become more comprehensible when we remember how much time and ingenuity is spent working around the limitations—eg. McIlroy1982 is justifiably proud of the months of clever algorithm design & optimization he did to get a useful spellchecker to fit in RAM in under 1MB that we would write in a few lines of JavaScript… that we had a LLM write for us, to boot.

For example, he described how they prototyped Xanadu in Smalltalk (which made sense, as a highly-productive, pleasant language/OS whose object-oriented model matches hypermedia beautifully), but then had to cross-compile it to C++ and compile that… which took about a week to compile. Not a minute, or an hour, or even a day, but a week. (And I thought that Gwern.net’s multi-hour compile-times were bad for my development speed!) He also had to waste a lot of time dealing with C++ silliness and compile issues.¹

Getting this to run at all for the PCs we saw was a challenge, and one reason for the party.

I had, I must admit, sometimes wondered how the Xanadu years at Autodesk could have so little to show for multiple fulltime man-years, when implementing the various client-side transclusion or popup features on Gwern.net were typically a few days of work for Said Achmiz. But hearing some war stories from the horse’s mouth helped put things in proper perspective, and remind me just how incredibly compute-impoverished people were at that time. (Perhaps worse than the CPU was the storage: I take for granted being able to host any PDF or PostScript or HTML file I need, but a meaningful fraction of my hosted documents exceed the total hard drive space of a mid-range 1990_36ya PC, which might have a 50 MB hard drive. Meanwhile, the Markdown source files of the Gwern.net essays are themselves ~40MB, the annotations twice that, and the final site is 221,438 MB!)

The college magazines were a surprise and entertaining to leaf through: young Ted already liked to write, quite a lot, and dispense his advice. Someone mentioned that Nelson had dreamed of being a Hollywood director and regretted that he went into technology instead; I thought that made sense and explained some things about his auteur approach to software development (like his insistence he is “not a programmer”, 50 years later) or use of film-editing metaphors like “edit decision lists”.

I also had never sat down to read Computer Lib/Dream Machines properly, and leafed through a few pages. It was interesting to see such a large book, in multiple columns to get as much in as possible: Nelson can’t assume the reader knows anything about computers and has to start from scratch to explain the basic concepts like bytes or files. (You really would need the magnifying glass if you were older.)

The later (and much more obscure) book Future of Information was also interesting for an unusual structure of chapters, where you could read in multiple orders, with a central summary chapter.

Yuxi Liu points out that Xanadu resembles another famous long-running boil-the-ocean project with close connections to databases & AI with a charismatic leader who never changed his mind, averse to open-source, and which showed similar signs of ‘pathological science’: Douglas Lenat’s Cyc. And thinking about it and re-reading Xanadu materials, I agree.

I briefly poked at the Xanadu PCs, impressed that they were running at all, but I and most of the party-goers bounced off them. The UI was too alien. We really needed to see James demo them, or something like that: hypertext systems do not lend themselves to immediate exploration, especially when they are running in OSes on computers no one there has used in 25 years, if ever.

But I didn’t need to use them much to look at the screen displaying the stereotypical Project Xanadu demo, the opening of the Book of Genesis with its famous lines zig-zagging off to the right to denote transclusions or commentary on passages, and have a sudden realization:

“Oh my god—It’s completely unreadable.”

The lines were confusing clutter, especially as they crisscrossed (a perennial problem in sidenotes layout, made far worse by the outlines). None of the ‘sidenotes’ were readable because the screen was so small. Even as you simply scrolled, for many possible positions, due to the lines you were unable to read anything! How could a document UI where often, you could read nothing, have ever seemed like a good idea?² The UI was just terrible—it could never have worked. Even on a large screen like my 4k monitor, I wouldn’t want that.

And then I thought about the choice of text, and I realized that the UI wasn’t the real problem; and the problem, for all these many decades, wasn’t the team either.

The whole concept of side-by-side range transclusions is a solution in search of a problem.

The range-specific transclusion & commentary made sense for the Book of Genesis, where there are detailed commentaries on every line, and much Biblical criticism sorting out how it’s redacted from multiple contradictory texts (like the famously self-contradicting multiple stories of creation), but as I thought to myself about how “hey, we can do sidenotes and range transcludes/commentaries with bidirectional backlinks on Gwern.net too, and we do do it in my ”Suzanne Delage” short story analysis!”³, I suddenly realized: we can, but we mostly don’t, because no one really needs to do that. Hardly any text in the world actually needs to be fisked, or have specific lines or paragraphs transcluded; hardly anyone is doing Talmudic commentary, nested layer upon layer. Most real instances of citations are citations to the target as a whole. And even with those, we wouldn’t want to see 99% of them, and we wouldn’t know how to organize the good 1% anyway because they may each have a different purpose. (Just look at online comments sometime: do we sort them by date, by popularity or some sort of karma or ‘page rank’, by length, by whether the page author responded directly to them…?)

In retrospect, I think it’s telling that in neither Lib/Dream nor Future did I see any instances of ‘transclusion’ on paper. Nelson could have used any amount of side-by-side layout or range-transclusion in his books, because software is no obstacle: he was laying them out by hand, and could draw or illustrate or copy anything he wanted in any arrangement. But he didn’t, because… it’s just not that useful for books. Not even his. (You don’t need transclusions when you can just cite an earlier passage.)

Note that even in film, where it is trivial to put two sources of footage side by side, and have one ‘comment on’ the other, this is rarely done; instead of relying on horizontal or vertical juxtaposition, film uses the third dimension of time, developing a rich vocabulary of cutting and other tricks for doing transitions, which do the same thing (to an extent we can’t realize until we watch films created before ones like Battleship Potemkin). Spatial juxtaposition is used in various circumstances, but is nowhere near a default. Only in rare circumstances is it conventional to do anything like a picture-in-picture layout. A major niche is live broadcasts where it’s infeasible to shoot from multiple angles & edit coherently; in doing a live interview, it is conventional to have two talking heads simultaneous, but then when edited for later, when time permits, it is often turned into a cut-by-cut sequence. Similarly, a streamer, for example, cannot easily ‘cut’ because they are busy doing the actual stream, do not have a large number of film sources to cut between, do not have skilled staff who can be tied up doing, but if they were making a ‘greatest hits’, they might rearrange or zoom in. (I would bet that when Hololive streamers perform at the big annual concert, involving tens of millions of dollars, they do not adopt streaming-like side-by-side formatting, but go for more typical concert cinematography, just like the Miku Expo Vocaloid concerts.) Notably, where budgets for live broadcasts are very high and quality is a top priority, such as NBA games or NFL or Met HD opera broadcasts, and one can have a large number of cameras as well as a ‘director’ and multiple staff, they typically do not settle for any kind of static picture-in-picture or side-by-side arrangement; instead, they choose to cut a single feed rapidly between different cameras, which lets them ‘follow the action’, order cameramen to move around in advance to make a new angle on the fly, and try to create a meaningful rhythm based on their assessment of the game flow.

And that’s why we didn’t bother adding that specific transclusion capability to Gwern.net until around 2023, motivated by my literary analysis of a short story. I don’t need it, English Wikipedia doesn’t need it, Reddit doesn’t need it, Twitter doesn’t need it, ~100% of personal blogs do not need it… It’s just not that useful—unless you are doing literary criticism or commentary on a text, which describes ~0% of the World Wide Web (in 2025 or 1989_37ya). All those lines look cool and futuristic, but the moment I think about how I would use them in an actual Gwern.net essay and how it would look to read, I start to get a headache.

The famous Project Xanadu UI/UX is a “science fiction interface”, like the 3D gesture interfaces in the Minority Report movie or the virtual reality of Snow Crash: everyone looks at them and is dazzled and wowed by how cool they look, except they are a terrible idea which would be a misery to use and give you “gorilla arm”. You try out a prototype or mockup, and almost as soon as you start using them, you realize that no, this isn’t workable and you have to throw it out completely.

This reminded me of the Hollywood director comment: you could say that Project Xanadu is a Hollywood director’s idea of what a World Wide Web should be. You can only believe in it if you never try to actually use it for any real project. If Ted Nelson had been less charismatic, and less compelling a writer, or had less faith in his own vision, this would have been clearer sooner. Since it was not, like Douglas Engelbart, Nelson enjoys (if that is the word) the honor of being a living embodiment of Cunningham’s Law: becoming one of those historic figures whose importance was that they were so wrong on such an important topic they helped popularize that they inspired others to become right.

In fact, that also explains something about Xanadu that had always puzzled me, its emphasis on copyright. It’s hard to think of any part of the modern world more pernicious and opposed to a useful hypertext system than our current maximalist copyright law, but the Xanadu 17 principles spend more time trying to make the Web safe for copyright holders than they do on such minor things as “transclusion”—it needs to support micropayments at arbitrary levels of transclusion, needs to have default licenses, eschew Net Neutrality, allow permissionless transclusion of arbitrary resources, etc.:

I’d always wondered how anyone could believe any of this was either possible or desirable: of course copyright holders would refuse to comply with any of this, and would reject any Xanadu with horror and loathing, as it blows up countless business models and IP regimes and deals, and would be bogged down instantly with tragedy of the anticommons, holdouts, prima donnas, controlling copyright owners, etc. But of course, if you saw yourself as really a Hollywood director at heart, you would have strong feelings about the sanctity of copyright, you would be fiercely opposed to following the logic of transclusion to free software/open source (despite the last Xanadu principle, Nelson apparently has never believed in FLOSS and insisted on NDAs), and you would believe in a kind of semantic web where you expect… anything to work, really.

As opposed to the reality of what happens when you get 8 billion people online and make much of the world run off the Internet: a nightmarish dark forest where everything that can go wrong will go wrong arbitrarily many times a day; every single invariant you believed in turns out to be broken somewhere in the world⁴; and every nice feature—like trackback bidirectional links—will be ruthlessly abused by spammers, fools & knaves, nation-state actors, emergent bugs or hackers…

Most of those principles make sense only if you don’t ever try it in the real world, where you will discover that many of them are not just of questionable value, or would be fiercely opposed by many entities, but outright illegal. (One of the systems that most embodied the principles of robust decentralized storage of files, which moved them closer to users requesting them for efficiency, was… Freenet, which immediately became notorious for child pornography.)

So, to me, Project Xanadu is a case-study in why designers must mock-up and prototype their designs before too much is invested in them. Xanadu wasn’t the victim of “Worse is Better”; it was just a solution in search of a problem.⁵ (Something similar seems to be true of Arbital: it had one real user, Eliezer Yudkowsky, who had envisioned it, but struggled to communicate the vision or create a clear prototype and set of use-cases, mocked up as necessary; and like Xanadu, it was a square peg of what ought to have been a long-term non-profit FLOSS effort shoved into the round hole of a startup or software company.)

Where were the Xanadu demos and use-cases laid out on paper with some scissors and glue, if need be?⁶ Why are they always just a wildly unrepresentative use-case like “The Book of Genesis”? If you sat down and tried to turn some ordinary technical documentation or the nearest magazine laying near you into a bunch of range-transclusions… how well would it work? How much can, or should be, some text with transclusions spliced in between? (Even in this page about Xanadu, there’s thus far only one point where I need to present a large block transcluded from elsewhere, the list of Xanadu principles copied from Wikipedia—which is covered by a popup link already, and trivial to inline.)

I think if you did this, the answer rapidly becomes, “yes, I need hyperlinks, those are dead-useful; I need to be able to link arbitrary media types like images, audio, video, or specific pages in a book; I need links which don’t linkrot; I need a scripting language (which Xanadu is silent about), but… I don’t really need most of these other things. If I were making some sort of distributed collaborative editor like Google Docs, then I may need something like CRDTs or Xanadu’s similar ”enfilades”, but I don’t need that in my published writing!”

And for me, when I started working on Gwern.net, almost every feature was driven by a use-case I had (mostly because I am lazy); even with Said Achmiz, our approach has been to wait for several use-cases, and then implement it while enabling them all. (Invariably, as Gwern.net has become so large and complex, we discover edge-cases and have to revise the design, and for many changes, abandon them.)

For hypertext, I ask myself, “what do I need that a basic blog-like HTML page or an English Wikipedia page does not provide?”, and the answer is: “I don’t need to transclude just a bunch of random paragraphs from a URL.⁷ I don’t need a bunch of tiny atomic 1-sentence long pages nested in a Table of Contents longer than the actual contents, like GNU Info encourages you to write. What I need is some sort of summary or abstract. I need a reader to be able to browse as fast and friction-free as possible, and quickly skim references or trace interesting citations. I need something like… Lupin’s Wikipedia popups, which allow you to navigate WP with just the most causal mouse navigation, but on steroids.”⁸

I would say the flaw of Xanadu’s UI was treating transclusion as ‘horizontal’ and side-by-side and assuming that all reading/writing must be done at the lowest raw level of text (motivating the ‘tumblers’ etc.), when it should have been ‘vertical’ with popups, and ‘zooming in’ and ‘zooming out’ at different levels of abstraction (link-icon → title → abstract → section etc.) of the text (which motivates an entirely different set of concerns—being able to specify arbitrary ranges becomes much less important, especially as any key ranges can just be hoisted into a higher level).

Once you have popups offering seamless navigation, you are in effect using transclusion everywhere—just inside the popups. And once you have gone all-in on the idea of offering abstracts for everything, it’s natural to generalize it: if you have two versions of a URL, one ‘small’ (the abstract) and one ‘large’ (the whole URL), why not have a ‘medium’ as well?

There is no natural stopping point here, so you can simply embrace a outliner-style hierarchical view of “semantic zoom”.⁹ This leads to many elegant Gwern.net UI/UX design patterns, like: the disclosure/collapses to allow in-place zooming of everything from sections to sentences in paragraphs, or the tag-directories (which rely heavily on transcluding abstracts in place of the link as a whole to create ‘annotated bibliographies’), or the backlinks which improve so much over the backlinks of a Xanadu or English Wikipedia by doing a ‘reverse’ transclusion to show the transclude/link context in the other pages, etc. And the local archive system means that not only do links not break, you can ‘preview’ many URLs by popping up & transcluding the cleaned local archive version.