`/``s, backlinks & links interoperate etc., it all Just Works™ seamlessly on the part of both author & reader.
But a system which threw away the metadata of anchors & IDs would struggle to do any of this: the 1:1 links or the distinct URL-anchors would collapse down to hopelessly-ambiguous many:many maps.
Hyperlinks (or targets like [``](!W "Div and span") elements) are assigned unique IDs manually, by Pandoc automatically, or using the annotation metadata to generate a predictable ID.^[I use author names for my IDs, because that metadata is usually available due to annotations and is easily guessed & written. But other implementations might prefer to instead generate consistent-but-unique IDs by simply stripping or escaping the URL in question (eg. into [Base64](!W) or URL-encoding), or by feeding it into a web-browser-supported hash function like SHA-256 (truncated to 8 chars—there are not nearly enough URLs on any page to worry about collisions).]
(HTML IDs cannot start with a number or contain a period, among other restrictions, so the generated IDs typically take the form of `#surname-year`; thus, a link to the GPT-3 paper [Brown et al 2020](https://arxiv.org/abs/2005.14165#openai "GPT-3: Language Models are Few-Shot Learners"){.backlink-not} will have the ID in this page of `#brown-et-al-2020` and that link in this paragraph can be addressed as `/design#brown-et-al-2020`.
Duplicate IDs are fixed by either a global override to disambiguate two links, or per-page manually-assigned IDs.)
Backlinks are implemented as an offline process which parses the Markdown sources for pages & the HTML for annotations, and extracts URLs+IDs.
They *could* be implemented at compile-time, but it is not a good fit for a static site generator like Hakyll, so I simply run the backlinks as a cron job at night.
Backlinks are not always of any interest (eg. links in newsletters are usually devoid of any interesting commentary or context), and can be disabled on a per-link basis with a `.backlink-not` attribute.
They are then turned into appropriately-formatted HTML lists of transclusions, one snippet per URL.
These are then transcluded *again* in collapsed sections for that URL, however that is displayed.[^lazy-collapse]
[^lazy-collapse]: Showing the context for the backlink requires downloading either the annotation or page, to narrow down to the ID's context. Showing backlink context can use up a lot of space, and rendering all that HTML is expensive, particularly for backlink sections which have scores of backlinks.
So the collapsing serves as [lazy evaluation](!W), and avoids doing that unless the reader requests it. (Since backlinks are all known at compile-time, it would be possible to precompute the context, but not too easy.)
Because the backlinks exploit the general transclusion & collapse functionality, their frontend integration is mostly a matter of just updating the generated HTML snippets and the occasional template.
One might ask how one handles a URL+ID when the URL is not a simple HTML page on Gwern.net, but is perhaps a PDF or a URL on another site (and the URL may not even exist due to linkrot)?
The answer is simple: it is just rewritten to to the URL of the *annotation* HTML snippet + ID, and transcluded.
(It's transclusions all the way down!)
The main challenge comes from a few edge-cases where the backlink popups wouldn't work seamlessly.
For backlinks from an external URL's annotation, the pop-up JavaScript must guess the URL inside the annotation to provide context.
Additionally, a `data-target-id` attribute must be stored inside backlinks to distinguish IDs from actual anchors.
(We considered simply concatenating them like `#target#id`, but that in-band encoding simply led to different ambiguities.)
#### Other Uses
The backlink database has a few other uses:
- A backlink database is a forward-link database if one inverts the query or schema. So if one wants a list of links in a page (ie. its forward-links), one can skip parsing the file and just query the backlink database.
This enables site-wide analyses of links; one use is combining it with the [link-icon](#link-icons) & site-of-the-day features to list domains which are used frequently enough to justify a link-icon or inclusion in site-of-the-day, and to look for frequently-linked but unannotated links.
- The backlinks are also used to generate neural net [embeddings](!W "Word embedding") for annotations (currently used only for the 'similar links' recommendations—is this the [Semantic Web](!W)‽—but I intend to use for other purposes like tag refactoring), enriching their metadata with their site-wide context.
## Similar Links {.collapse}
Gwern.net provides **similar links** ([eg. this page](#similars-section){.aux-links}) where possible, which are a list of annotations which are semantically 'similar' to the current one.
'Similar' is currently defined as an ordinary [_k_ nearest-neighbors lookup](https://en.wikipedia.org/wiki/Nearest_neighbor_search) on a neural net embedding of the annotation text.
Uniquely, similar-links are also ["sorted" by embedding distance](#sort-by-magic), which organizes the list of similar links in a logical fashion & makes it more readable.
## Link Bibliographies {.collapse}
Gwern.net pages & annotations provide a **link bibliography** ([eg. this page](#link-bibliography){.aux-links}): a list of their hyperlinks in order, with the annotation automatically expanding.
Because of the annotations, we do not provide the usual 'Reference' or 'Bibliography' section, but rather something closer to an 'annotated bibliography'.
This lets one skim the 'annotated bibliography' of a page, rather than having to pop up links one by one.
## Tags {.collapse}
![]()
` tags as browser hints
- JavaScript:
- the HTML tables are sortable via [tablesorter](https://mottie.github.io/tablesorter/docs/) (Christian Bach; MIT/GPL)
- the MathML is rendered using [MathJax](!W)
- analytics are handled by [Google Analytics](!W)
- [A/B testing](!W) is done using [ABalytics](https://github.com/danmaz74/ABalytics) (Daniele Mazzini; MIT) which hooks into Google Analytics (see [testing notes](/ab-test "'A/B testing long-form readability on Gwern.net', Gwern 2012"){#gwern-ab-testing-2}) for individual-level testing; when doing site-level long-term testing like in the [advertising A/B tests](/banner "'Banner Ads Considered Harmful', Gwern 2017"){#ads-3}, I simply write the JavaScript manually.
- [Generalized tooltip popups](/static/js/popups.js "'
- *correctness* (consistent rendering):
The fundamental reason for not using system fonts is that there are not many of them, they vary across operating systems & devices, usually aren't great (lacking alternatives & features like smallcaps, & often basics like Unicode), and can be buggy (eg. Apple ships a [Gill Sans](!W) digitization—not an obscure font!—which is [>22 years old & has broken kerning](https://github.com/ForumMagnum/ForumMagnum/issues/6713)).
I initially used system ["Baskerville"](!W), but they looked bad on some screens (similar issue to people imitating LaTeX by using [Computer Modern](!W) on screens) and the highly limited selection of system fonts didn't give me many options. The [Google Fonts](!W) Baskerville was OK but lacked many features & was slower than hosting my own webfont, so Said Achmiz convinced me to just switch to self-hosting the ['screen serif'](https://blog.obormot.net/Screen-serif-fonts) Source family, whose appearance I liked, which could be subset down to only necessary characters to be faster than Google Fonts & not a bottleneck, and which wasn't widely used then despite being FLOSS & high-quality & actively maintained (so helped my personal branding).
We were then repeatedly forced to add more fonts to fix display bugs: fonts could look quite different on Linux & Mac, and the system "sans" for the Table of Contents looked bad on Windows. The more carefully designed the appearance, the more small differences in sizes or appearance between the 'same' font on different platforms screwed things up. Link-icons, sidenotes, emoji, return-arrows, miscellaneous Unicode looking rather different or breaking—all of these have run into platform issues, and later features like citation subscripts or [inflation-adjustments](#inflation) surely would if we couldn't tune their CSS to a known font.
(Should we let readers set their own fonts? Reader, be real. It is 2023, not 1993. No one today sets their own fonts or writes custom CSS stylesheets—which would break layout & icons on many sites anyway—and they *especially* do not on the mobile devices which are ~50% of my site traffic.)
- *smallcaps*: used extensively in the site design for varying levels of emphasis in between bold & italics, so high quality smallcaps is critical; true smallcaps is provided by Source (italics may yet be added [at my request](https://github.com/adobe-fonts/source-serif/issues/46)), while unavailable in most fonts
- *consistent monochrome emoji* via [Noto](https://en.wikipedia.org/wiki/Noto_fonts) [Emoji](https://en.wikipedia.org/wiki/Noto_fonts#Emoji) (before, emoji would be different sizes in link-icons, some would be unpredictably colored on some platforms and scream on the page)
- [IBM Plex Mono](!W) for source code: *more distinct confusable-characters* in IBM Plex Mono compared to an ordinary monospace system font (Plex Mono has an [OpenType](!W) feature for slashed-zeros which can be enabled in just source code), and looks good on Macs.^[IBM Plex Mono was chosen in part via using the [CodingFont](https://www.codingfont.com/) 'tournament'; [Adobe Source Code Pro](!W) also ranked high, and we used it initially, but Plex Mono edged it out, with its useful alternatives and a somewhat better overall appearance. (Who knew IBM could commission such a nice monospace font?)]
- the Source family also provides both [**tabular & proportional numbers**](https://en.wikipedia.org/wiki/Typeface#Typesetting_numbers) (also called ["old-style"](https://practicaltypography.com/alternate-figures.html#oldstyle-figures)), which most fonts don't, and which makes tables vs text more readable (proportional numbers would break visual alignment inside tables analogous to proportional vs monospace fonts for code, while tabular numbers look large & obtrusive inside regular text)
- *icons* via [Quivira font](http://www.quivira-font.com/), and rare characters like [interrobang](!W), [asterism](!W "Asterism (typography)"), [backwards question mark](!W "Irony punctuation"), shield-with-cross (for link icon), BLACK SQUARE ■, and consistency in fallback characters for rare Unicode points not in the subsets.
- efficient dropcaps fonts [by [subsetting]{.smallcaps}](/dropcap#web-dropcap-implementation)
- **image optimization**: PNGs are optimized by `pngnq`/`advpng`, JPEGs with `mozjpeg`, SVGs are minified, PDFs are compressed with [`ocrmypdf`'s](https://github.com/ocrmypdf/OCRmyPDF) [JBIG2](!W) support. (GIFs are not used at all in favor of WebM/MP4 `
Gwern.net implements a simple hierarchical/DAG [tag](!W "Tag (metadata)") system for all links, modeled on [Wikipedia's categories](https://en.wikipedia.org/wiki/Help:Category) (see [`Tags.hs`](/static/build/Tags.hs)).
It is designed to be browsed via pop-ups, and integrate naturally with the filesystem.
These hierarchical tags correspond to the filesystem hierarchy: a URL can be 'tagged' with a string `foo`, in which case it is assigned to the `/doc/foo/` directory.^[This replaces an earlier [Hakyll-based tag system](https://jaspervdj.be/hakyll/reference/Hakyll-Web-Tags.html). The Hakyll approach was quite simple and intended just for small blogs, and had no way to handle tagging local files, much less arbitrary URLs. (The tag code was also black magic I couldn't modify.) Meanwhile, the evolving filesystem hierarchy for my local files already *looked* like a tag system, and the evolution was easy. ]
If the tag string has a forward-slash in it, then it refers to a nested tag, like `foo/bar` → `/doc/foo/bar/`.
Thus, a file added to a Gwern.net directory like `/doc/foo/bar/2023-smith.pdf` is inferred to automatically be tagged `foo/bar`.
(Because it is hierarchical, it cannot be tagged both `foo` and `foo/bar`; that is interpreted as simply `foo/bar`.)
As it would be a bad idea to copy/symlink files around, a given URL (such as a file) can be tagged arbitrarily many times.
This is tracked in the same metadata database as the annotations, and can be edited like any other part of the annotation.
It is implemented as a [standalone batch process](/static/build/app/generateDirectory.hs), which reads a list of directories, queries the annotation database for annotations which match each directory's implied tag, and writes out a Markdown `index.md` file with a list of transclusions, which is then compiled normally.
### Properties
Tags are first-class citizens, in that they are pages/essays of their own, and can be tagged like any other URL:
#. Tags as **pages**:
Tags can have introductions/discussions of the topic, for cases where the meaning of the tag might not be obvious (eg. ["inner-monologue"](/doc/psychology/inner-voice/index) or ["dark knowledge"](/doc/psychology/dark-knowledge/index)).
These introductions are processed like an essay, and indeed, may just be *transcluded* from a regular page (eg. the [boogers tag](/doc/biology/booger/index "‘Booger Picking’, Gwern 2021") or [efficient Transformer attention](/doc/ai/nn/transformer/attention/index "‘Efficient Attention: Breaking The Quadratic Transformer Bottleneck’, Gwern 2020")).
#. **Tagged** tags:
Tags can themselves be 'tagged', and appear under that tag (and vice-versa); these tags are not recursive, however, and make no attempt to avoid cycles. They are meant more in the spirit of a 'see also' cross-reference.
#. Tagging **any URL**: as tags are on URLs/annotations, they treat different URLs+[anchor](https://en.wikipedia.org/wiki/HTML_element#Anchor)-fragments as different.
Thus, you can create annotations for multiple anchors on a page or multiple pages inside a PDF, and annotate & tag them separately. See the [backlinks discussion](#multiple-links) of how useful this hack can be. (Because anchors do not need to exist inside a URL—the browser will simply load the URL normally if the anchor can't be found—you can even treat anchors as a way to 'tag' URLs with arbitrary metadata while requiring no database or other software support[^affiliation-anchors].)
[^affiliation-anchors]: I abuse anchors in this way to track 'affiliation' of URLs, both for easier reference/search and for setting link-icons. For example, I regard DeepMind authorship of a paper as being a helpful thing to know, and so I append `#deepmind` to any DeepMind-related URL (eg. `https://arxiv.org/abs/1704.03073#deepmind`). I find it particularly helpful for tracking Chinese AI research, where they have a habit of quietly dropping revealing papers on Arxiv with no PR or Western attention.
When I link that URL, the link will get the DeepMind logo as its link-icon, and it is easier for me to search if I can remember that it was DM-related. This will not break the link, because the anchor is client-side only (unlike if you wanted to abuse query-parameters this way—many servers would ignore a malformed url like `foo?deepmind`, but many others would throw an error); thus, I can copy-paste back and forth between Gwern.net and Reddit or Twitter, and the latter will continue to work normally (they track the full URL but usually drop the anchor for purposes like searching). Because it's overloading the anchor, I can define new affiliations any time I please, and am up to 51 affiliations as of 2023-04-19; and I could encode anything else I might want to encode by using a new convention. I could encode the date by writing `#2023-04-19`, or the author, `#john-smith`, or little notes like `#todo`. As long as they do not happen to be a *real* anchor, they will work. (This is why a number of past web design hacks like [`#!'](https://en.wikipedia.org/wiki/URI_fragment#Proposals) [URLs](https://en.wikipedia.org/wiki/Single-page_application) or ["Text Fragments"](https://developer.mozilla.org/en-US/docs/Web/URI/Fragment/Text_fragments) (crude [content-addressable URLs](!W "Content centric networking")) have also exploited anchors, for their backwards-compatibility.)
This hack does come with costs. First, it creates spurious anchors which my linkcheckers will warn about but must be ignored as deliberate errors. Second and more seriously, while it works fine on external URLs, it begins to cause problems with local URLs: consider a case like `/doc/reinforcement-learning/model-free/2016-graves.pdf#deepmind`—this URL is not itself a problem for annotations, which does everything by URL, but it is a problem for anything at the file level, which sees only `.../2016-graves.pdf`. There is no `#deepmind` at the file level! This requires flaky hacks like looking up every annotation with the file as a prefix to see if there is an annotation with some anchor. I intend to remove this hack in favor of storing affiliations inside the annotation metadata; however, I may keep it as a convenient way to input affiliations.
### Use & Appearance
The primary way to browse tags is via popups on annotated URLs:
The tag popups give an overview of the tag as a whole: how many tagged items there are of what type, how it's tagged & access to its broader parent tag, the raw tag name, image thumbnails (extracted from the most recent annotation with an image), and a compact table-of-contents which will pop up those annotations.
Standard features like link-bibliographies are supported, and it's all implemented by popup and/or transclusion.^[The tag popups used to simply transclude/load the tag page into the popup. This turned out to be unpredictable & slow for the mega-tags like [`psychology`](/doc/psychology/index) which had hundreds to thousands of entries (and are in desperate need of refactoring), which were also not transclusions at the time, and so could take >10s to load.]
Like the backlinks, there is little difference between tags and everything else—it all Just Works™ from the reader's perspective.
For more in-depth reading, the tags are available as standalone HTML pages.
The idea for these pages is that one might be searching for a key reference, or trying to catch up on the latest research.
So, these are organized as the preface/introduction (if any), then the parental & children & cross-referenced tags (with arrows to indicate which), then the annotations in reverse chronological order (requiring a date & title), then the Wikipedia links (which come last, inasmuch as they do not have well-defined 'dates'); then, a 'miscellaneous' section lists URLs which have at least 1 tag but otherwise lack key metadata like a title, author, or date; finally, the link-bibliography, which is the concatenation of all of the annotated entries' individual link-bibliographies.
These items all make heavy use of the laziness of transcludes & collapses to render acceptably—they are so dense with hyperlinks that a fully-transcluded page would bring a web browser to its knees (likely one reason that websites like Wikipedia do not even try to provide similar interfaces).
Tags are a key way of organizing large numbers of annotations.
In some cases, they replace sections of pages or entire pages, where there would otherwise be a hand-maintained bibliography.
For example, I try to track uses of the [DNM Archive](/dnm-archive#works-using-this-dataset) & [Danbooru20xx](/danbooru2021#publications) datasets to help establish their value & archive uses of them; I used to hand-link each reverse-citation, while having to also tag/annotate them manually.
But with tags+transclusions, I can simply set up a tag solely for URLs involving uses of the dataset (`darknet-market/dnm-archive` & `ai/anime/danbooru`), and transclude the tag into a section.
Now each URL will appear automatically when I tag it, with no further effort.
### Features
Convenience features:
#. **Generated** tags: There are two special tags which are 'generated' (more so):
- [`newest`](/doc/newest/index), which lists the most recently added annotations (as a sort of live equivalent of the [monthly newsletter](https://gwern.substack.com/ "‘Gwern.net newsletter (Substack subscription page)’, Gwern 2013") & let me easily proofread recently-written annotations)
- and the root tag-directory itself, [`doc`](/doc/index), which lists *all* tags by path & human-readable short-name (to show the reader the full breadth of tags available).
#. **Short ↔ Long** tag-names:
For brevity, the Gwern.net tag taxonomy does not attempt to be a perfect categorical pyramid.
It supports 'short' or 'pet names', which are short human-readable versions of otherwise-opaque long tag names.
(For example, `genetics/heritable/correlation/mendelian-randomization` → "[Mendelian Randomization](!W)".)
In the other direction, it attempts to intelligently guess what any short tag might refer to: if I attempt on the CLI to run the command to upload a new document like [upload](/static/build/upload.sh) 1981-knuth.pdf latex, the tag code will guess that `design/typography/tex` is meant, and upload to that tag-directory instead.
#. Inferred or **automatic tags**:
- To bootstrap the tag taxonomy, I defined rules that any URL linked by a page would get a specific tag; for example, the [DNB FAQ](/dnb-faq "‘Dual n-Back FAQ’, Gwern 2009"){.backlink-not} would impose the `dual-n-back` tag. This proved to be too free with the tags, and has been removed.
- A locally-hosted file typically has a tag encoded into its path, as discussed before. (This excludes the special-case of the local mirrors in `/doc/www/`, and a few mirrors or projects.)
- Domain matches can trigger a tag, in cases where either a domain is a tag of its own (eg. [_The Public Domain Review_](https://publicdomainreview.org/) has its own tag at [`history/public-domain-review`](/doc/history/public-domain-review/index) so it is convenient to auto-tag any URL matching `publicdomainreview.org`), or where the website is single-topic (any link to [EvaMonkey.com](https://www.evamonkey.com/) will be an [`anime/eva`](/doc/anime/eva/index) tag).
#. CLI tools: [`changeTag.hs`](/static/build/app/changeTag.hs) and `upload` allow editing & creating annotations in large batches, and [`annotation-dump.hs`](/static/build/annotation-dump.hs) enables search/browsing:
I use `changeTag.hs` (shortcut: `gwt`) as a kind of [bookmarking](!W "Bookmark (digital)") [tool](!W "Social bookmarking") to 'tag' any URLs I come across. ([Tab-completion](!W) is easily provided by listing all the directory names & turning them into tags.) For example, an interesting [Arxiv](https://en.wikipedia.org/wiki/ArXiv) link will get a quick `gwt https://arxiv.org/abs/2106.11297 attention/compression t5`; this will create [the annotation for it](https://arxiv.org/abs/2106.11297 "‘TokenLearner: What Can 8 Learned Tokens Do for Images and Videos?’, Ryoo et al 2021"){.backlink-not}, pulling all its metadata from Arxiv, running all the formatting passes like paragraphizing, generates an embedding for it which will be included in all future similar-links recommendations, adding it to the local-archiving queue, and tagging it under [`ai/nn/transformer/attention/compression`](/doc/ai/nn/transformer/attention/compression/index) & [`ai/nn/transformer/t5`](/doc/ai/nn/transformer/t5/index). Beats doing that manually!
Meanwhile, `annotation-dump.hs` (shortcut: `gwa`) helps me make some actual use of tagging to refind things eg. `gwa https://arxiv.org/abs/2106.11297 | fold --spaces --width=100`:
, showing [syntax highlighting](https://en.wikipedia.org/wiki/Syntax_highlighting), shortcuts like a full Gwern.net URL to view the annotation, tags, which YAML file database it's in, etc.](/doc/cs/shell/2023-04-18-gwern-gwernnet-commandline-annotationdump-queryexample.png)
These can be grepped, piped, edited in a text editor, etc. This can be combined with `gwt` for bulk edits: grep for particular keywords, filtering out already-tagged annotations, pipe into `less`, review by hand, and copy the URLs of ones to tag/un-tag. Which can be further combined with `linkExtractor.hs` to extract links from given Markdown pages, see if they are already tagged with a tag, and present just the untagged ones for review.
For example, when I wanted to populate my [Frank Herbert](/doc/fiction/science-fiction/frank-herbert/index) tag, I extracted the links from my two _Dune_-related pages, grepped for any annotation *mentioning* any of those links, filtered out any annotation which was already tagged 'Frank Herbert', and printed out just the URL of the remainder for review, and tagged many of them:
~~~{.Bash .collapse}
TMP=$(mktemp /tmp/urls.txt.XXXX)
cat ./dune-genetics.md ./dune.md | pandoc -f markdown -w markdown | \
linkExtractor | \
sort --unique | grep -E -v -e '^#' >> "$TMP"
cat "$TMP"
gwa | grep -F --color=always --file="$TMP" | \
grep -F -v -e '"fiction/science-fiction/frank-herbert"' | \
cut --delimiter=';' --fields='1-4' | less
gwt 'frank-herbert' […]
~~~
#### Future Tag Features
Future work: the Gwern.net tag system is incomplete due to poor tooling.
Tags are an unsolved problem, currently solved by human brute force, but a far better tagging future is possible with deep learning.
When I look at the history of tagging, [folksonomies](!W), and bookmarking services like Wikipedia, [del.icio.us](!W "Delicious (website)") & [Archive of Our Own](!W) (see ["Fan is A Tool-Using Animal"](https://idlewords.com/talks/fan_is_a_tool_using_animal.htm)), what I see is a tool with too much friction for individual readers.
It is easy to set up a simple tag system, and handle a few hundred or thousand links.
(Indeed, anyone who starts keeping bookmarks at all will quickly develop an ad hoc category or tag system.)
It is not so easy to keep using it for years productively; all tag GUIs are clunky and require many seconds to do anything, and their 'automation' is minimal.
Such systems should get easier & faster & smarter to use over time, but usually get harder & slower & dumber.
Like [spaced repetition](/spaced-repetition "‘Spaced Repetition for Efficient Learning’, Gwern 2009") or [complex personalized software](/backstop#knuth), the average user tastes the initial fruits of using tags, goes perhaps a bit overboard, begins to bog down under the burden of maintenance, doesn't quite have time to split up tags or populate obscure tags, and the system begins to careen out of control with 'mega' tags containing half the world while obscure tags have only 1 entry—as the technical debt escalates, the user gets ever less value out of it, and simply looking at the tags becomes more painful as one sees the undone work pile up, like an email inbox.
One sees many blogs which use 'tags' but in a completely pointless way: there is one tag which is on every other post and has thousands of entries, and then each entry will have 1 tag which is used pretty much only on that entry & never again.
No one reads or uses those tags, including their authors.
In this failure mode, particularly evident on [Tumblr](!W) & [Instagram](!W), as [Hillel Wayne](https://buttondown.com/hillelwayne/archive/tag-systems/) notes, tags become utterly debased ["metacrap"](https://people.well.com/user/doctorow/metacrap.htm) by huge swathes of redundant tags being slapped onto every post—if you ever need to refind a particular post, it sure won't be through the tags...^[Amusingly, while useless for their ostensible intended purpose, Instagram tags were useful for early landmark results in deep learning scaling (particularly [Mahajan et al 2018](https://arxiv.org/abs/1805.00932#facebook "Exploring the Limits of Weakly Supervised Pretraining")), helping to establish that neural nets could learn from *billions* of images when the expert conventional wisdom was that millions was all that was useful & neural nets were 'fundamentally unable to scale'.]
At the other extreme is Twitter: Twitter's famous ['hashtags'](!W) *used* to be widely used, and were key organizing tools... but somewhere along the way, real Twitter users seemed to stop using them & they became spam.
Is it any wonder that most users eventually acknowledge that it's a waste of time unless they begin a second career as reference librarians, and give up, depending on their search skills to refind anything they need?
(And to be fair, for many users, they probably did not really need tagging to begin with. It was an attractive nuisance for them, an illusion of productivity—like alphabetizing one's bookshelf.)
This is why sites that *do* make productive use of tags tend to be sites catering to niches, with power users, and highly active curators (named things like 'librarians' or [WikiGnomes](https://en.wikipedia.org/wiki/Wikipedia:WikiGnome)) of some sort which will clean up & enforce standards.
For example, Wikipedia editors put an enormous amount of effort into maintaining an incredibly elaborate category system, with extensive bot tooling; and Wikipedia editors (if not regular readers) benefit, because they do use it extensively for both content editing and organizing the infinite amount of meta-editing content (like categories of [overloaded](https://en.wikipedia.org/wiki/Category:Categories_requiring_diffusion) [categories](https://en.wikipedia.org/wiki/Category:Overpopulated_categories)).
Archive of Our Own likewise is renowned for its extensive tag system, which is ['wrangled'](https://archiveofourown.org/wrangling_guidelines/11) by rabid fans into a reasonably consistently-applied folksonomy of characters/franchises/topics, and those are used heavily by its readers to navigate the >10m fanfics (in lieu of any more legible way to navigate the seas of fanfics—after all, the point of fanfic is that anyone can do it, posing curation problems which don't exist for the original works).
Or to put it another way, current tag systems are not like a Ted Nelson or _Minority Report_-esque experience of wizards weaving with the stuff of thought, casually grouping & orchestrating flocks of items on the screen; but more like going to the DMV to fill out forms in triplicate, or using tweezers to move a pile of sand grain by grain.
Successful tag systems are like the Pyramids of Egypt: monumental feats of labor by thousands of people laboring for years to push ungainly building blocks precisely into place.
Tags are powered by herds of human brains, tediously, one by one, learning a tiny fragment of the total tag folksonomy, adding it via clunky software, drumming their fingers while their web browsers clunk through the back-and-forth, spending hours refactoring a list of a thousand entries in 1 tag into 2 tags after staring at the list for a while trying to imagine what those 2 tags could be, and doing this all with no visible reward.
For a shared resource like Wikipedia, this is worthwhile; for your own personal files... the return on investment is dubious. (Also, people are lazy & forgetful.)
But with modern tools, particularly DL NLP tools like document embeddings, the tag experience could be so much better—even magical.
The 3 major pain points of a tag system are tagging new items, refactoring big tags into smaller tags (typically a single tag into 2–4), populating a new/small tag with existing items, and reading/searching large tags.
All of these can be made drastically better.
Implemented with care and an eye to performance, these 4 techniques would remove most of the pain of a tag system; indeed, curating tags might even be pleasant, in a popping-bubblewrap sort of way:
#. to **automatically tag a document** with high accuracy is well within 2023 capabilities.
Many documents come with a summary or abstract which can be embedded. For those which don't, 2023+ LLMs generally have long enough context windows to embed entire documents (at some expense, and perhaps lower embedding quality); they are also generally capable of writing accurate summaries.
For >90% of my annotations, the appropriate tags would be extremely obvious to a classifier (eg. [random forests](!W)) trained on my existing corpus of tags+embeddings^[Embeddings which are enhanced by including their internal links, backlinks, manually-curated similar-links (the 'see-also' sections of annotations), and their prior tags, it's worth noting: all these should enhance the taggability & clusterability of the embeddings.], and the tags could be automatically generated.^[You can use LLMs directly for tagging, with tricks like finetuning or including a list of valid tags to choose from, but these will probably be less accurate than a classifier, will tend to be slower & less suitable for [real-time active-learning of tags](#active-learning), and embeddings are reusable for other purposes.]
Indeed, I view my tagging efforts as partially justified by training a future classifier on it, and this is just the costly bootstrap phase.
And as the tag collection gets larger, the accuracy of tagging improves & the user will be asked to tag fewer items, rewarding the user.
#. **Refactoring tags into sub-tags** is harder, but probably doable as an interactive clustering problem.
After a few hours re-tagging one's documents, one wants to yell at the computer: "look, it's obvious what this tag should be, just split them up the obvious way and do what I mean!" It's often *so* obvious one can get halfway there with regexps... but not the other half.
One can take the embeddings of an overly-large tag, and run a clustering algorithm like [_k_-means clustering](!W)^[An alternative would be [_k_-medoids](!W), which would construct clusters whose 'center' is a specific datapoint (with _k_-means, there is not necessarily a datapoint at the center of a cluster), making interpretability easier for the user and possibly creating higher-quality clusters. We wouldn't want to use [DBSCAN](!W) because it would ignore many points as 'outliers'; this is reasonable with real-world data where datapoints may be unpredictable or outright garbage, but in tagging, we can assume that all data is valid, and so we want to keep 'outliers' and consider assigning them their own tag—maybe they are simply immature.] on it for various _k_, like 2–10. Then one can present the different clusterings as sets, with the most central items in each cluster as its prototypical examples. It should be obvious to an expert user what the clusters are, and what the best _k_ is: 'these 3 clusters make sense, but then at 4 it breaks down and I can't tell what #3 and #4 in that are supposed to be.'
Given a clustering, one can then put the prototypical examples into a tool like GPT to ask it what the name of the tag for those examples ought to be. (This is similar to how the OpenAI ChatGPT interface will automatically 'title' each ChatGPT session to provide meaningful summaries, without the user having to do so.[^auto-summarization] Could they? Of course. But that is work.) The user will approve or provide his own.
[^auto-summarization]: For example, my last few sessions' automatic titles/summaries: "Insulated container temperature", "Dangerous metadata dates", "Haskell Recursive File Listing", "Plastic Pipes Reduce Build-up", "Template for unsupported citations", "Buffer rewrite condition modification", "Git pull with automatic merge", "Grep for non-printable characters", "Alternative to `(keyboard-quit)`", "Reverse Markdown List Order", "Goal Setting Theory & Conscientiousness", "Odin's [Favorite Spice](!W "Allspice")?", "Cat lounging against chair", ["Decrypt Poem Message"](https://news.ycombinator.com/item?id=34977741), "Ladybug Forklift Certification".
Why not do this for everything, like filenames, and end the ['blinking 12' problem](!W "Blinking twelve problem") of `Untitled (89).doc`?
After all, the problem with these is due less to the *intrinsic complexity* or difficulty—it takes all of a minute to read the manual or figure it out or decide what to call a file—so much as the irksomeness of taking a minute to do so every time one would have to--every power outage, Daylight Savings, VCR upgrade, random document---for what is ultimately a tiny benefit per-instance. (A real clerical benefit, which adds up in aggregate, but still, small individually, and thus easily outweighed by the hassle.) But an LLM can do it easily and won't complain.
With the clustering chosen & labels, the tag can be automatically refactored into the new tags. The effort to refactor a tag goes from 'several hours of extremely tedious work actively reading through thousands of items to try to infer some good tags and then apply it, one by one' to 'a minute of pleasant consideration of several options presented to one'.
As the number of tags increases, the number of necessary refactorings will decrease ([power law](/doc/design/2007-halpin.pdf "‘The complex dynamics of collaborative tagging’, Halpin et al 2007"), apparently, which makes sense given [Zipf's law](!W)) and the automatic tagging of future items will improve (both because the semantics becomes richer and because if a tag-cluster could be found in an unsupervised fashion by clustering embeddings, then it would be even easier to predict those tag-clusters given a labeled dataset), again rewarding the user and improving in quality over time.
(Much less frequently, we will want to combine tags. But that's trivial to automate.)
#. **Populating a rare tag**:
Sometimes a user will create a useful tag, or a small cluster will pop out out of the clustering because it is so distinct. If it's a good tag, it may have many valid instances, but scattered across the whole dataset. In this case, the automatic tagging of new items, or refactoring existing tags, will not help. You need to go back over existing items.
In this case, creating the rare tag would integrate well with [[active learning]{.smallcaps}](https://en.wikipedia.org/wiki/Active_learning_(machine_learning)) approaches.
The simplest active learning approach (uncertainty sampling) would look something like this: the user creates a new tag, and adds a few initial examples.
The tag classifier immediately retrains on this, and creates ranked list of all untagged instances, sorted by its estimated probability. The user looks over the list, and tags a few on the first screen's worth.
They are tagged, the rest on that screen ignored henceforth[^negative-tags], and the classifier immediately retrains, and produces another ranked list. A tag classifier like a random forests can train on quite large datasets in seconds, so this could proceed in near-realtime, or indeed, asynchronously, with the user just tapping 'yes'/'no' on instances as they pop up on the screen while the classifier trains & reclassifies in a loop in the background.
Like the refactoring, this demands much less of the user than the traditional manual approach of 'work really hard'.
(Such semi-automated tagging approaches are widely used in ML industry to create datasets like [JFT-300M](https://arxiv.org/abs/1707.02968#google "‘Revisiting Unreasonable Effectiveness of Data in Deep Learning Era’, Sun et al 2017") because they make labeling vastly more efficient, but has not been seen much in end-user software.)
Within minutes, the new tag would be fully populated and look as if it had been there all along.
#.
Presenting a tag's contents in a logical order—**sorting by semantic similarity**, which looks like a "sort by magic":
But we can go further.
*Why* is a mega-tag such a pain to read or search?
Well, one problem is that they tend to be a giant messy pile with no order aside from reverse-chronological.
Reverse-chronological order is bad in many cases, even blogs (consider a multi-part series where you can only reach them all by the tag, which of course shows you the series in the worst possible order!), and is used simply because... how *else* are you going to sort them?
At least that shows you the newest ones, which is not always a good order, but at least is *an* order.
To go beyond that, you'd need some sort of semantic understanding, the sort of deeper understanding that a human would have (and of course, human brains, particularly the brain of your human user, are too expensive to use to provide some sensible order).
Fortunately, we have those document embeddings at hand.
We could try clustering with _k_-means & titling with an LLM again, and display each cluster one after another, treating the clusters as 'temporary' or 'pseudo' tags.^[This could, in fact, be how we do the clustering refactoring to begin with: we simply cluster by default (arbitrarily sorting across-clusters, and then semantic-sort within-cluster), and the user can hit a little button to 'canonize' a cluster as a new tag.]
(We can easily name the anonymous clusters [with an LLM](/static/build/tagguesser.py) by feeding in the metadata like titles and asking for a tag name. The user may reject it, but even a wrong tag name is extremely helpful for making it obvious what the right one is, and breaks "the tyranny of the blank page" and decision fatigue.)
Clusters don't respect our 2D reading order, but there are alternate ways of clustering which are intended to project the high-dimensional embedding's clustering geometry [down to fewer dimensions](!W "Nonlinear dimensionality reduction"), like 2D, for use in graphs, or even 1D—eg. [t-SNE](!W "t-distributed stochastic neighbor embedding") or [UMAP](https://arxiv.org/abs/1802.03426 "‘UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction’, Lel et al 2018").
I don't know if they work well in 1D, but if they work better at slightly higher dimensionality, then it can be easily turned into a sequence with minimum total distance as a [traveling salesman problem](!W).
A simple way to 'sort' which doesn't require heavy-weight machinery is to 'sort by semantics': however, not by distance from a specific point, but greedily pairwise.
One selects an arbitrary starting point ('most recent item' is a logical starting point for a tag), finds the 'nearest' point, adds it to the list, and then the nearest unused point to *that* point, and so on recursively.
I find with Gwern.net annotations, the greedy list sorting algorithm works surprisingly well.
It naturally produces a fairly logical sequence with occasional 'jumps' as the latent cluster changes, in contrast to the naive 'sort by distance' which would tend to 'ping-pong' or 'zig-zag' back & forth across clusters based on slight differences in distance.^[Imagine you are sorting a list of items A–Z ([ABCDEFGHIJKMLNOPQRSTUVWXYZ]), where you have pairwise distances like 'A is closer to B than C or Z'. If you pick, say, 'H', and then simply 'sort by distance' to 'H' to form a 1D list, the result would merely 'ping-pong' back and forth: [HIGJFKEMDOCPB...]. (The more equidistant clusters there are, the worse this ping-pong effect is.) This would be meaningful in that 'P' really is slightly closer to 'H' than 'B', but the back-and-forth would look confusing to any reader, who wouldn't see the underlying A–Z. However, if you sorted pairwise greedily, you would get a list like [HIJKMLN...YZABCDEFG], and aside from the 'jump' at A/Z, this would be meaningful to the reader and much more pleasant to browse. It would also be easier to curate, as you can *see* the sequence and also the 'jump', and, say, decide to edit A–G into its own tag, and then further refine what's left into H–L & M–Z tags.]
This would implicitly expose the underlying structure by preserving the local geometry (even if the 'global' shape doesn't make sense), and help a reader skim through, as they feel 'hot' and 'cold', and can focus on the region of the tag which seems closest to what they want. (And if the tag really needs to be chronological, or the embedding linearization is bad, there can just be a setting to override that.)
This approach would work for anything that can be usefully embedded, and would probably work even better for images given how hard it is to organize images & how good image embeddings like CLIP have become (eg. [Concept](https://github.com/MaartenGr/Concept) or [SOOT](https://every.to/napkin-math/6-new-theories-about-ai "‘6 New Theories About AI: Software with superpowers § GPT-4’, Armstrong 2022")).
This will wind up inevitably producing some abrupt transitions between clusters, but that tells you where the natural categories are, and you can easily drag-and-drop the cluster of images into folders & redo the trick inside each directory.
This would make it easy to drag-and-drop a set of datapoints, and select them, and define a new tag which applies to them.
And because embeddings are such widely-used tools, there are many tricks one can use.
For example, the default embedding might not put enough weight on what you want, and might wind up clustering by something like 'average color' or 'real-world location'.
But [embedders can be prompted](https://arxiv.org/abs/2212.09741 "‘One Embedder, Any Task: Instruction-Finetuned Text Embeddings (INSTRUCTOR)’, Su et al 2022") to target specific use-cases, and if that is not possible, you can manipulate the embedding directly based on the embedding of a specific point such as a prototypical file (or a query/keyword prompt if text-only or using a cross-modal embedding like CLIP's image+text): embed the new file or prompt, weighted multiply all the others by it (or something), then re-organize.
(And you can of course finetune any embedding model with the user's improvements, contrastively: push further apart the points that the user indicated were not as alike as the embedding indicated, and vice-versa.
This works most easily with the model that generated the embedding, but one can come up with tricks to finetune other models as well on the user actions.)
Or you can experiment with "embedding arithmetic": if the default 2D layout is unhelpful, because the most visible variation is focused on unhelpful parts, one can 'subtract' embeddings to change what shows up. And you can do this with any number of embeddings by doing arithmetic on them first. For example, you can 'subtract' a tag _X_ from every datapoint to ignore their _X_-ness, by averaging every datapoint with tag _X_ to get a "prototypical _X_"; the new embeddings are now "what those datapoints mean besides the concept encoded tag _X_".^[If you had a corpus of restaurant reviews, you might want to cluster them by the type of food, but your default embedding keeps grouping them by geography when reduced to a 2D GUI; no problem, just subtract the tags for all of the cities like "New York City", and then what's left afterwards will probably be clumping by French vs Asian fusion vs Chinese etc. and you can easily lasso each cluster and name them and create new tags with hardly any toil.] (If the right tag or datapoint doesn't exist which emphasises the right thing---just make one up!)
By sequentially subtracting, one can look through the dataset as a whole for 'missing' tags; indeed, if every tag is subtracted, the residual clusters might still be surprisingly meaningful, because they have structure that no tag yet encoded.
One could also try *adding* in order to emphasize a specific _X_.^[This is abstract enough that I doubt it could be easily explained to a user, but I think it could be encoded into the UI in a usable way. Like one could present a 2D plot, where each datapoint and averaged-tag is presented, and they can be clicked on to 'strengthen' or 'weaken' them (where each degree of strength corresponds to an embedding arithmetic operation, but weighted, like 10% per click). If you are interested in the _X_-ness of all your datapoints, you simply click on the _X_ dot to 'strengthen' it a few times until the updated 2D plot makes sense.]
[^negative-tags]: One idea I have not seen much, but which would be useful as automation is added, is a concept of 'negative tags' or 'anti-tags': asserting that an item is definitely *not* a tag.
Tags are typically presented as a two-valued binary variable, but because the default for tag systems is typically to be untagged, and because most tag systems are incomplete, that means that errors are highly skewed towards errors of omission rather than commission. An item with tag _x_ but not tag _y_, is almost always indeed an instance of _x_; however, it will often be _y_ too. So the absence of a tag is much less informative than the presence of a tag. But there is no way to distinguish between "this item is not tagged _y_ because no one has gotten around to it" and "because someone looked closely and it's definitely not _y_".
In regular tag use, this merely results in some wasted effort as users periodically look at the item and double-check that it's not _y_. With automation, this can be a serious obstacle to things like active learning working at all: if we have no way of marking 'it's *definitely* not _y_', then when we attempt to find instances of _y_ which are not labeled _y_, we will every time have to ignore the same false positives. (And we also can't train our classifier to ignore those false positives, even though those would be the most valuable to train on because they are the ones which most fooled our classifier.)
# Abandoned
[**See Gwern.net Design Graveyard.**](/design-graveyard "'Design Graveyard', Gwern 2010"){#gwern-designgraveyard-2 .include-annotation .backlink-not .redirect-from-id}
# Tools
Software tools & libraries used in the site as a whole:
- The source files are written in [Pandoc](https://pandoc.org/) [Markdown](!W) (Pandoc: John MacFarlane et al; GPL) (source files: Gwern Branwen, CC-0). The Pandoc Markdown uses a number of extensions; pipe tables are preferred for anything but the simplest tables; and I use [semantic linefeeds](https://rhodesmill.org/brandon/2012/one-sentence-per-line/) (also called ["semantic line breaks"](https://sembr.org/) or ["ventilated prose"](https://vanemden.wordpress.com/2009/01/01/ventilated-prose/)) formatting.
- math is written in [LaTeX](!W "LaTeX") which compiles to [MathML](!W), rendered statically by MathJax (Apache license) into HTML/CSS/fonts; copy-paste of the original math expression is handled by a JavaScript copy-paste listener
- syntax highlighting: we originally used [Pandoc's builtin](https://github.com/jgm/skylighting) [Kate](!W "Kate (text editor)")-derived themes, but most clashed with the overall appearance; after looking through all the existing themes, we took inspiration from [Pygments's](https://pygments.org/) [algol\_nu](https://xyproto.github.io/splash/docs/longer/algol_nu.html) (BSD) based on the original [ALGOL](https://en.wikipedia.org/wiki/ALGOL_60) report, and typeset it in the [IBM Plex](https://en.wikipedia.org/wiki/IBM_Plex) Mono font^[An unusual choice, as one does not associate IBM with font design excellence, but nevertheless, it was our choice after blind comparison of ~20 code fonts with [variant zeroes](!W "Slashed zero") (which we consider a requirement for code). An appealing newer alternative is [JetBrains Mono](https://www.jetbrains.com/lp/mono/) (which doesn't work as well with Gwern.net's style, but may suit other websites).]
- the site is compiled with the [Hakyll](https://github.com/jaspervdj/Hakyll/)v4+ static site generator, used to generate Gwern.net, written in [Haskell](!W "Haskell (programming language)") (Jasper Van der Jeugt et al; BSD); for the gory details, see [`hakyll.hs`](/static/build/app/hakyll.hs) which implements the compilation, RSS feed generation, & parsing of interwiki links as well. This just generates the basic website; I do many additional optimizations/tests before & after uploading, which is handled by [`sync.sh`](/static/build/sync.sh) (Gwern Branwen, CC-0)
My preferred method of use is to browse & edit locally using Emacs, and then distribute using Hakyll. The simplest way to use Hakyll is that you `cd` into your repository and `runghc hakyll.hs build` (with `hakyll.hs` having whatever options you like). Hakyll will build a static HTML/CSS hierarchy inside `_site/`; you can then do something like `firefox _static/index`. (Because HTML extensions are not specified in the interest of [cool URIs](https://www.w3.org/TR/cooluris/), you cannot use the Hakyll `watch` webserver as of January 2014.) Hakyll's main advantage for me is relatively straightforward integration with the Pandoc Markdown libraries; Hakyll is not that easy to use, and so I do not recommend use of Hakyll as a general static site generator unless one is already adept with Haskell.
- the CSS is borrowed from a motley of sources and has been heavily modified, but its origin was the [Hakyll homepage](https://jaspervdj.be/hakyll/) & [Gitit](https://github.com/jgm/gitit); for specifics, see [`default.css`](/static/css/default.css)
- Markdown syntax extensions:
- I implemented a Pandoc Markdown plugin for a custom syntax for interwiki links in Gitit, and then ported it to Hakyll (defined in `hakyll.hs`); it allows linking to the English Wikipedia (among others) with syntax like `[malefits](!Wiktionary)` or `[antonym of 'benefits'](!Wiktionary "Malefits")`. CC-0.
- inflation adjustment: [`Inflation.hs`](/static/build/Inflation.hs "'InflationAdjuster', Gwern 2019") provides a Pandoc Markdown plugin which allows automatic inflation adjusting of dollar amounts, presenting the nominal amount & a current real amount, with a syntax like `[$5]($1980)`.
- Book affiliate links are through an [Amazon Affiliates](!W) tag appended in the `hakyll.hs`
- image dimensions are looked up at compilation time & inserted into `popups.js', Achmiz 2019") for loading introductions/summaries/previews of all links when one mouses-over a link; reads annotations, which are manually written & automatically populated from many sources (Wikipedia, Pubmed, BioRxiv, Arxiv, hand-written...), with special handling of YouTube videos (Said Achmiz, Shawn Presser; MIT).
Note that 'links' here is interpreted broadly: almost everything can be 'popped up'. This includes links to sections (or div IDs) on the current or other pages, PDFs (often page-linked using the obscure but handy `#page=N` feature), source code files (which are syntax-highlighted by Pandoc), locally-mirrored web pages, footnotes/sidenotes, any such links within the popups themselves recursively...
- the floating footnotes are handled by the generalized tooltip popups (they were originally implemented via [`footnotes.js`](https://ignorethecode.net/blog/2010/04/20/footnotes/)); when the browser window is wide enough, the floating footnotes are instead replaced with marginal notes/*sidenotes*[^sidenotes-history] using a custom library, [`sidenotes.js`](/static/js/sidenotes.js) (Said Achmiz, MIT)
.](/doc/cs/css/sidenotes.png "Image of a webpage with 1-column layout but footnotes typeset in the left and right margins as 'sidenotes', near the text that they annotate."){.invert}
- image size: full-scale images (figures) can be clicked on to zoom into them with slideshow mode—useful for figures or graphs which do not comfortably fit into the narrow body—using another custom library, [`image-focus.js`](/static/js/image-focus.js) (Said Achmiz; GPL)
- error checking: problems such as broken links are checked in 3 phases:
- [`markdown-lint.sh`](/about#markdown-checker): writing time
- [`sync.sh`](/static/build/sync.sh): during compilation, sanity-checks file size & count; greps for broken interwikis; runs HTML tidy over pages to warn about invalid HTML; tests liveness & MIME types of various pages post-upload; checks for duplicates, read-only, banned filetypes, too large or uncompressed images, etc.
- [Link rot tools](/archiving "'Archiving URLs', Gwern 2011"){#archiving-2}: [`linkchecker`](https://github.com/linkchecker/linkchecker), [ArchiveBox](https://github.com/ArchiveBox/ArchiveBox), and [archiver-bot](https://hackage.haskell.org/package/archiver)
[^sidenotes-history]: Sidenotes have long been used as a typographic solution to densely-annotated texts such as the [Geneva Bible](!W "Geneva Bible#Format") ([first 2 pages](https://github.com/raphink/geneve_1564/releases/download/2015-07-08_01/geneve_1564.pdf)), but have not shown up much online yet.
 [_Historical and Critical Dictionary_](!W), demonstrating recursive footnotes/sidenotes (1737, volume 4, pg901; source: Google Books)](/doc/design/typography/sidenote/1737-bayle-dictionary-vol4-pg901.jpg "Screenshot of Google Books https://books.google.com/books?id=JmtXAAAAYAAJ&pg=PA900 , showing advanced typography in a single page which contains body text, footnotes, and (recursively) sidenotes to footnotes, of Pierre Bayle's famous Enlightenment text, the 'Historical and Critical Dictionary' (pg900 of volume 4 of the 1737 English edition).")
An early & inspiring use of margin/side notes.
## Implementation Details
> The Programmers’ Credo: "We do these things not because they are easy, but because we thought they were going to be easy."
>
> [Maciej Cegłowski](!W) ([2016-08-05](https://x.com/Pinboard/status/761656824202276864))
There are some tricks & details that web designers might find interesting.
Efficiency:
- **fonts**:
- Adobe [[Source]{.smallcaps} Serif](!W "Source Serif Pro")/[Sans](!W "Source Sans Pro") (originally Gwern.net used [Baskerville](https://en.wikipedia.org/wiki/Baskerville#Digital_versions))
Why use our own [webfonts](!W) instead of just using pre-existing web-safe/system fonts? One might ask if the font overhead (non-blocking download of ~0.5MB of fonts for the most complex pages like the [GPT-3 fiction page](/gpt-3 "‘GPT-3 Creative Fiction’, Gwern 2020"){.backlink-not}) is worth it, compared to trusting in fonts that may be installed already and are 'free' network-wise. This is what our webfonts buys us: