Changelog
Monthly chronological list of recent major writings/changes/additions to Gwern.net (see also the monthly newsletter).
This page is a changelog for Gwern.net: a monthly reverse chronological list of recent major writings/changes/additions.
Following my writing can be a little difficult because it is often so incremental. So every month, in addition to my regular subreddit submissions, I write up reasonably-interesting changes and send it out to the mailing list in addition to a compilation of links & reviews (archives).
For a feed of recently-added links & references, see the “newest links” page.
2024
November 2024
October 2024
-
Gwern.net: new Halloween dropcaps
September 2024
-
Idea: error-ensuring codes for synthetic media watermarking; hybrid LLMs: conditioning AR LLMs on diffused embeddings
-
Why do cats bite spontaneously or during petting? Maybe you accidentally look like prey
-
text2epositive.py
: experimental GPT-4 script for rewriting English statements from negation to affirmatives -
Gwern.net: revised index page & header (screenshot); enabled help page & popup window manager keybindings
August 2024
-
It’s still a good time to write online, but also Why To Not Write A Book; and how I write
-
East Asian cat/earwax survey (“yes”)
-
Gwern.net:
date-guesser.py
: a LLM script to try to extract dates from arbitrary text input;daterange-checker.py
; towards a useful RSS feed
July 2024
-
Gwern.net: experimental date-range subscripts; archive system default reverted back to local archives now that JS hides rewrites to original URL for copy-paste/clicking; 404 page now offers guesses based on Levenshtein distance of broken URL to other URLs in the sitemap; image thumbnail optimization for popups/mobile
June 2024
May 2024
April 2024
-
Gwern.net: authors are now hyperlinked; Google CSE site-search has been restored (magnifying-glass icon in theme togglebar,
/
keybinding); April Fools easter egg
March 2024
-
InvertOrNot.com, Mattis Megevand: a NN-powered API for whether to invert images in a website dark-mode
-
Gwern.net: use of InvertOrNot.com in dark-mode; correct date-sorting on “newest links”; links created/modified in past 2 months now get special black-star symbol to highlight novelty ( = ‘changed’); large tables now have sticky headers & scroll-in-place; adblock PSA
February 2024
-
Gwern.net: new file-transclusion feature—browse images/videos/documents in the annotations & tag-directories (eg. Midjourney, DALL·E 3); +video thumbnails; new ‘GTX’ text format to replace YAML; GPT-4-V demos:
image-margin-checker.py
,invertornot.py
January 2024
2023
December 2023
-
N/A
November 2023
October 2023
-
Idea: make an ‘InvertOrNot.com’ (a simple, useful, but missing web API)
-
Idea: lobby webserial/comic sites to make more use of ‘prefetch’/‘preload’ for faster reading
-
Gwern.net: dropcaps: extended to support bitmap graphics, and randomized selection (used in Halloween & Christmas-mode, and for the new ‘dropcats’ dropcap set we made for cat-related pages); improved Halloween & Christmas-mode with generated logos; scrolling now disables popups temporarily
September 2023
August 2023
-
indent/justification A/B test finished (usual too-small-to-measure)
-
Gwern.net: widescreen-desktop floating section ToC/scrollspy
July 2023
June 2023
-
On The Impossibility of Knowledge of Retrocognitive Knowledge
-
Gwern.net: ‘sort by magic’ of similar-links & tags using embeddings (with auto-tagging); +interview formatting support; margin note aggregation at beginning of sections;
latex2unicode.py
May 2023
-
Idea: DRL free-play
-
Gwern.net: ‘demo mode’: show readers a label or feature n times, and then stop (primary use: animate the theme toggle collapsing, and label collapsed blocks, so they learn what those are, and then hide those); +inline-collapses
April 2023
March 2023
-
Design: on the popup systems on the link-icons; on the local archives; outbound link-tracking post-mortem
-
Trying out Japanese tanka poetry with GPT-3 (
davinci-002
feels almost as creative asdavinci
but more tractable, while not being creatively useless like ChatGPT-3) -
Gwern.net: header-based redesign of desktop theme; within-section backlinks; backlinks can now display original context
February 2023
January 2023
-
Gwern.net:
-
moved domains; image/doc hierarchy merged
-
new Quote/Site/Annotation X-of-the-Day feature
-
2022
December 2022
-
N/A
November 2022
-
Mirror: ISIS propaganda magazine Rumiyah
-
Gwern.net: print mode CSS: looks much better, pages should print reasonably now (eg. icons aren’t screwed up, transclusions expand so nothing is lost, layout looks like regular pages more)
Minor Gwern.net changes since August 2022 not covered previously:
-
transcluded link-bibliographies, similar-links, and backlinks: these are now transcluded at the bottom of the page so you can “just keep reading”.
The links at the top of pages/annotations will continue to popup in desktop mode (as expected), but in mobile in popins, they instead jump down to the transcluded version (as opposed to creating a new popin for what might be a disappointment). This should feel pretty natural and friction-free.
-
similar-links/backlinks transclude by default at the end of annotations as well, similarly enabling ‘just keep reading’ and jumping-to-anchor behavior
-
-
‘partial’ link annotations: links which don’t have a true annotation but do have some metadata like tags or backlinks will not be marked as annotated links, but they will still pop up a bare-bones annotation showing what is known about that link. This avoids promising the reader too much; if you hover over a link, you know not to expect too much, and maybe what there is will be useful.
-
backlinks: now display the context of the reverse citation in the original page (like their popup but displayed b default via transclusion)
-
collapses: uncollapse on hover in popups/tag-directories
-
popup speed optimizations
-
WP annotations: lots of small style/bug-fixes to various special-cases like big inline images or mismatched curly quotes (due to a truly ancient bug, turns out, so old that it’s due to JavaScript changing how
for-in
works (!)) or thumbnails, removing admonitions & ‘list of X’ sidebar infoboxes -
similar-links embeds with more metadata like backlinks, and filters out several kinds of unhelpful links now; these should increase relevance; backlinks also filter out more
-
transclude templating system
-
a justification+hyphenation A/B test has begun to measure if the book-like indentation has measurably harmful effects; if it does, guess we’ll revert to the more Internet-standard approach of no-indentation+newline.
-
similar-links/backlinks: automatically put into multi-columns if an improved heuristic says to (lots of previous multi-column lists were incorrect and shouldn’t’ve been in columns because the entries were too few and/or text too long)
-
JS works around flaws in backlinks analysis to guess the intended anchor
-
-
remove a lot of compile-only or unused classes/IDs from the HTML to simplify it
-
Windows: force Source Sans use so ToCs look a lot better (the default font stack, while well-intentioned, looked bad)
-
lots of renaming
-
obviously, tons of minor bug fixes, added link icons, added link-live domains, tests and checks (eg. a PHP script for dumping all defined HTML classes from the final compiled HTML was quite helpful in linting and finding a number of misspelled or outdated classes in content pages), in addition to the usual daily drumbeat of writing annotations & adding links…
-
October 2022
-
Dynamic programming solutions to “52 Cards Win a Dollar” puzzle
-
Gwern.net: special occasions mode: Halloween (rubrication)
September 2022
August 2022
-
Gwern.net:
-
new dynamic lazy transclusions:
transclude.js
-
sidenotes/margin notes: margin notes now appear exclusively in the left column and sidenotes in the right column, to keep visual simplicity and emphasis the left-to-right summary → detail hierarchical reading
-
July 2022
-
N/A
June 2022
-
Gwern.net: ‘partial’ annotation popups (enhancement of live-links)
May 2022
-
Gwern.net: popup system rewrite (faster & rendering now independent of page length using shadow DOM)
April 2022
-
Gwern.net: switched full cross-page live popups back to scraped thumbnail+abstracts+ToCs for performance; unified annotation tags & legacy essay tag system; removed auto-smallcaps, Disqus; experiment with commissioned art for Batman story
March 2022
-
It Looks Like You’re Trying To Take Over The World (with use of new “reader-mode” eg.)
-
Questions: Why does microwave tea taste bad?; Indo-Chinese numerical rhetoric
-
Gwern.net: new reader-mode (hides most of UI until hover/Alt-key or mode disabled, for text-centric reading; control: )
February 2022
-
Gwern.net: backlinks & similar-links now recursively pop up; experimental Ar5iv use; wrote GPT-3-using
paragraphizer.py
tool & used to add line-breaks to ~1k annotations; link icon system rewrite (faster, better+test-suite, >220 new link icons); live link popup system rewrite (similarly, >850 domains to popup);
January 2022
2021
December 2021
-
Gwern.net: GPT-3-powered “similar links” feature added to popups/indexes
November 2021
October 2021
-
compiled 40 subject-area tags
September 2021
-
Gwern.net: link bibliographies re-enabled; new server
August 2021
July 2021
-
Gwern.net: “link tags” added
June 2021
-
Refactored 60 pages
-
LinkAuto.hs: a Pandoc library for automatically turning user-defined regexp-matching strings into links (discussion)
May 2021
April 2021
March 2021
-
Gwern.net: mobile “popins” are finally enabled! (example); new Wikipedia popups (this 7th implementation enables recursive WP popups)
February 2021
-
Gwern.net: popups: can now be moved, stickied, and full-screened (another step towards our ambition of Windows-95-in-the-browser!)
January 2021
-
Danbooru2020: A Large-Scale Crowdsourced & Tagged Anime Illustration Dataset
-
Gwern.net: +return-to-top floating button; popups: can now be disabled (use the ‘gear’ icon); final reimplementation (dynamic JS now; memoizing the recursive inlining, however clever & elegant, turns out to have painful edge-cases & still not be efficient enough—web browsers really don’t like loading hundreds of kilobytes of extra HTML)
2020
December 2020
November 2020
-
Gwern.net: dark-mode rewrite complete (fixes page load flash & laggy scrolling) arabesque navigation bar in footer with JS keyboard shortcuts; IBM Plex Mono font & custom ALGOL-like syntax highlighting for code blocks; new sun/moon icons for horizontal rulers; images in Wikipedia popups; new internal link/citation convention for multiple citations
October 2020
-
more Gwern.net fixes of infelicities & outdated infrastructure: major ones: line-breaking on slashes, hyphenation & justified text on Chrome desktop browsers using Edward Kmett’s
hyphenation
at compile-time1; width-full mobile images; rewrote GoodReads conversion for much more readable book review page; browsable directories; popup annotations now render more like body.Dark mode users should have noticed it’s improved considerably; most of the other fixes either are more subtle (consistency, correct formatting, edge-cases, bold rather than italics), or rewriting old hacks to pay off technical debt. In general, it should just look nicer and more pleasant, and be a little easier for me to write.
Minor ones:
-
I wrote up things we’ve tried & abandoned
-
rewrote GoodReads conversion for much more readable book review page; began compiling all movie/TV/opera reviews into a single page; began copying all MAL anime/manga reviews & localizing into a single page; all GR reviews have been moved to Gwern.net permanently
-
dark mode:
-
popup whiteness fixed
-
image handling fixed (images are inverted & colorspace-rotated based on automated heuristic + manual tagging); tweaks like highlighting to make inline code blocks readable
-
brightness decreased to enable a pure black background without the white foreground being blinding
-
-
new test/demo page: revealed a number of minor bugs, makes it much easier to test changes for correctness
-
popup annotations:
-
popup annotations now render more like body: they have link icons, paragraphs, properly indented lists, multi-column support etc
-
annotations (titles/authors/abstracts) are auto-smallcaps like the body, making them more consistent and reducing the annotation toil in adding smallcaps spans
-
automatic annotations get some rewrite rules like ‘N=’ → ‘n =’
-
annotations can now be ‘definitions’ (popup annotations for non-link text)
-
popups can now be arbitrary IDs, not just sections or annotations (same as the reverse-footnote popups)
-
links pop up with a ‘context’, to support my new citation convention: every citation is either a fulltext hyperlink with annotation, or a link back to another use of that citation with more context/discussion. (This exploits the new automatic ID generation so a link like
[Foo et al 2020](/doc/ai/2020-foo.pdf)
gets an HTML ID like#foo-et-al-2020
, making it extremely easy to link elsewhere likefor more details on the scaling laws, see the earlier discussion of scaling papers like [Kaplan et al 2020](#kaplan-et-al-2020)
.)
-
-
tooltips are shorted versions of link annotations, as a fallback for non-JS or mobile users
-
bolded abstracts (manual & automatically)
-
Arxiv abstracts now parsed as LaTeX first (fixing a lot of weirdness); wrong publication dates bug fixed & reported upstream (upstream refuses to fix, so forked)
-
PLOS/PubMed/Wikipedia abstracts formatting greatly improved (and fixed upstream)
-
MedRxiv support
-
<video>
tag support -
removed broken reverse search link
-
better author truncation
-
author initials get periods (“John H Smith” → “John H. Smith”; surprisingly common in a lot of bibliographic metadata)
-
rebuilt all Wikipedia & Arxiv link annotations to bring up to date and with new formatting
-
Wikipedia annotations now include image thumbnails, which get auto-inverted like regular images for dark mode viewers
-
-
automatic annotations stored as YAML; more consistent, simpler implementation, and makes a lot easier to copy automatic annotations into the manual annotations for improvement
-
annotations can now be selected/copy-pasted (this was disabled previously, for unclear reasons)
-
added columns
-
-
refactored the site infrastructure into a separate repo
-
mobile:
-
simplified mobile appearance, reduced italics over-usage
-
width-full (edge-to-edge) mobile images (uses ~20% more horizontal space)
-
-
typography: rewrite pass to insert
<wbr>
to enable line-breaking on slashes; rewrite pass to insert soft hyphens for hyphenation & justified text on Chrome desktop browsers using Edward Kmett’shyphenation
-
fixed Safari double hyphen bug, reported upstream
-
-
revised list hierarchy to use bold (ie. bold → smallcaps → italics; smallcaps → italic → bold or smallcaps → bold → italic made no visual-emphasis sense, and I think it resulted in smallcaps overuse); switched ordered lists to uppercase Roman numerals to avoid confusion with lowercase alphabetic numerals; added a third-level icon, and made ordered/unordered lists & blockquotes cycle through 3 styles based on depth
-
‘previous’/‘top of page’/‘next’ navigation links added to all pages beneath the footnotes
-
exploiting that new metadata about page sequencing, a JS widget will automatically load the next/previous page if you are at the bottom/top of a page & hit a down/up key twice within 0.5s
-
-
simplified sidenotes appearance (removed double-lining, merged the number box with the main box)
-
auto-smallcaps: catches more acronyms/initialisms, and rewritten to avoid deeply nested duplicate spans (an unfortunate consequence of substitution + Pandoc AST walking, requiring a switch to bruteforce string substitution); headers no longer smallcaps (which looked weird)
-
metadata blocks are compiled to HTML; before, they were inlined as raw Markdown, which was unfortunate, but tricky to fix
-
index is now a generated page rather than a pile of manually-edited HTML, so it will no longer be perpetually out of sync in style/content
-
custom asterism based on the list icons instead of horizontal ruler
-
inflation adjusters: tooltips were wrong; Bitcoin adjuster now interpolates or extrapolates for missing exchange-rate dates instead of crashing
-
browsable directories on Gwern.net & TWDNE (eg. /doc/)
-
changed all dates to YYYY-MM-DD for consistency
-
cut down wasted whitespace margin in lists/blockquotes (which looked especially ridiculous on mobile), and headers
-
added missing link icons for 10+ domains, fixed spacing & overlapping, and harmonized opacity of all link icons; fixed Substack background, OpenAI size
-
Pandoc now uses Mathjax directly, enabling colored equations
-
collapse sections have ellipses
-
‘abstract’ class generalized, so pages can have multiple abstracts, simplifying formatting (the manual dropcaps+smallcaps are then redundant)
-
internal links to a previous or later link ID (usually a section) now have an arrow icon pointing up or down (respectively) as a navigation aid
-
experimental use of instant.page prefetching
-
tag pages now in columns
-
improved admonition icons (ugly or didn’t show up in dark mode, or at all on Android) & spacing
-
the
<code>
syntax highlighting (outlined box) looks better inside links -
removed dead CSS & code
-
shortened page descriptions & titles
-
updated last-modified metadata on most pages
-
checked all pages for MIME types and status, revealing a number of infinite redirect errors
-
WWW archive: no crawl headers; force UTF-8 charset header for Markdown sources (for Chrome)
-
all Twitter.com → Nitter (static, more reliable)
-
converted simple LaTeX formulas to HTML where possible; added spaces to various equalities/inequalities
-
fixed blurriness on highlighting/selecting text
-
margin notes are now selectable/clickable (z-index error)
-
set up /r/mlscaling as a more convenient way to browse “Matters of Scale” topic links
Known outstanding bugs:
-
no popup annotations for mobile
-
some users report popups ‘cycle’ rapidly in and out of existence; we have been unable to replicate this and don’t know why it happens
-
dark mode is slow, because browsers implement filters like inversion very slowly. Sorry.
-
dark mode is slow to load, yielding a ‘flash of white’
Injecting CSS doesn’t work because browsers seem to ignore it until after the page loads, with the exception of setting a class, which however breaks all the existing dark mode CSS.
-
margin notes overlap with sidenotes, and sidenotes may be overlap as well (desktop)
-
sidenotes are laid out even when they’re in collapsed sections
-
popup annotations may not correctly display LaTeX/math
-
-
compiled all movie & anime reviews
September 2020
-
Gwern.net: dark mode image handling fixed (images are inverted based on automated heuristic + manual tagging); annotations can now be ‘definitions’ (popup annotations for non-link text); expanded tooltips as fallback for link annotations; bolded abstracts & revised list hierarchy to use bold; changed all dates to YYYY-MM-DD for consistency; added missing link icons for 7 domains, fixed spacing & overlapping, and harmonized opacity of all link icons; Pandoc now uses MathJax directly, enabling colored equations; internal links to a previous or later section now point up or down (respectively) as a navigation aid; experimental use of instant.page prefetching; asterism instead of horizontal ruler; simplified mobile appearance; many miscellaneous bug fixes
August 2020
July 2020
-
TWDNEv3.5: GPT-3 text-snippets upgrade
June 2020
May 2020
-
Ganbooru prototype: 256px BigGAN trained on Danbooru2019; Danbooru2019 Figures dataset released
-
Gwern.net:
-
The newsletter moved to Substack due to reaching the TinyLetter 5000-subscriber limit
-
popups.js
: +support for reverse-footnote popups
-
April 2020
March 2020
-
Gwern.net: dark mode theme switcher; relocated from S3 to Nginx, please report any server bugs like inaccessible pages
February 2020
-
Gwern.net:
-
added features:
-
new local link archive feature (uses SingleFile)
-
“admonitions” & multi-column lists (Markdeep-inspired);
-
auto-smallcaps plugin
-
-
removed features:
<q>
quote syntax highlighting due to maintenance/annotation burden; removed PNG preview popups due to low screenshot quality (the Web is why we can’t have nice things) & many reader complaints
-
January 2020
-
Danbooru2019: A Large-Scale Crowdsourced & Tagged Anime Illustration Dataset
-
This Waifu Does Not Existv3: 100k StyleGAN 2 anime portrait samples
2019
December 2019
-
On Correlation & Causality in the Social Sciences (see also my Replication Crisis bibliography)
-
Gwern.net: Tufte-CSS margin notes support (glossing experiment—demonstrations in Folk-Music/Hydrocephalus/GoodReads/Internet Search Tips)
November 2019
-
Gwern.net: reverse citation links now available in popups via Jose Luis Ricon’s new Semantic Scholar search engine (which prioritizes reviews/meta-analyses which cite a given paper)
September 2019
-
On Selective Emigration and Personality Trait Change in Scandinavia
-
Gwern.net: added Bitcoin support to
Inflation.hs
; CSS optimizations to cut mobile load time by half; width-full image support
August 2019
-
Gwern.net:
-
popups.js
: a new JS library which reads link annotations and displays them in a popup (eg. WP displays title/summary, and papers display title/author/date/abstract); works on mobile; generalizes & obsoleteswikipedia-popups.js
:popups.js
reads static annotations, removing the runtime spam, and importantly, annotations are now cached at compile-time, so additional sources can be included. It currently provides link annotations for Arxiv, bioRxiv, Pubmed Central, Crossref, Gwern.net, and hand-written annotations, in-popup YouTube video support, with a fallback to Chrome/Ghostscript-generated screenshot previews for all other URLs -
added more custom-SVG/text link icons for link icons (custom: arXiv/BioRxiv (unofficial), Google Scholar, Pubmed, Internet Archive, Guardian, NYT, New Yorker, Washington Post, DeepMind, OpenAI, MIRI, Erowid)
-
July 2019
-
Gwern.net: tooltip popups for Wikipedia article summaries (
wikipedia-popups.js
)
June 2019
May 2019
-
TWDNEv2: upgraded anime faces to ‘portrait’ samples (bigger crops of faces, showing ears/hats etc) & some 345M plot summaries; GPT-2 poetry: upgraded to 345M (plus new “nucleus sampling” generation)
April 2019
-
Gwern.net: deprecated RSS (replaced by the newsletter); added “dark mode” support for Safari/FF (inverts colors for night reading on browsers supporting new CSS media query)
March 2019
-
Finetuning the GPT-2-117M Transformer for English Poetry Generation
-
Glue Robbers: Sequencing Nobelists Using Collectible Letters
-
rewrote “The Gift of the Amygdali”
-
Gwern.net CSS changes: Tufte-style sidenotes (for displaying footnotes in margins on wide screens) via Said Achmiz’s new JS library,
sidenotes.js
; static compilation of MathJax’s rendering of MathML math viamathjax-node-page
(enables math for JS-disabled users & eliminates the <5s rendering time on math-heavy pages like the embryo selection page); added support for dropcaps (Goudy Initialen, yinit, Deutsche Zierschrift; Cheshire; Kanzlei; other examples) for thematic emphasis; disabled Disqus ads; second ad A/B test ended, so disabled banner ad; prototype automatic inflation-adjustment of dollar amounts via new Hakyll plugin; outdent headers & left shift H1 headers for scannability; more Tufte-style tables with different styling for small/simple tables vs complex width-full tables; better highlighting on ToC;image-focus.js
now runs in slideshow mode; Wikipedia link annotations; resized & optimized images with advpng/mozjpeg
February 2019
-
Gwern.net CSS/HTML/JS changes: click-to-zoom images (using
image-focus.js
); headers are now self-links; Tufte CSS-style epigraph support; Table of Contents: Wikipedia-style section numbering, margin & size tweaks, lightweight subset of Source Sans Pro (for Mac users); nicer diamond list icons; sleeker sidebar (especially nice on mobile); PDF/internal/section links are now annotated with icons; borders on tables, image figures, and blockquotes; old-style numerals in text & tabular numerals in tables; justified text (but not in Chrome due to decade-old lack of hyphenation); narrowed maximum body-width in characters & made line-height responsive to body-width (hopefully addresses the perennial complaints that pages are always too wide/too narrow/lines too close); quote highlighting disabled by default; collapsible code-blocks; inline smallcaps support; optimized SVG logo & favicon; page-specific CSS overrides enabled; list paragraph bugs in Pandoc fixed; compressed JPEGs; changed code syntax-highlighting scheme to match overall esthetics better; miscellaneous responsive design/mobile improvements-
image-focus.js
(JS): release of new, correct, lightweight, dependency-free JS library written by Said Achmiz for implementing “click to zoom” on images (useful for large images/graphs)
-
January 2019
-
Danbooru2018 released: a dataset of 3.33m anime images (2.5tb) with 92.7m descriptive tags
-
Gwern.net CSS/JS changes courtesy of Said Achmiz: implemented collapsible sections for hiding digressions/especially large sections, for more readable pages; more varied headers to convey the semantic hierarchy; added use of a lightweight subset of Source Serif Pro, as a better Baskerville font with real small caps; improved font selection on Macs; minimized MathJax library use, for faster loading
-
The Tragedy of Grand Admiral Thrawn (why he had to be assassinated at the end of the Thrawn trilogy)
2018
December 2018
-
Internet Search Tips: effective use of Google/Google Scholar/Libgen & other resources for papers/books/pages (I’m also offering bounties for papers/books I haven’t been able to get)
November 2018
-
Cat Sense, Bradshaw 201311ya: Are We Good Owners? (on cat psychology, genetics, domestication, & dysgenics)
-
Embryo selection: FAQ, multi-stage selection, chromosome/gamete selection, optimal search of batches, & robustness to error in utility weights
-
Urban Area Cost-of-Living as Big Tech Moats & Employee Golden Handcuffs (speculations on Bay Area real estate & SV’s future)
October 2018
September 2018
June 2018
May 2018
-
ZMA sleep self-experiment (inconclusive but suggestive of benefits)
-
Bacopa quasi-experiment (no correlates)
April 2018
March 2018
-
moved newsletter from MailChimp to TinyLetter due to exorbitant MailChimp fees
February 2018
-
Danbooru2017: a new dataset of 2.94m anime images (1.9tb) with 77.5m descriptive tags
-
tweaked Gwern.net SVG logo courtesy of Kezarin Brando
January 2018
2017
November 2017
October 2017
September 2017
-
Gwern.net updates: Google AdSense banner ads removed (due to initial analysis of A/B test results); URL scheme changed to replace spaces by hyphens & delete commas/apostrophes (due to persistent user error); footer moved to sidebar; re-tagging pages; font/CSS tweaked for faster loads & wider margins; hosted documents reorganized & expanded with personal archives; began buying & scanning all cited books to provide fulltexts; added sitemap generation to expose fulltexts to search engines
July 2017
June 2017
April 2017
March 2017
-
Gwern.net: HTTPS now mandatory; HTML sections rewritten using HTML5 semantic markup for hopefully better compatibility with screen readers & offline browsers like Pocket/ReadItLater; “importance” metadata added to all pages to rank them; “belief” metadata renamed more intuitively as “confidence”; finish adding 301 Redirects for all broken links & common typos; renamed & all darknet market pages for current terminology; rewrote Patreon profile
February 2017
January 2017
2016
December 2016
-
compiled my epigrams
-
Gwern.net: CSS, logo, and index page redesigns; redirects for broken incoming URLs; scripts & YAML updates for upstream Pandoc/Hakyll changes
November 2016
-
switched Gwern.net to HTTPS
October 2016
September 2016
-
frequency of catnip response in cats: a meta-analysis (and launched a survey to expand the dataset)
August 2016
July 2016
June 2016
May 2016
-
Wikipedia article on Genome-wide complex trait analysis (GCTA)
March 2016
February 2016
January 2016
2015
December 2015
October 2015
September 2015
-
modafinil user survey (ran 2015-09-26–1m2015-10-26)
-
Value of Information for suicide (example cost-benefit analysis of weakly predicting suicide)
July 2015
-
A/B tests: finished metadata test; proposal towards recurrent neural network for reinforcement learning of CSS
-
Air conditioner upgrade cost-benefit analysis
-
Gwern.net:
-
switched to Patreon for donations
-
optimized website loading (removed CSE, A/B testing, non-validating XML, outbound link-tracking; simplified Disqus; minified JS, and fully async/deferred JS loading)
-
June 2015
-
rewrote Gwern.net CSS to be mobile-friendly; should now be readable in an iPhone 6 browser
-
wrote two summer poems, on earthworms and the rain
May 2015
-
DNM arrests compilation finished & summaries calculated
-
analysis of Effective Altruists’ donations as reported in the LW survey
March 2015
-
LLLT self-experiment finished: no effect (so earlier correlative results were grossly overestimated; I’m starting to expect this from anything non-randomized…)
-
an ICE subpoena on my Reddit account caused me a good deal of trouble
-
playing with inferring Bayesian networks for my Zeo & body weight data (powerful generalization of SEMs, but requires a lot of data before networks stabilize)
February 2015
-
DNM arrests: split out of Silk Road page & began bringing up-to-date with all known incidents across all markets
-
electric vs stove kettle boiling-time analysis: collected some simple data on my kettles & demonstrated some statistics tools on the dataset like a Bayesian measurement-error model
January 2015
-
A/B tests: no positive results from sidebar test; set up in-page metadata format
2014
December 2014
Nothing finished.
November 2014
-
A/B indentation test: no real result, defaulted to 2em
October 2014
QS:
-
Redshift self-experiment: screen-reddening software shifts bedtime forward by 20 minutes
-
LLLT re-analysis: no change in sleep as hypothesized by another LLLT user
-
analysis of sceaduwe’s spirulina/allergies self-experiment (no reduction in allergies)
September 2014
QS:
-
Noopept experiment (no benefits)
-
Treadmill spaced repetition experiment: expanded analysis to cover treadmill’s impact on successive reviews with SEM (no additional damage to recall beyond that implied by the original damage)
Stats:
-
anthology on how everything is correlated
-
probability/gerontology problem: can one visit 566 centenarians before any die? No.
Technology:
-
My window tracker/time-logger of choice is arbtt which records X window info for later classification and analysis; but one of the challenges is you don’t know how to set up arbtt or improve your environment or write classifications rules. So I wrote a tutorial.
-
Time-lock crypto: wrote a Bash implementation of serial hashing time-lock crypto, link to all known implementations of hash time-lock crypto
August 2014
-
Nootropics: initial results on LLLT, correlates with large increases; began followup randomized experiment
-
purchased a North Paw compass belt
July 2014
QS/self-experiments:
-
lithium orotate experiment finished: no effects positive or negative
-
magnesium citrate experiment finished: initial benefits but apparent cumulative overdose led to net negative effect and mixed effects on sleep
-
sleep correlations:
-
alcohol: no harm
-
optimal bedtime: a little earlier than usual
-
optimal wakeup time: a little earlier than usual
-
Statistics:
-
do causal networks explain why correlation ≠ causation is so often true?
-
a little example of estimating scores from censored data
Debunking:
Misc:
Site:
-
finished floating footnotes test: no apparent harm, as hoped
-
began paragraph indentation test (responding to anonymous complaint)
June 2014
-
finished & analyzed caffeine-pill wakeup pilot trial, began blinded self-experiment
-
BeeLine Reader A/B test finished: no version performed statistically-significantly better
-
compiled & expanded selection of my poems
May 2014
-
short interview with Mike Powers on current darknet markets
April 2014
-
an anonymous user sent in a fanfiction about Satoshi Nakamoto, which was too good to simply delete
-
I have posted the past issues of the newsletter online:
March 2014
Bitcoin:
Tech:
-
BeeLine Reader A/B test
February 2014
-
Radiance: finished.
-
Public release of the Mnemosyne spaced repetition dataset (18GB of 121.2m flashcard reviews, collected ~2004–102014)
-
verse: obituary for my dog; a poem apropos the end of Genshiken Nidaime
-
A/B testing: header capitalization finished (result: upcase title & all section headers); began table of contents placement
January 2014
Darknet markets:
-
I have begun systematically spidering all operational English darknet markets
General:
-
Haskell Summer of Code: 2013 review
Statistical:
-
font/number size & table of contents background AB test finished, with consideration of value of interactions
Radiance:
-
diff of “Radiance” and part I of Radiance
Site:
2013
December 2013
Darknet markets:
-
compiled an updated table of all known darknet markets with lifetimes
-
wrote up an essay on 3 attempts to blackmail/extort/scam
Statistical:
-
analyzed a self-experiment about low level laser therapy improving reaction time
-
power simulation of the penalty from omitting key covariates in A/B testing
Radiance:
-
transcribed novella “Radiance”
-
transcribed & annotated the autobiographical essay Old Legends by Gregory Benford on his physics career, SF & science, the “Star Wars” program, Edward Teller, etc
-
tracked down and scanned a copy of The Astounding Investigation: The Manhattan Project’s Confrontation with Science Fiction (Berger 198440ya, Analog)
Personal:
Site:
-
signed up for MailChimp & started a monthly mailing list for Gwern.net updates
-
my Gratipay is up to $9.97$7.352013 a week. I thank all donators.
November 2013
Another busy month:
-
DNB meta-analysis: corrected multiple mistakes based on Redick’s review
-
Lunar cycle effect on sleep: split out, acquired 2 other Zeo users’ data, expanded into a multi-level model & power analysis (Cajochen still has not responded)
-
Modafinil: price table update
-
Silk Road:
-
estimating DPR’s net fortune based on the FBI numbers
-
Darknet market survival analysis
-
publicly betting Sheep & BMR to shut down soon (which was vindicated even quicker than expected)
-
Mike Power email interview
-
BBC Radio 5 interview (not yet broadcast)
-
doxed the owner of Sheep Marketplace (see https://pastebin.com/raw.php?i=9spTATw6 & /doc/darknet-market/sheep-marketplace/2013-11-03-sheepmarketplace-doxxing.maff)
-
-
Bitcoin:
-
short essay on Zerocoin prospects
-
bets: update on bet with qwertyoruiop ₿<$67.79$502013—conceded defeat, learned a lesson about panicking, and paid up; altogether admirable
-
-
Radiance: finished transcription & added all notations
-
AB testing: blockquote finished, status quo performed best
-
Site:
-
deleted Flattr, trying out Gratipay for donations
October 2013
My focus for October was coping with the fallout from the bust of Silk Road—dealing with the revelations, copying the SR forums, tracking down leads, talking to various people, recording the ensuing arrests, tracking the darknet markets popping up in its wake… I still have much material to work through, but some things I managed to do included:
-
Drugs 2.0: “Your Crack’s in the Post” (book chapter)
-
rescuing the vendor public profiles and using some in my page
-
betting all and sundry that BlackMarket Reloaded & Sheep Marketplace will be busted or shut down within a year (no takers)
-
researching the digitalink” bust
I decided to post my most extensive self-experiment yet, on LSD microdosing. While there was a lot of criticism, I still regard it as worthwhile and setting a new benchmark for any future research in that area.
My anti-linkrot system benefited from comments on Hacker News telling me how to use archive.today; this may help me out quite a bit in the future.
A/B testing has been active since Hacker News traffic furnished large sample sizes:
-
foreground/background test: pure black/white was best
-
font size 100–120%: default 100% was best
-
blockquote formatting testing zebra striping: started but not yet finished
September 2013
-
compiled a small meta-analysis of creatine’s effect on intelligence
-
combined previous progress reports all into a single page (this page) & added it to the sidebar. Should help out everyone who keep asking for an RSS feed of major changes rather than every change.
-
started a new A/B test on foreground/background colors
-
expanded the Google Alerts over time analysis with fresh emails from the past half-year, for its Hacker News submission
-
started a bigger Noopept self-experiment
-
did some spaced repetition research using the Mnemosyne logs: found weekly & time of day effects on memory performance—with a clear circadian rhythm
August 2013
-
A/B testing: the line-height test found no difference, so I did a quick one where I tested an empty test to check the A/B testing tool I’m using; I successfully failed to reject the null. The next test is whether underlining hyperlinks annoys people or not.
-
Book reviews: I wrote a Haskell program to parse my GoodReads ratings & reviews into flat Pandoc Markdown; it works somewhat well, it seems to be eating blockquotes & neutering hyperlinks, I’m not sure why. Was also an opportunity to clean up some reviews: inline some of them, spellcheck, expand references & links, which was a lot of work. But it’s nice to have my reviews gathered somewhere with a readable interface. kiba thinks the work may pay for itself in affiliate revenue with Amazon, but I’m skeptical.
-
Scholz’s Radiance: Added a hundred pages or so. Annotating some of it is quite difficult; Scholz’s familiarity with Wagner’s operas is a challenge, since I’ve only ever read his Ring Cycle.
-
I’ve started two new self-experiments:
-
Magnesium citrate on mood/productivity
-
Treadmill desk usage on spaced repetition scores
-
-
Touhou music growth rate: made a little more progress on Touhou music, with an analysis # of releases vs time: seems like we may’ve reached peak Touhou in 200915ya.
-
Silk Road mirrors: I’ve started hosting public copies of subsets of the darknet markets; these are backups for particular incidents or timeseries
-
Spaced repetition statistics: I’ve been analyzing my Mnemosyne data and the giant public database for time of day effects. While my results aren’t conclusive, my analysis of 48m flashcard reviews from the public database finds that the best time to study your flashcards seem to be noon. A little surprising, you’d think that late at night, before bedtime, would be the best time.
-
My forgotten cleaning methods like Sand polls aren’t done because I got a much lower rate of responses to the polls than I was hoping for and only got enough responses in the final poll the other day.
July 2013
Interesting things I wrote during July:
-
Google Alerts: Statistical analysis of all my emails from Google Alerts to see whether/when they started to be less useful.
-
2013 Lewis meditation quasi-experiment: A Quantified Selfer and a few other guys did some meditation while doing an arithmetic game; turned out to be a perfect application for multilevel modeling
-
Sleep and lunar phases: A recent paper claimed that there’s a phase-of-the-moon effect on circadian rhythms; since I have so much sleep data on myself, I thought I’d see if there’s any effect…
-
Sand: continues to progress; I closed the LW poll and set up 3 new polls on Gwern.net to test problems with the original poll.
-
Betting Made a list of things I’ve bet on or at least tried to bet on with people (as opposed to prediction market use). Disappointingly short.
-
Scholz’s Radiance: I’ve started transcribing and annotating one of my favorite tech/lit novels. It’s mostly done.
-
Cicadas for dinner: I finally got around to eating the cicadas I caught during the most recent Maryland emergence; so of course I had to write up this outré dining.
Right now, I’m listening through my Reitaisai 10 downloads (more Touhou music work); and working with this coach who is interested in predicting triple-jump performance by college athletes, and has collected a bunch of data about triple-jumpers.
June 2013
Hm, what did I get done this June…
It was a little boring, honestly; jury duty was a mental distraction where I couldn’t plan to do anything but I ultimately wound up going in once and not being picked up for the jury! So I spent a lot of time simply digging up fulltext papers for citations, so at least there’s now something like another 100 papers available online for melatonin/nicotine/modafinil etc…
-
I was thinking of trying to meta-analyze the correlation of lithium in drinking-water with suicides/murders/mental-illness, but after I got copies of all the citations I knew of, I’m not sure the data is homogenous to do that, which is disappointing, and the meta-analysis papers/textbooks I’ve checked don’t seem to be very encouraging about the utility of doing it with epidemiology stuff.
-
And reorganizing & fixing broken links & updating various pages (I figured out how to make a fun forest plot showing the active/passive split in dual n-back studies)
-
My long-running font A/B test finished but with the most boring possible results of close to zero difference between the 4 fonts
-
I researched an old family friend in his 90s who has never been willing to talk about his government work during the Cold War and found some stuff using released Census records, but that’s not really of interest to other people, and I decided to not make it public. Likewise when I added ~40 book reviews from my old notes to my Goodreads account.
-
I managed to trace an arrested drug dealer back to his Silk Road account, which was somewhat interesting: Reddit discussion
-
Is one familiar with Fukuyama & ‘the end of history’? I think he’s right but no one seems to agree with me, so I wrote a short essay defending him Like most political essays, it’s probably worse than I realize.
-
From my perspective, probably the most interesting thing I wrote all month was some criticism of early SF, pointing out the obsolete science behind some stuff written off as fantasy.
May 2013
-
I’ve made a little more progress on the Touhou project
-
Added 2 new studies to the DNB meta-analysis and also a new covariate (whether payment reduces gains: it doesn’t)
-
updated my analysis of SDr’s sleep data
-
oh wait, I did do a new project: applied survival analysis to modeling fiction reviews
And in my Google analysis (which I credit to last month and not May, even if it went viral in early May), I added in random survival forests, fixing that gap in evaluating prediction methods, which was a little burden of guilt off my mind.
While I was at it, I reproduced that recent paper analyzing Bitcoin exchange shutdown or theft risk. The dude’s since given me his source code. (What can I say? Survival analysis is a great hammer, and it cost me enough tears and sweat to learn how to use the R library that I plan to use it everywhere I can.)
-
I started a little Noopept self-experiment using the Noopept someone gave me, but unfortunately they gave me too little for the results to be very meaningful (see the power analysis); but maybe they will have a trend and I can try a bigger experiment later.
-
TruBrain sent me a month’s supply of their all-in-one nootropic, but I haven’t tried it yet because it would interfere with the Noopept. I also purchased magnesium l-threonate, which is a disappointment so far, and some nicotine patches, which I haven’t used yet. A small sample order from a new modafinil website selling the usual Indian Modalert is in progress but hasn’t arrived yet. (It ultimately did not arrive as the delivery required a signature.)
-
I have a weird little literature/historical/survey article in progress; we’ll see where that goes
-
I posted an analysis I wrote a few months ago in private of whether a particular vendor on Silk Road is a federal mole (probably not)
Actually, I guess it was overall a pretty productive month. Probably helped that jury duty has so far turned out to be a bust: I’ve been on call since 21 May but have yet to actually go into the courthouse to do anything.
2012–201311ya
But where shall wisdom be found?
And where is the place of understanding?
Man knoweth not the price thereof;
neither is it found in the land of the living
…for the price of wisdom is above rubies.Job 28:12 (KJV translation)
Here is material I’ve worked on in the 477 days since my last update. In roughly chronological & topical order, here are the major additions to Gwern.net:
-
I interviewed translator Michael House about his work in Japan as a translator
-
finished data collection for my hafu anime statistics page and begun analysis. (I’ve achieved good coverage of characters, found an astonishingly consistent absence of Korean characters, and confirmed the blond-haired/blue-eyed stereotype; but my original thesis doesn’t seem to work and the data is too unevenly distributed to identify time trends.)
-
judged the 2011 & 2012 results for the Haskell Summer of Codes and the accuracy of my predictions
-
did a meta-analysis on whether dual n-back increases IQ, and examining possible biases and various claims about what makes the training work or not work
-
did another meta-analysis on whether iodine increases IQ, etc
-
modafinil:
-
checked for subjective effects of blinded modafinil
-
updated my modafinil price-chart twice, and expanded with brand data and a new armodafinil table
-
researched modafinil-related prosecutions & convictions in the USA
-
and any connection with schizophrenia
-
-
tried kratom
-
did 2 potassium experiments; neither improved my mood/productivity, and one damaged my sleep
-
my Silk Road page has been expanded with a BBC interview, putting SR in a historical cypherpunk context, an updated account of all arrests & law enforcement actions, and application of basic statistics to ordering
-
ran 2 sleep experiments on the timing of taking a vitamin D supplement: I found that taking vitamin D before bed substantially damaged my sleep, while taking vitamin D after waking up did not hurt & somewhat helped. (A later re-analysis revealed this result to be due to a flaw in the analysis; a better-done analysis revealed no particular improvement or damage to sleep.)
-
checked whether a walking desk (treadmill) damaged typing speed or accuracy
-
I have run 3 Wikipedia experiments establishing that: Talk page edits are ignored by editors; random link deletions (and their restoration) are also ignored by editors; and external link suggestions on Talk pages are also ignored by readers. (I take the former 2 as indicative of the decline in edit activity and rise of deletionist beliefs on Wikipedia.)
-
tried some economic/historical analysis: “Reasons of State: Why Didn’t Denmark Sell Greenland to the USA?
-
Defending sunk costs essay (LW discussion)
-
Slowing Moore’s Law: Why You Might Want To and How You Would Do It
-
tried estimating the bandwidth of a Death Note
-
compiled predictions for Harry Potter and the Methods of Rationality
-
looked into Conscientiousness and online education; studies so far are useless from a meta-analytic standpoint
-
tripled length of appendix dealing with the reliability of mainstream science (methodological flaws, replication rates, etc)
-
finished meta-ethics essay, The Narrowing Circle
-
explained the philosophy saying one man’s modus ponens is another man’s modus tollens
-
speculation about a restoration of the British monarchy
-
clean up & exploratory data analysis of SDr’s lucid dreaming data
-
Turing-completeness in surprising places (inventory of particularly “weird machines”; relevant to computer and AI security)
Transcribed or translated:
-
Douglas Hofstadter’s superrationality columns (from Metamagical Themas, 198539ya)
-
The Iron Law Of Evaluation And Other Metallic Rules, Rossi 198737ya (lessons from the large RCTs evaluating social & welfare interventions)
-
The Ups and Downs of the Hope Function In a Fruitless Search, et al 1994
-
Shiny balls of Mud: William Gibson Looks at Japanese Pursuits of Perfection (200222ya)
-
1996 NewType interview with Hideaki Anno (translated by me, with the help of an EGFer)
-
1997 Animeland interview with Hideaki Anno (bought, transcribed, and translated by me with the help of other LWers)
More technical:
-
added edit history statistics/visualization for Gwern.net using GitStats
-
site traffic updates: July–December 2011, January 2012–July 2012, July 2012–Jan 2013
-
There’s also been a lot of backend changes: switching to Amazon S3+Cloudflare, adding error pages, metadata like tags, A/B testing, but no need to go into detail.
Personal:
-
posted summaries of my personality & attitudes & my RSS feed collection
2011
I’d like to poison your mind
With wrong ideas that appeal to you
Though I am not unkind…
Per my past practice of linking stuff I think LWers will find interesting, here is what I’ve been up to lately:
-
Politicians are Ethical: applying base-rate neglect
-
Modafinil: a dual-pronged argument for low-balling estimates of modafinil harm by pointing to both temporal and quality discounting of health in old age. (If you missed it the first time around, I still think my mini-tutorial on drug ordering with Bayes’s theorem is worth reading.)
-
The Narrowing Circle: argument that the usual belief of ‘moral progress’ and the ‘expanding circle’ assume many of their conclusions by pointing to the beliefs and classes of entities discarded along the way. (As many LWers share those assumptions and will be unsympathetic, the interesting parts may be the appendices on perpetuities and waqfs, inasmuch as those bear directly on cryonics.)
-
Both Modafinil and Spaced repetition have been expanded with scores more links to studies & PDFs. (Nicotine and Melatonin are next.)
-
Worldbuilding: The Lights in the Sky are Sacs is a silly bit of SF/alternate history speculation involving floating hydrogen-sac organisms.
-
Wikipedia and Knol has been completed, as the 7 predictions I made on the matter have been judged thanks to Google’s recent announcements; I blew one.
-
Stuff which is incomplete or which is just a pile of notes:
-
The morality of sperm donation (sketch of argument smart utilitarians should donate sperm)
-
notes on intermittent fasting
-
I stuck a link in the footer of every page to a Google spreadsheet form, borrowing the idea from Luke Muehlhauser—I’ve only gotten 1 feedback so far, IIRC, but that was before I put it in the footer and updated all the pages a few hours ago. (As of 2015, there are hundreds of responses and I consider the feedback form to have paid its way; see my later writeup detailing the benefits.)
-
Since removed in April 2021 when Chrome desktop support for hyphenation became sufficiently widespread.↩︎