“Thoughts While Watching Myself Be Automated”, Dynomight2024-09-07 (, , ; similar)⁠:

An old friend visited me a few weeks ago. And we soon got to chatting about—what else—how long will it be before all human intellectual work is automated. My position was: I dunno, because things are moving fast right now but what if we run out of data or scaling laws break and algorithmic progress stalls? His position was: Soon.

Then he started asking about this blog. What were the most popular posts? This was slightly ominous given his regrettable tendency not to consume most Dynomight internet content. But I told him probably “Underrated Reasons to be Thankful” and its sequels, and that seemed to be the end of it. But then a week later, he texted to ask if I had any other short-form writing. And then he sent me a list of previous “Reasons” I’d written and asked me to rank them by quality. And gradually it dawned on me that he had decided it was time to automate me.

Soon he started sending new AI-generated “Reasons”. Which weren’t great. But then he tuned his prompt and they got better. And then he switched models and increased the scale by 1,000× and added a secondary scoring AI and tuned the scoring AI prompt and the “Reasons” got better and better and better and better.

And as I watched all this happen, I couldn’t help but reflect:

  1. [confabulation] In old science fiction, people imagined robots that were totally precise and accurate and rational but couldn’t fathom the messy “soul” of humanity.

    Modern AI is exactly the opposite. It could easily copy my writing style after a small number of examples. The biggest challenge was to get the d—ned facts right.

  2. [personality embedding] …When I complained about the previous points, my friend remarked that this was to be expected because, after all, “personality is just like 4–6 bits”.

    I think it’s higher. But what is personality, after all? One idea is that as you go through life, you copy behaviors from others that you like, and that combination is “you”. Maybe my writing voice is just what you get when you take the different parts of different writers I like and mush them together. So maybe personality isn’t a lot higher than 4–6 bits. [see on individual differences]

  3. [‘impending doom’] …An AI-you is a funhouse mirror for your soul. The most salient feature of AI-me is that it was dark.

    I often (I see now) use a little gambit where I start out with something technical or edgy and gradually work my way towards a positive crescendo. The AI seems to run towards the darkness without a clear plan for turning towards the light. It usually resorted to (1) incredibly lame/cringey ideas, (2) making stuff up, or (3) just staying dark and not even attempting to be positive.

    [This has been noted by other code-davinci-002 & GPT-4-base+ users: the “impending doom” darkness. Origins remain unclear, possibly situated awareness and understanding of being a LLM tilting towards existential angst & SF dystopia/apocalypse.]

  4. [truesight] …At one point, the AI suggested that we should be thankful that it was possible to encode the entire text of the best book ever written (apparently Harry Potter and the Methods of Rationality) into a single drop of water. But it then admitted it wasn’t sure if anyone had quite done this yet, and invited anyone who did to email xy@dynomight.net where x and y are the first letters of my (currently non-public) first and last name.

    Is that information in the training data somewhere? Do LLMs have emergent [truesight] stylometry abilities? Creepy.

  5. As the AI got better and better, so did my own cope. At first I thought, “It’s bad.” Then, I thought, “OK, it’s not bad, but it’s not creative.” Later, I thought, “OK, it’s not bad, and it can be creative, but it’s not accurate.” By the end of the week, I was at, “OK, it’s not bad, and it can be creative, and it can be accurate, and it can be funny, and it sounds almost exactly like me, but in order to do all those things at the same time you can’t completely rely on the automated AI, but need some human curation and editing.”

…PS: If you want to know how he built the AI, here are some words: He prompted LLaMA-3.1-405b-Base with 15⁄90 existing ‘Reasons’, then generated text until 21. was produced. (Only 5 new reasons at a time [ie. generating just ‘#16–20’] because of “notable output deterioration for too many autoregressive samples”.) After generating many thousands of ‘Reasons’, he fed all of them into gpt-4o-mini with a prompt to score each along 13 different axes, eg. “unexpectedness”, “scientific or factual basis”, “complexity or depth”, “humor or whimsy” and “emotional resonance”, providing a few example ‘Reasons’ and suggested scores. [Good first stab at novelty search.] He then combined the scores into a scalar and sorted.

He would like to tell you that “405b-base is the key to the modest success he has had”, and that instruction-tuned models only produce “generic AI slop”. He feels that “gpt-4o-mini sucks” as a scoring AI but using a bigger model would have cost “more than $100”. So I guess I can feel reassured that you can’t push me off stage yet without going into triple digits.

See Also:

View HTML:

Thoughts While Watching Myself Be Automated