Built a token-wise likelihood visualizer for GPT-2 over the weekend. There are some interesting patterns and behaviors you can easily pick up from a visualization like this, like induction heads and which kinds of words/grammar LMs like to guess.

Jan 24, 2023 · 4:52 AM UTC

14
90
15
786
0
A particularly interesting example to run through this is source code, where because of the regular structure the LM does much better (much lower perplexity). Indentations and punctuation are particularly easy wins for GPT-2.
3
5
1
48
You can also use this viz to probe GPT-2 for what it thinks about different topics, which is kind of fun. You can imagine extensions of this "fill in the blank" UX become useful for writing workflows.
1
2
38
Replying to @thesephist
This is so useful! Any thoughts on what it would it take to turn something like this into an interactive web page people could try out for themselves? I wonder if one of the LLMs-compiled-to-WebAssembly could handle this
1
18
You could definitely do this with transformers.js and a small model like gpt2-small since the model needn't be large to have the padagogical effect. I currently just have a demo that runs GPT2-xl on the server. One of the many things I haven't yet had time to make public 🫠
1
1
14
more replies
Replying to @thesephist
I’d love your thoughts on mapping a color space to probability. I once prototyped something similar and found the huge variance in likelihoods for different words made that a bit tricksy, but this looks really good.
2
6
My coloring algorithm is roughly: min, max = mean(log_probs) ± 2.5 * stddev(log_probs) hue = token_logprob.clamp(min, max).scale(0, 150) color = f"hsl({hue}deg 60% 85%)" Key is to scale probs to hue in the log space, and then clamp at µ±2.5stddev.
1
24
more replies
Replying to @thesephist
Have you heard of circuitsvis? It's a great open source library that also does this. Would also strongly recommend TransformerLens github.com/alan-cooney/Circu… and github.com/neelnanda-io/Tran…

GitHub - neelnanda-io/TransformerLens: A library for mechanistic interpretability of GPT-style...

A library for mechanistic interpretability of GPT-style language models - GitHub - neelnanda-io/TransformerLens: A library for mechanistic interpretability of GPT-style language models

github.com
1
3
20
Yes, TransformerLens and Neel are fantastic <3
4
Replying to @thesephist
Open-source? I was thinking of building a similar UI, as I’m sure many have. Would love to contribute. Awesome work!
1
Replying to @thesephist
Interesting results.
Load more