𝔊𝔴𝔢𝔯𝔫@gwernJul 17Reminder for GPT-3 users: "Sampling can prove the presence of knowledge but not the absence."
The prompt & sampling hyperparameters matter a 𝘭𝘰𝘵. The overwhelming majority of the time GPT-3 "failed" me, I eventually found a prompt+settings which worked—the failure was mine.
99,870
3,552
3.6%
View Tweet activity
𝔊𝔴𝔢𝔯𝔫@gwernJul 9Ganbooru work has been far slower than I expected in April, so as a stopgap I'm experimenting with upgrading plot summaries using GPT-3 through the OA API: gwern.net/TWDNE#gpt-3 I'll call this one TWDNE v3.5. The summaries are pretty good.
48,881
385
0.8%
View Tweet activity
𝔊𝔴𝔢𝔯𝔫@gwernJul 14Expired: T2 (Turing Test)
Tired: SuperGLUE, T5
Wired: T3 (Tesla Turingnator Test): when a robot can autonomously navigate itself, stopping only at superchargers, to the household of arbitrary small children and locate them.
40,969
257
0.6%
View Tweet activity
𝔊𝔴𝔢𝔯𝔫@gwernJul 7Q. "What do Catholics call it when you eat only pickles for Lent?"
A. "The penance of Rick-and-Mortification of the flesh."
40,576
256
0.6%
View Tweet activity
𝔊𝔴𝔢𝔯𝔫@gwernJul 23Idle thought: should I add a "GPUs go brrr" meme or are there too many memes & graphs in this writeup as 'tis?
40,019
309
0.8%
View Tweet activity
𝔊𝔴𝔢𝔯𝔫@gwernJul 12"'I'm disgusted watching people on trains stroke their iPads with those strange gestures!', Tom said spunkily."
39,118
291
0.7%
View Tweet activity
𝔊𝔴𝔢𝔯𝔫@gwernJul 19I'm a little impressed it's memorized enough URLs that that reliably works.
𝔊𝔴𝔢𝔯𝔫@gwernJul 22When you have a half-working prompt, GPT-3 feels like trying to teach a superintelligent cat to do a trick: it's not that it 𝘤𝘢𝘯'𝘵, it just 𝘸𝘰𝘯'𝘵—you know you're making progress because it did perfectly last time, but this time it rolled over and started licking its butt.
𝔊𝔴𝔢𝔯𝔫@gwernJul 10ME (writing up): "2-player 0-sum games w/simultaneous gradient descent on deep nonlinear NNs easily diverge, requiring tuning of $\Beta_1$/$\Beta_2$ and $\epsilon$ LRs..."
ME (training): "Oh no! Mr G is not wiggly enough! And he's so high he'll fall off! Me help G LR go faster!"
𝔊𝔴𝔢𝔯𝔫@gwernJul 4I dunno, seems kinda grim.
"My bottom is made of rice and rapidly dissolving away into the curry. I don't have long."
"Why did you ever believe you were immortal? Nothing in this universe lasts, least of all decorative rice animals. Kiss me now, you ducking fool!"
𝔊𝔴𝔢𝔯𝔫@gwernJul 19But you know Transformers have already been shown to be universal and equivalent to RNNs, and there are multiple ways to build in history.
14,838
431
2.9%
View Tweet activity
𝔊𝔴𝔢𝔯𝔫@gwernJul 23One fun scenario: standalone complexes. Imagine Qanon, except Q is a specific, regularly updated, GPT-3 prompt that everyone talks to to develop the conspiracy further and get personal guidance.
13,646
523
3.8%
View Tweet activity
𝔊𝔴𝔢𝔯𝔫@gwernJul 1Although GPT-3's "Devil's Dictionary" is even better: pastebin.com/G1MwG6gg
""Researcher"
[noun] A form of modern industry based on a mix of small molecules of grant money and arbitrary experimental methods."
13,580
2,185
16.1%
View Tweet activity
𝔊𝔴𝔢𝔯𝔫@gwernJul 22Being so infamous, I thought it'd be a cinch to imitate you, but GPT-3 had odd difficulty 'locating' you. If I used the full title or the Amazon summary, it'd veer into generically positive summary/reviews etc. I had to cut back my prompt quite a bit. (No Brooks or golf here.) pic.twitter.com/pnCXF4EQN5
13,370
3,081
23.0%
View Tweet activity
Get your Tweets in front of more people.
Use Tweet Activity to track how your Tweets are doing.
Engagements
Showing 31 days with daily frequency
Engagement rate
3.3%
Jul 31
3.7% engagement rate
Link clicks
9.3K
Jul 31
263 link clicks
On average, you earned 301 link clicks per day
Retweets without comments
431
Jul 31
10 Retweets without comments
On average, you earned 14 Retweets without comments per day
![](data:image/svg+xml,<svg xmlns="http://www.w3.org/2000/svg" width="128" height="128"><rect fill-opacity="0"/></svg>)
