“GPT-3 Nonfiction § Ferrucci2020”, Gwern2020-06-19 (, )⁠:

Nonfiction writing by OpenAI’s GPT-3 model, testing logic, commonsense reasoning, anagrams, PDF/OCR cleaning, creative nonfiction, etc

AI researcher David Ferrucci argues that NN model scaling is a dead end, because GPT-2 cannot provide sensible completions to a story about what happens when you water a dying plant, since GPT-2 only generates ~2⁄15 sensible completions.

I find GPT-3 generates 10⁄10.