“What to Expect When You’re Expecting…GPT-4. What Comes After ChatGPT? 7 Predictions for 2023 § GPT-4”, 2022-12-25 ():
…I guarantee that minds will be blown. I know several people who have actually tried GPT-4, and all were impressed. It truly is coming soon (Spring of 2023, according to some rumors). When it comes out, it will totally eclipse ChatGPT; it’s safe bet that even more people will be talking about it…
In many quarters, expectations are really, really high.
In technical terms, GPT-4 will have more parameters inside of it, requiring more processors and memory to be tied together, and be trained on more data. GPT-1 was trained on 4.6 gigabytes of data, GPT-2 was trained on 46 gigabytes, GPT-3 was trained on 750GB. GPT-4 will be trained on considerably more, a substantial fraction of the internet as a whole. As OpenAI has learned, bigger in many ways means better, with outputs more and more hu-manlike with each iteration. GPT-4 is going to be a monster…As far as I can tell from rumors, GPT-4 is architecturally essentially the same as GPT-3.
…Shiny things are always fun to play with, and I fully expect GPT-4 to be the shiniest so far, but that doesn’t mean that it is a critical step on the optimal path to AI that we can trust. For that, we will, I predict, need genuinely new architectures that incorporate explicit knowledge and world models at their very core.