@sama also said that we shouldn't think too much about parameter count. Lots of room to make current models better w/out more params, so the current work is probably to optimize GPT-4 and make it better and/or cheaper to run. Azure bills are adding up!
patmcguinness.substack.com/