“The inside Story of ChatGPT: How OpenAI Founder Sam Altman Built the World’s Hottest Technology With Billions from Microsoft”, 2023-01-25 (; backlinks):
The AI future, according to Altman, could be spectacular—unless it goes spectacularly wrong. Why Big Tech giants and business leaders everywhere are losing sleep over generative AI
…OpenAI had already created one of the world’s most powerful LLMs. Called GPT-3, it takes in more than 175 billion statistical connections and is trained on about 2⁄3 of the internet, all of Wikipedia, and two large data sets of books. But OpenAI found it could be tricky to get GPT-3 to produce exactly what a user wanted. One team had the idea of using reinforcement learning—in which an AI system learns from trial and error to maximize a reward—to perfect the model. The team thought that a chatbot might be a great candidate for this method since constant feedback, in the form of human dialogue, would make it easy for the AI software to know when it had done a good job and where it needed to improve. So in early 2022, the team started building what would become ChatGPT.
When it was ready, OpenAI let beta testers play with ChatGPT. But they didn’t embrace it in the way OpenAI had hoped, according to Greg Brockman, an OpenAI cofounder and its current president; it wasn’t clear to people what they were supposed to talk to the chatbot about. For a while, OpenAI switched gears and tried to build expert chatbots that could help professionals in specific domains. But that effort ran into problems too—in part because OpenAI lacked the right data to train expert bots. Almost as a Hail Mary, Brockman says, OpenAI decided to pull ChatGPT off the bench and put it in the wild for the public to use. “I’ll admit that I was on the side of, like, I don’t know if this is going to work”, Brockman says.
The chatbot’s instant virality caught OpenAI off guard, its execs insist. “This was definitely surprising”, Mira Murati, OpenAI’s chief technology officer, says. At the San Francisco VC event, Altman said, he “would have expected maybe one order of magnitude less of everything—one order of magnitude less of hype.”