Even the most connected tech entrepreneurs in the world are struggling to secure capacity. During a 2023 May 16 congressional hearing on AI, OpenAI CEO Sam Altman said it would be better if fewer people used ChatGPT because of the processor bottleneck. “GPUs at this point are considerably harder to get than drugs”, Elon Musk told The Wall Street Journal CEO Council Summit on May 23. Being Musk has its perks, though. Earlier this year, startups clamoring for Oracle computing capacity were abruptly told that a buyer had snapped up much of Oracle’s spare server space, people familiar with the matter said. The buyer, the startups were told, was Musk, who is building his own OpenAI rival called X.AI, the people said.
…UBS analysts estimate an earlier version of ChatGPT required about 10,000 graphic chips. Musk estimates that an updated version requires 3–5× as many of Nvidia’s advanced processors.
…Some investors are combing their networks for spare computing power while others are orchestrating bulk orders of processors and server capacity that can be shared across their AI startups. Startups are shrinking their AI models to make them more efficient, buying their own physical servers with relevant graphics chips or switching to less-popular cloud providers such as Oracle until the shortage is resolved, according to AI investors and startups. Other founders are simply begging salespeople at Amazon and Microsoft for more power.
…Some companies are blocking off cloud capacity for fear they won’t be able to access them later. “People are now just continuing to pay for them even if they don’t need them”, said Adam Wenchel, CEO of Arthur, which builds tools to protect companies from AI risks such as data leaks.
…Server manufacturers and their direct customers say they are facing waits of more than 6 months to get Nvidia’s latest graphics chips. The CEO of Supermicro, one of the largest server-makers, said the company’s back-orders of systems featuring graphic chips was at its highest level ever and that the company was rushing to add manufacturing capacity.
…Kanjun Qiu, the CEO of Generally Intelligent, an AI research company, has been buying advanced graphic chips since last year for her own servers, allowing her to ride out the current shortage. A venture capitalist recently messaged her asking if she had spare capacity that she could rent to other startups. Qiu hasn’t decided whether to part with her chips.
…Altman and other OpenAI representatives have told founders that the company is working to address the issue in partnership with Microsoft, its largest investor and data-center provider.