The revelation that ChatGPT, the astonishing artificial-intelligence chatbot, had been trained on an Nvidia supercomputer spurred one of the largest single-day gains in stock-market history. When the Nasdaq opened on May 25, 2023, Nvidia’s value increased by about $200b. A few months earlier, Jensen Huang, Nvidia’s CEO had informed investors that Nvidia had sold similar supercomputers to 50 of America’s hundred largest companies. By the close of trading, Nvidia was the 6th most valuable corporation on earth, worth more than Walmart and Exxon Mobil combined. Huang’s business position can be compared to that of Samuel Brannan, the celebrated vendor of prospecting supplies in San Francisco in the late 1840s. “There’s a war going on out there in AI and Nvidia is the only arms dealer”, one Wall Street analyst said.
…Huang has a practical mind-set, dislikes speculation, and has never read a science-fiction novel. He reasons from first principles about what microchips can do today, then gambles with great conviction on what they will do tomorrow. “I do everything I can not to go out of business”, he said at breakfast. “I do everything I can not to fail.”
Huang believes that the basic architecture of digital computing, little changed since it was introduced by IBM in the early 1960s, is now being reconceptualized. “Deep learning is not an algorithm”, he said recently. “Deep learning is a method. It’s a new way of developing software.” The evening before our breakfast, I’d watched a video in which a robot, running this new kind of software, stared at its hands in seeming recognition, then sorted a collection of colored blocks.
The video had given me chills; the obsolescence of my species seemed near. Huang, rolling a pancake around a sausage with his fingers, dismissed my concerns. “I know how it works, so there’s nothing there”, he said. “It’s no different than how microwaves work.” I pressed Huang—an autonomous robot surely presents risks that a microwave oven does not. He responded that he has never worried about the technology, not once. “All it’s doing is processing data”, he said. “There are so many other things to worry about.”
In May, hundreds of industry leaders endorsed a statement that equated the risk of runaway AI with that of nuclear war. Huang didn’t sign it. Some economists have observed that the Industrial Revolution led to a relative decline in the global population of horses, and have wondered if AI might do the same to humans. “Horses have limited career options”, Huang said. “For example, horses can’t type.” As he finished eating, I expressed my concerns that, someday soon, I would feed my notes from our conversation into an intelligence engine, then watch as it produced structured, superior prose. Huang didn’t dismiss this possibility, but he assured me that I had a few years before my John Henry moment. “It will come for the fiction writers first”, he said. Then he tipped the waitress a thousand dollars, and stood up to accept his award…At Denny’s, Huang told me to expect a world in which robots would fade into the background, like household appliances. “In the future, everything that moves will be autonomous”, he said.
…Following the interview, Huang took questions from the audience, including one about the potential risks of AI “There’s the doomsday AIs—the AI that somehow jumped out of the computer and consumes tons and tons of information and learns all by itself, reshaping its attitude and sensibility, and starts making decisions on its own, including pressing buttons of all kinds”, Huang said, pantomiming pressing the buttons in the air. The room grew very quiet. “No AI should be able to learn without a human in the loop”, he said. One architect asked when AI might start to figure things out on its own. “Reasoning capability is 2–3 years out”, Huang said. A low murmur went through the crowd.
…Nvidia executives were building the Manhattan Project of computer science, but when I questioned them about the wisdom of creating superhuman intelligence they looked at me as if I were questioning the utility of the washing machine. I had wondered aloud if an AI might someday kill someone. “Eh, electricity kills people every year”, [DL researcher Bryan] Catanzaro said. I wondered if it might eliminate art. “It will make art better!” Diercks said. “It will make you much better at your job.” I wondered if someday soon an AI might become self-aware. “In order for you to be a creature, you have to be conscious. You have to have some knowledge of self, right?” Huang said. “I don’t know where that could happen.”
…Huang’s vision is to unify Nvidia’s computer-graphics research with its generative-AI research. As he sees it, image-generation AIs will soon be so sophisticated that they will be able to render 3-dimensional, inhabitable worlds and populate them with realistic-seeming people. At the same time, language-processing AIs will be able to interpret voice commands immediately. (“The programming language of the future will be ‘human’”, Huang has said.) Once the technologies are united with ray-tracing, users will be able to speak whole universes into existence. Huang hopes to use such “digital twins” of our own world to safely train robots and self-driving cars. Combined with VR technology, the Omniverse could also allow users to inhabit bespoke realities.
…Although Huang lived at the academy, he was too young to attend its classes, so he went to a nearby public school. There, he befriended Ben Bays, who lived with his 5 siblings in an old house with no running water. “Most of the kids at the school were children of tobacco farmers”, Bays said, “or just poor kids living in the mouth of the holler.” Huang arrived with the school year already in session, and Bays remembers the principal introducing an undersized Asian immigrant with long hair and heavily accented English. “He was a perfect target”, Bays said. Huang was relentlessly bullied. “The way you described Chinese people back then was ‘Chinks’”, Huang told me, with no apparent emotion. “We were called that every day.” To get to school, Huang had to cross a rickety pedestrian footbridge over a river. “These swinging bridges, they were very high”, Bays said. “It was old planks, and most of them were missing.” Sometimes, when Huang was crossing the bridge, the local boys would grab the ropes and try to dislodge him. “Somehow it never seemed to affect him”, Bays said. “He just shook it off.” By the end of the school year, Bays told me, Huang was leading those same kids on adventures into the woods. Bays recalled how carefully Huang stepped around the missing planks. “Actually, it looked like he was having fun”, he said. Huang credits his time at Oneida with building resiliency. “Back then, there wasn’t a counselor to talk to”, he told me. “Back then, you just had to toughen up and move on.” In 2019, he donated a building to the school, and talked fondly of the (now gone) footbridge, neglecting to mention the bullies who had tried to toss him off it.
…Huang excelled in high school, and was a nationally ranked table-tennis player. He belonged to the school’s math, computer, and science clubs, skipped two grades, and graduated when he was 16. “I did not have a girlfriend”, he said…“There were, like, 250 kids in electrical engineering, and maybe 3 girls”, Huang told me. Competition broke out among the male undergraduates for Mills’s attention, and Huang felt that he was at a disadvantage. “I was the youngest kid in the class”, he said. “I looked like I was about 12.” Every weekend, Huang would call Mills and pester her to do homework with him. “I tried to impress her—not with my looks, of course, but with my strong capability to complete homework”, he said. Mills accepted, and, after 6 months of homework, Huang worked up the courage to ask her out on a date. She accepted that offer, too.
…“I find that I think best when I’m under adversity”, Huang said. “My heart rate actually goes down. Anyone who’s dealt with rush hour in a restaurant knows what I’m talking about.”…Short on money, Huang decided that his only hope was to use the conventional triangle approach and try to beat the competition to market. In 1996, he laid off more than half the 100 people working at Nvidia, then bet the company’s remaining funds on a production run of untested microchips that he wasn’t sure would work. “It was 50-50”, Huang told me, “but we were going out of business anyway.”
When the product, known as RIVA 128, hit stores, Nvidia had enough money to meet only one month of payroll. But the gamble paid off, and Nvidia sold a million RIVAs in 4 months. Huang encouraged his employees to continue shipping products with a sense of desperation, and for years to come he opened staff presentations with the words “Our company is 30 days from going out of business.” The phrase remains the unofficial corporate motto.
…Employee demographics are “diverse”, sort of—I would guess, based on a visual survey of the cafeteria at lunchtime, that about a third of the staff is South Asian, a third is East Asian, and a third is white. The workers are overwhelmingly male.
…When CUDA was released, in late 2006, Wall Street reacted with dismay. Huang was bringing supercomputing to the masses, but the masses had shown no indication that they wanted such a thing. “They were spending a fortune on this new chip architecture”, Ben Gilbert, the co-host of Acquired [Huang interview], a popular Silicon Valley podcast, said. “They were spending many billions targeting an obscure corner of academic and scientific computing, which was not a large market at the time—certainly less than the billions they were pouring in.” Huang argued that the simple existence of CUDA would enlarge the supercomputing sector. This view was not widely held, and by the end of 2008 Nvidia’s stock price had declined by 70%.
In speeches, Huang has cited a visit to the office of Ting-Wai Chiu, a professor of physics at National Taiwan University, as giving him confidence during this time. Chiu, seeking to simulate the evolution of matter following the Big Bang, had constructed a homemade supercomputer in a laboratory adjacent to his office. Huang arrived to find the lab littered with GeForce boxes and the computer cooled by oscillating desk fans. “Jensen is a visionary”, Chiu told me. “He made my life’s work possible.”
Chiu was the model customer, but there weren’t many like him. Downloads of CUDA hit a peak in 2009, then declined for 3 years. Board members worried that Nvidia’s depressed stock price would make it a target for corporate raiders. “We did everything we could to protect the company against an activist shareholder who might come in and try to break it up”, Jim Gaither, a longtime board member, told me. Dawn Hudson, a former NFL marketing executive, joined the board in 2013. “It was a distinctly flat, stagnant company”, she said.
In marketing CUDA, Nvidia had sought a range of customers, including stock traders, oil prospectors, and molecular biologists. At one point, the company signed a deal with General Mills to simulate the thermal physics of cooking frozen pizza. One application that Nvidia spent little time thinking about was artificial intelligence. There didn’t seem to be much of a market.
At the beginning of the 2010s, AI was a neglected discipline. Progress in basic tasks such as image recognition and speech recognition had seen only halting progress. Within this unpopular academic field, an even less popular subfield solved problems using “neural networks”—computing structures inspired by the human brain. Many computer scientists considered neural networks to be discredited. “I was discouraged by my advisers from working on neural nets”, Catanzaro, the deep-learning researcher, told me, “because, at the time, they were considered to be outdated, and they didn’t work.”
Catanzaro described the researchers who continued to work on neural nets as “prophets in the wilderness.” One of those prophets was Geoffrey Hinton, a professor at the University of Toronto. In 2009, Hinton’s research group used Nvidia’s CUDA platform to train a neural network to recognize human speech. He was surprised by the quality of the results, which he presented at a conference later that year. He then reached out to Nvidia. “I sent an e-mail saying, ‘Look, I just told a thousand machine-learning researchers they should go and buy Nvidia cards. Can you send me a free one?’” Hinton told me. “They said no.”
Despite the snub, Hinton encouraged his students to use CUDA, including a Ukrainian-born protégé of his named Alex Krizhevsky, who Hinton thought was perhaps the finest programmer he’d ever met. In 2012, Krizhevsky and his research partner, Ilya Sutskever, working on a tight budget, bought two GeForce cards from Amazon. Krizhevsky then began training a visual-recognition neural network on Nvidia’s parallel-computing platform, feeding it millions of images in a single week. “He had the two GPU boards whirring in his bedroom”, Hinton said. “Actually, it was his parents who paid for the quite considerable electricity costs.”
Sutskever and Krizhevsky were astonished by the cards’ capabilities. Earlier that year, researchers at Google had trained a neural net that identified videos of cats, an effort that required some 16 thousand CPUs. Sutskever and Krizhevsky had produced world-class results with just two Nvidia circuit boards. “GPUs showed up and it felt like a miracle”, Sutskever told me.
AlexNet, the neural network that Krizhevsky trained in his parents’ house, can now be mentioned alongside the Wright Flyer and the Edison bulb. In 2012, Krizhevsky entered AlexNet into the annual ImageNet visual-recognition contest; neural networks were unpopular enough at the time that he was the only contestant to use this technique [most used SVMs]. AlexNet scored so well in the competition that the organizers initially wondered if Krizhevsky had somehow cheated. “That was a kind of Big Bang moment”, Hinton said. “That was the paradigm shift.”
In the decade since Krizhevsky’s 9-page description of AlexNet’s architecture was published, it has been cited >100,000×, making it one of the most important papers in the history of computer science. (AlexNet correctly identified photographs of a scooter, a leopard, and a container ship, among other things.) Krizhevsky pioneered a number of important programming techniques, but his key finding was that a specialized GPU could train neural networks up to a hundred times faster than a general-purpose CPU. “To do machine learning without CUDA would have just been too much trouble”, Hinton said.
Within a couple of years, every entrant in the ImageNet competition was using a neural network. By the mid-2010s, neural networks trained on GPUs were identifying images with 96% accuracy, surpassing humans. Huang’s 10-year crusade to democratize supercomputing had succeeded. “The fact that they can solve computer vision, which is completely unstructured, leads to the question ‘What else can you teach it?’” Huang said to me.
The answer seemed to be: everything. Huang concluded that neural networks would revolutionize society, and that he could use CUDA to corner the market on the necessary hardware. He announced that he was once again betting the company. “He sent out an e-mail on Friday evening saying everything is going to deep learning, and that we were no longer a graphics company”, Greg Estes, a vice-president at Nvidia, told me. “By Monday morning, we were an AI company. Literally, it was that fast.” Around the time Huang sent the e-mail, he approached Catanzaro, Nvidia’s leading AI researcher, with a thought experiment. “He told me to imagine he’d marched all 8000 of Nvidia’s employees into the parking lot”, Catanzaro said. “Then he told me I was free to select anyone from the parking lot to join my team.”
…Huang rarely gives interviews, and tends to deflect attention from himself. “I don’t really think I’ve done anything special here”, he told me. “It’s mostly my team.” (“He’s irreplaceable”, the board member Jim Gaither told me.) “I’m not sure why I was selected to be the CEO”, Huang said. “I didn’t have any particular drive.” (“He was determined to run a business by the time he was 30”, his co-founder Chris Malachowsky told me.) “I’m not a great speaker, really, because I’m quite introverted”, Huang said. (“He’s a great entertainer”, his friend Ben Bays told me.) “I only have one superpower—homework”, Huang said. (“He can master any subject over a weekend”, Dwight Diercks, Nvidia’s head of software, said.)
Huang prefers an agile corporate structure, with no fixed divisions or hierarchy. Instead, employees submit a weekly list of the 5 most important things they are working on. Brevity is encouraged, as Huang surveys these e-mails late into the night. Wandering through Nvidia’s giant campus, he often stops by the desks of junior employees and quizzes them on their work. A visit from Huang can turn a cubicle into an interrogation chamber. “Typically, in Silicon Valley, you can get away with fudging it”, the industry analyst Hans Mosesmann told me. “You can’t do that with Jensen. He will kind of lose his temper.”
Huang communicates to his staff by writing hundreds of e-mails per day, often only a few words long. One executive compared the e-mails to haiku, another to ransom notes. Huang has also developed a set of management aphorisms that he refers to regularly. When scheduling, Huang asks employees to consider “the speed of light.” This does not simply mean to move quickly; rather, employees are to consider the absolute fastest a task could conceivably be accomplished, then work backward toward an achievable goal. They are also encouraged to pursue the “zero-billion-dollar market.” This refers to exploratory products, such as CUDA, which not only do not have competitors but don’t even have obvious customers. (Huang sometimes reminded me of Kevin Costner’s character in Field of Dreams, who builds a baseball diamond in the middle of an Iowa cornfield, then waits for players and fans to arrive.)
Perhaps Huang’s most radical belief is that “failure must be shared.” In the early 2000s, Nvidia shipped a faulty graphics card with a loud, overactive fan. Instead of firing the card’s product managers, Huang arranged a meeting in which the managers presented, to a few hundred people, every decision they had made that led to the fiasco. (Nvidia also distributed to the press a satirical video, starring the product managers, in which the card was repurposed as a leaf blower.) Presenting one’s failures to an audience has become a beloved ritual at Nvidia, but such corporate struggle sessions are not for everyone. “You can kind of see right away who is going to last here and who is not”, Diercks said. “If someone starts getting defensive, I know they’re not going to make it.”
Huang’s employees sometimes complain of his mercurial personality. “It’s really about what’s going on in my brain versus what’s coming out of my mouth”, Huang told me. “When the mismatch is great, then it comes out as anger.” Even when he’s calm, Huang’s intensity can be overwhelming. “Interacting with him is kind of like sticking your finger in the electric socket”, one employee said. Still, Nvidia has high employee retention. Jeff Fisher, who runs the company’s consumer division, was one of the first employees. He’s now extremely wealthy, but he continues to work. “Many of us are financial volunteers at this point”, Fisher said, “but we believe in the mission.” Both of Huang’s children pursued jobs in the hospitality industry when they were in their twenties; following years of paternal browbeating, they now have careers at Nvidia. Catanzaro at one point left for another company. A few years later, he returned. “Jensen is not an easy person to get along with all of the time”, Catanzaro said. “I’ve been afraid of Jensen sometimes, but I also know that he loves me.”
…The buildings’ design won several awards and made Ko’s career. Still, Hao Ko recalled his time on the project with mixed emotions. “The place was finished, it looks amazing, we’re doing the tour, and he’s questioning me about the placement of the water fountains”, Ko said. “He was upset because they were next to the bathrooms! That’s required by code, and this is a billion-dollar building! But he just couldn’t let it go.” “I’m never satisfied”, Huang told me. “No matter what it is, I only see imperfections.”
…Huang told me that he didn’t know Lisa Su growing up; he met her only after she was named CEO “She’s terrific”, he said. “We’re not very competitive.” (Nvidia employees can recite the relative market share of Nvidia’s and AMD’s graphics cards from memory.) Their personalities are different: Su is reserved and stoic; Huang is temperamental and expressive. “She has a great poker face”, Mosesmann, the industry analyst, said. “Jensen does not, although he’d still find a way to beat you.” Su likes to tail the incumbent, and wait for it to falter. Unlike Huang, she is not afraid to compete with Intel, and, in the past decade, AMD has captured a large portion of Intel’s CPU business, a feat that analysts once regarded as impossible. Recently, Su has turned her attention to the AI market. “Jensen does not want to lose. He’s a driven guy”, Forrest Norrod, the executive overseeing AMD’s effort, said. “But we think we can compete with Nvidia.”
…Since then, Nvidia has been overwhelmed with customer requests. The company’s latest AI-training module, known as the DGX H100, is a 370-pound metal box that can cost up to $500,000. It is currently on back order for months. The DGX H100 runs 5× as fast as the hardware that trained ChatGPT, and could have trained AlexNet in less than a minute. Nvidia is projected to sell half a million of the devices by the end of the year…The gross profit margin on Nvidia’s equipment approaches 70%. This ratio attracts competition in the manner that chum attracts sharks. Google and Tesla are developing AI-training hardware, as are numerous startups. One of those startups is Cerebras, which makes a “mega-chip” the size of a dinner plate. “They’re just extorting their customers, and nobody will say it out loud”, Cerebras’s CEO Andrew Feldman, said of Nvidia. (Huang countered that a well-trained AI, model can reduce customers’ overhead in other business lines. “The more you buy, the more you save”, he said.)