[cf. Nadella interview] Yesterday Microsoft held a special event in Redmond to announce the new Bing, which incorporates a ChatGPT-like model into search [Prometheus/Sydney]. Microsoft was coy about what OpenAI model under-girded the new Bing, other than saying it was more advanced than the GPT-3.5 model behind ChatGPT; perhaps this is the launch of GPT-4…Microsoft clearly thinks this is a big deal: not only did this announcement warrant a special event, but Satya Nadella labeled this announcement as heralding a new era akin to the PC/server and mobile/cloud eras, and was explicit that Microsoft has Google in their sights.

…After Microsoft’s event I had the opportunity to talk with Microsoft CTO Kevin Scott and OpenAI founder and CEO Sam Altman:

Ben Thompson: …I’m talking to Kevin Scott, which I think is even better: from what I gather you’ve really been at the center of the relationship between Microsoft and OpenAI. What was the genesis of the partnership there?
Kevin Scott: Yes, to what you just said, we have been in the current instantiation of our partnership with OpenAI for about 3 and a half years now. I think very early in OpenAI’s history, they were on Azure briefly and then moved to Google for a bit and then after I joined Microsoft, I was looking after a bunch of the AI work that we were doing. Sam and I, funny enough, have known each other since he tried to recruit me to be the Head of Engineering at his startup, Loopt, a million years ago.
B. Thompson: So this is really going full circle.
K. Scott: Yeah. So Satya asked me to take point on having the conversation with OpenAI and seeing what we ought to be doing with them. I think that was in 2018, actually, we just ground through a bunch of things, decided that it would actually be a very good idea for Microsoft and OpenAI to partner, and we’ve been working super closely together ever since.
Sam Altman: Yeah. I agree with all that, but I think that’s underselling Kevin in this whole thing. I think the way it worked is that Satya and I had a few-minute conversation at a conference together in the summer of 2018 and said, “Hey, maybe we should figure something out more.”

But then from then on it was, really, we’ve worked with Kevin from the very beginning and very in-depth, and Kevin is the one that got what we were doing. Kevin also is most of the reason of why we’ve wanted to partner with Microsoft from the very beginning. Now we like many, many people there and feel very aligned, but I think it’s mostly been Kevin and me driving things and it’s been great.
Scott: Yeah, a ton of fun.

Thompson: So tell me about this when you were thinking about this partnership. I mean, you have this connection with Kevin, but then just broadly speaking from a business perspective, was it just that Microsoft came to the table with more funding, or were there competitive concerns like in the long run you saw yourself competing with Google? What things went into your decision making, Sam, as you were deciding to commit in this direction?
S. Altman: We trusted Kevin and Satya too, of course, but really I’m a big believer that you do a deal like this with people, not a company. And we had a lot of concerns.

There are many reasons I don’t like many of the big tech companies, and there weren’t that many options for us that could have done the scale not only of the capital we need, but compute and hardware and just general muscle as well. We were delighted that of that shortlist of people one of them was Microsoft, and, again, specifically Kevin and Satya. I don’t think we would’ve done the deal had we not had the relationships and felt the very high degree of alignment.

B T: How has the partnership changed over time? I mean, you just signed a new agreement. I’m sure you can’t disclose specific details, but I’m curious—you make an agreement up upfront, I’m sure there are a lot of disputes about exactly where the lines are drawn. Has there been a big shift in that as you’ve gone through this experience of working together or are things still mostly the way you thought they would be at the beginning?
K S: I think at the beginning the clear thing was that we had alignment, that we both really powerfully believed in this vision of powerful—what we call foundation models now—but these big models that we could use as platforms to develop lots of things on top of.

We had a shared vision of what you needed to do at scale. We, Microsoft, believed that OpenAI was one of, if not the best—and I would argue actually it is the best AI team pound-for-pound on the planet—and that we could work together. They would help us do the highest ambition things with their infrastructure, which would be beneficial not just for OpenAI, but for Microsoft and all of the other customers who were using us to do AI things, and then we could help OpenAI commercialize some of the things that they were doing.

I think one of the things people miss is that OpenAI has been able to accomplish all of this amazing stuff, and it’s a very small group of people. The last thing I think you would want a world-class team like OpenAI to do is have to be distracted by building an enterprise salesforce, although obviously they can build whatever salesforce they want because we are independent companies.
Altman: …I think we have extremely aligned vision and values about where we think this is going to go and what we want for the world, but extremely complementary skill sets and assets and things that we want.

The things that OpenAI wants and the things that Microsoft wants are compatible and not very overlapping. That’s led to where I think we’re crushing the game together jointly on technological progress, soon and increasingly so on business progress, and in terms of pushing for what we want about responsible deployment of these very powerful systems and massive shared benefit to the world.

…I don’t quite agree with the characterization of “giving away” because I think we’re getting far more value back here. We’re happy for Microsoft to be doing at-scale commercialization, but there are things that we want back like more computers, human data from using these products, things like that, and our partnership allows for that. Also Microsoft is very good about being extremely understanding of when we need to go off and commercialize something on our own for some reason.
S: …Yeah. I think that is actually how it’s worked over and over again. If you think about GitHub Copilot, for instance, it started with this realization with GPT-3 that you had a natural language model that could write code, that it could translate an English language intention of something you wanted to accomplish with a piece of code and produce actual code. Then it was just a lot of work to figure out how to turn that into a product and it was work across a whole bunch of different teams. So multiple parts of OpenAI, multiple parts of GitHub, and multiple parts of Microsoft—and that one was especially complicated because it was 3 organizations that are working together, not just two.

T: …Just a couple of practical questions. Sam, you mentioned in a tweet that ChatGPT is extremely expensive, on the order of pennies per query, which is an astronomical cost in tech.
S A: Per conversation, not per query.
T: Oh, okay, that’s a good clarification. Are there similar cost concerns about the Bing? One of the interesting queries, and this is maybe a question more for you Kevin, but I did a really interesting query this morning where I asked Bing to generate a bit of code.

The first answer used an old version of an API I told it to access. So I told it to use the new version, and what’s interesting is it did it, but it took a really long time, several minutes in fact. It felt like it was going out there, finding the new API, parsing it in real time, and figuring out the answer. That’s both very cool and it also sounds extremely expensive. How is this going to be managed as you expand access?
S: One of the things that we’ve gotten a lot of confidence in over the past handful of years is our ability to performance-optimize all of this stuff, both on the training and the inference side of things. Already the cost envelope for Bing looks very favorable to us. We think we actually have a lot of flexibility in bringing it to market as an ad-supported product that we wouldn’t have had if we hadn’t spent so much money on performance optimization and the cost is just going to go down over time.
A: And I think this is a great example of where the partnership works together, which is like, we all want this to be much cheaper at inference time for various reasons. Can we share resources, share ideas, can we talk together? Can we do this part? You do that part. Can we get really good at driving this cost down?

We didn’t expect ChatGPT to be such a success, so we had not gotten as disciplined about optimization at that point as we needed to. But since then we really have, and it’s been a great example of the partnership really working.

T: …Kevin, I actually want to delve in real quick, you said advertising will make sense, which is really interesting. The new Bing does have a combined interface with ads, but what was striking to me about watching the demos is that everyone, almost every demo, goes to the chat interface right away because it just feels more compelling, it’s not stuff smooshed together.

However, that really kills the user-selects-the-winner-of-an-auction dynamic that drives search monetization. Satya in another interview said something along the lines of, “Well, maybe there will be other business models.” Is this going to be a subscription? Is it going to turn out that these models are not necessarily making search more profitable, they’re actually value-destructive, and that’s actually fine for you because you have 4% share, and maybe not so fine for someone that may be more dominant in this space?
A: …I think it’s fabulous for both of us. I think there’s so much upside for both of us here. We’re going to discover what these new models can do, but if I were sitting on a lethargic search monopoly and had to think about a world where there was going to be a real challenge to the way that monetization of this works and new ad units, and maybe even a temporary downward pressure, I would not feel great about that.
T: …From Microsoft’s perspective, is this going to be a funnel into new products or do you see it as an end goal in and of itself, winning search?
S: So I think you hit on a very important point which is even if the ad economics of this system doesn’t have the same economics that “normal search” has, if we gain share, it’s just great for Microsoft.

I think we have a lot of ability here, partially because we’ve done so much performance optimization work and we’re really confident around costs, that we can figure out what the business model is. The thing that I know having been a pre-IPO employee at Google is the search business that you have now is very different from the search business that we had 20 years ago, and so I really think we’re going to figure out what the ad units are, we will figure out what the business model is, and we have plenty of ability to do all of that profitably at Microsoft.
A: There’s so much value here, it’s inconceivable to me that we can’t figure out how to ring the cash register on it.
T: Right, and you guys have the luxury of figuring it out because Microsoft, you have plenty of other great business models in play, and OpenAI, you have funding from Microsoft.

A: What do you think of the product, Ben?
T: It’s good. Is it GPT-4 or is that not being disclosed?
A: I think the model numbers thing is a dumb framework anyway. The thing people thought, there’s been many versions of GPT-3 so it’s a better model, we need to figure out our naming at some point.