Unlocking Enterprise AI Through the Power of Open Ecosystems | The Six Five Summit

Hey everyone. Welcome back to the six five Summit. Daniel Newman here, CEO of the Futurum Group.

Very excited. This summit is rocking and rolling. We're here as part of the cloud infrastructure track, and I couldn't be more excited about this next conversation.

I am joined by Anil Nuri. Anil is the vice president and head of Intel's AI acceleration office, general manager, data center, AI category, and he's part of the Intel sales and marketing group. Now, that is a long title, and so I should get some points Anil just for reading that correctly.

But you have a big job at Intel. You have a big job. You're handling AI acceleration, you're handling consumption.

Uh, you're driving go to market as well as, as as marketing objectives for the company. You know, let's dive right in. I mean, it's a red hot topic.

Just, uh, you know, say hi to everybody. Welcome everybody. What's on your mind these days, ade as it pertains to ai?

I mean, AI is a super exciting topic, Daniel, and, you know, gen AI is transforming. I mean, every industry you can think of, right? Uh, customers wanna adopt gen ai, but they want to do it at a, you know, way that they can find it cost effective, scalable, and actually solve their business outcomes.

So, so that's where, uh, all the aspect of it is from Intel standpoint, I mean, the AI is in ingrained into everything we do. I mean, we are the only company, if you wanna think of it from the perspective of bringing AI compute to PCs to the edge, to the network, all the way to the data center. We have a full portfolio of technologies, products, silicon capabilities.

But even more interestingly, is if you think of this whole consumption of AI compute that we are looking at an exploding trajectory, uh, need to be manufactured as well. And with Intel's, you know, uh, foundry services and capabilities, we can actually also enable customers who want to build their silicon, uh, using our fab. So if you kind of think of it from that portfolio, we are able to address every part of the AI tam as, both as a foundry, as well as a product company, even though, you know, we may not be participating in all of them individually.

Yeah, you move really quick, and I appreciate that. You know, we talked to, um, you know, sash and, and, and Justin at, um, the recent vision conference, and you really did make an impression on me when it came to, you know, bringing AI everywhere. And look, you know, AI is in, its, its earliest days.

I know, you know, I love the sports analogies. Are we in the pre-game or the first quarter? Are we first inning?

Are we on the first tee? Um, but you could pick your sports and to some extent, you know, you're sort of seeing some parts of the AI market are moving a little quicker than others. Some products are selling a little faster than others.

But I wanna double click on what you just said, because I think one of the things that I get excited about, and I know sometimes, you know, the market, everyone wants to say, this company's the winner. I keep saying it hasn't been decided yet. There are early winners, you know, we maybe got runs in the first inning.

We keep with the sports analogy, but you guys are really focused on everywhere. So you said a couple things there. You talked about product portfolio, so from devices, you know, PCs to the edge, to the cloud, to the data center.

Uh, you talked about, uh, fabrication of founder. You guys are, you know, you're not just designing chips, uh, for compute, uh, for programmability, for, you know, asics, for for hardened ai, and as well as, you know, down the road. GPUs.

Talk a little bit about that kind of bringing AI everywhere focused, like double click on what you started to tell me at the gimme, that whole kind of portfolio look, uh, uh, uh, as to how you're approaching this. Yeah. Uh, so if you think about our PCs, right?

We've actually led the, you know, off the bat with our AI pc with our core ultra processors that launched last year, right? Um, in fact, we have shipped over 5 million units. We are gonna ship over 40 million units this year.

Just think about it. This is built-in ai, compute in your pc, and we are quickly following up with our next generation, you know, uh, you know, laptop processor, the lunar lake, uh, which will bring in even more AI compute. Now, these have on ship AI capabilities where you can keep your data with you ca you know, users can have their local, you know, AI agent or AI assistant, if you want to think about it.

Um, be able to, you know, do tasks on your PCs and enhance your productivity, uh, in enterprises. So there's clearly a lot of work around there on the edge platform. You know, again, this is where a lot of the inferencing is gonna happen on the edge.

You're not, you know, one of the things we, we have to think about AI is follow the data, right? Um, and if you think about the data itself, you know, um, a lot of the data that's created on the edge, very hard to go move them all over to the cloud. And so you're gonna have a lot of AI compute needed at the edge itself, right?

And so for that, we have our, you know, cyber edge, uh, platform, which not only comes with the, the, you know, fully validated, uh, with the silicon, with the platforms, uh, and with the software stack, right? You can actually deploy for edge use cases where there's lot of, uh, need both from an inferencing standpoint and also from a data ingestion standpoint. The third thing is the data center itself, right?

In the data center. You know, if you kind of think about where AI is today, like you said, it's a very early part of the earnings. There's a lot of rush towards, you know, creating new models.

And so if you kind of think about all this compute demand and everything that has gone up exponentially is about the race to build the largest models and the race to build the most, you know, useful models. And there's a lot of model development innovation going on, which requires a lot of compute, but the real thing that needs to be unlocked is how am I gonna use them and use them at scale? Um, and which is where inferencing at scale starts to come in, and that's gonna take a long process to be able to be effective and bring that business outcome thinking is how do I be able to actually solve the business outcomes and actually make it more productive, make it more, you know, uh, capable, whether it's in the context of, uh, you know, retrieving data or reasoning or generating new content.

And to connect all of these, you need a network, uh, that is open and scalable. And this is where it comes with the, yeah, ethernet based approach, a standards based, where we can actually have, uh, uh, you know, a, a multiple choice of vendors where there's a, you know, interoperable standards and, you know, ultra ethernet is a open standard that's been set up. Um, Intel's a founding member.

There's big name partners in there, uh, in the founding member list. And idea is to build the network that can actually scale for this future of ai. Yeah.

You know, Anil, when I when I hear you, I basically think, you know, there's a couple different ways we can cut this, but for the audience out there, right? That's kind of, you know, they're weighing and they're making decisions about the market, and they're basically saying, well, what's Intel doing? Intel has long been a leader in pc, long been a leader in data center compute.

Now we're in this kind of reset era, and now it's a new kind of pc, which, you know, and, and we've had some conversations here. We'll talk about a I PC's a whole nother session, but we'll definitely shown leadership early to market, lunar lake, uh, got pulled forward, some very compelling designs, of course, new competition, and that'll be weighed out as these get deployed. And then, you know, data center itself, you know, I often talk a Neil about, you know, the, the most inferencing still happens.

Our own future of intelligence data actually showed that it happens on CPUs. Now, that doesn't mean there's not a massive ramp of spend on GPU, and there's not the need for accelerated parallel compute, um, but a lot of people forget this. There's, there's still a really, really significant opportunity there.

And then there's the network, right? You gotta actually connect edge to cloud to the device, and then you need to actually have all these things tied together. She's talking about ultra ethernet.

There's a big debate, I call it the apple, uh, the Apple Android debate about what the network for AI is gonna look like. And by the way, it's gonna be huge because we're talking about up to a $400 billion TAM just for compute for, uh, AI in the next few years. I wanna have you touch on the open and closed thing that you just kind of alluded to.

I heard you talk about closed open, we talked about ultra ethernet. You know, this is gonna be huge. You've got companies that are basically coming out and saying, we do it all.

We're gonna offer you the whole stack. You gotta buy it all from us. You can't move from compute to compute, you can't move from open, uh, connectivity.

And some people like that. And then there's another side that's saying, lots of vendors, lots of disparity. Where is, where does intel stand?

Where do you see things falling? So it's actually a really good question, and I think you gotta step back in history and look at, you know, especially in the data center ecosystem, what was the real innovation engine, right? Uh, and if you think about the innovation engines in the data center, kind of, they're built on a very, very open, you know, set of frameworks, right?

Uh, start from the operating system itself. Linux was the one that's know, has the largest foothold in the data center. You kind of look at it in terms of, uh, you know, um, you know, the containerized, uh, uh, world that we live in, uh, Kubernetes, uh, that's built on a standard layer where you can actually have multiple, you know, uh, vendors, uh, play in, uh, you look at virtualization, again, a very standardized mechanics on, you know, how do I scale virtualization into the, you know, uh, data center ecosystem.

Um, as, uh, you know, as you think and get into the AI domain, now you start to look at, okay, how is AI getting, you know, uh, uh, leveraging the data center? It's actually, you know, PyTorch is probably the most commonplace of, uh, you know, abstraction point. If you ask any AI developer, most likely he knows he is coding on Python and, and running on a PyTorch framework.

And if you ask him what's under the hood, he's probably not even gonna know. And, and so, you know, where you get into the aspects of these abstraction layers where developers can actually broadly access, um, you start to look at it and saying, you know, the, in, in the AI world, right? In the gen AI world, especially with transformer a architectures, the dependency on something like a coda is a lot smaller than, you know, in A HPC or a high high performance computing, where they optimize every layer of the software optimization that needs to happen to get the best out of the hardware, right?

That's been more of an H-P-C-D-N-A, uh, in the ai DNA, it's been more time to value, time to results, and, and, and, you know, time to scale. And so they're more looking at it from a standards based as a bi touch level, right? Then in, when you start to look into these framing, um, you know, how do you build, and then when you look at models, uh, there's a proprietary models, right?

And there are a lot of open source and openly, you know, available models. Now, as you look at AI models deploying, customers want to be able to trust it. They wanna know what training corpus was used, what weights and biases were done.

And so customers are gonna be more looking at saying, Hey, especially as you get into the adoption of enterprises, they're gonna start looking and saying, Hey, can I trust the data that I'm gonna be training with? And can I train a patient on a model that I can trust the outcome from? And so that transparency is huge.

Security is huge. So being able to look at it from a aspect of how do we integrate this into an ecosystem where trust open compliance has always been a rich history in the data center domain, is how we philosophically look at it from Intel standpoint. And, and, and that's been our DNA as well is, you know, standards based approach.

And I see this is no different. As AI begins to scale, they're gonna look for, you know, frameworks where they can actually, you know, have these options, but in an open, standardized way. Now, it doesn't mean everything is open source, but clearly, you know, kind of modular and plug and play.

So, o Neil, I, I could probably drive this interview to an hour or more just having you answer all the questions. Unfortunately, we only have about 15 minutes, but I have one more, you know, question, question. We have a few minutes left, and I really want to hit on, you know, I know everyone out there, you know, I sort of alluded to early on about sort of the, the market conditions, the perspective that some companies have already won.

Uh, pat would never suggest that Pat Gelsinger absolutely sees the, the path to becoming an AI company. And, and I believe you, I've been on the record saying this. Um, I've been challenged though some of my other, some of the other analysts in the market will challenge things that I've said.

Media at times challenge things. I've said, um, your competitors are challenging some of this. Um, they're saying there's not enough validation that you're not winning enough customers.

Maybe it's at the pc, maybe it's in the data center. They're saying new architectures, new designs, new companies are gonna win. Tell everybody about the momentum that you're getting in ai, the wins that you're, you're achieving.

Because I think the metrics suggest that there is a lot of progress being made, and that Intel is well on its way to becoming an established leader in the AI space. Uh, Daniel, really good question. And, and before we jump into the customer traction, we kind of have to segment the market as well, right?

Uh, we haven't talked much about enterprises because this is a very important part of the segment. Uh, the reason for that is AI today has been primarily being, you know, created on openly, publicly available data that's like kind of web sourced and things like that. But we all know, and you especially, you know, would know this very well, is that more than 80% of the data both structured and unstructured, is sitting behind enterprise walls, right?

And, and so when you think about what do enterprise want, they want to have data compliance, data security. They want to be able to trust the models as we spoke earlier. They want to be able to be scalable, they want to deploy it at scale, um, and they want it to be accessible, uh, which means they wanna have vendor choice.

Uh, and last, and most importantly, do this at a affordable point. But most importantly, even if you do all this, they want an easy button. It needs to work.

They want to be able to have a turnkey solution, whether it's an inferencing solution, uh, you know, I'm doing knowledge discovery, a chat bot, or I'm trying to be able to create some generative applications. They wanna be able to have the easy button as well to go do this, right? So for this part of the, you know, work, we have actually worked with the Linux Foundation, and there's an open, uh, you know, um, platform for enterprise AI that was kicked off recently, uh, a month or so ago, to actually bring a structure, a standards approach to enterprise deployments at scale, right?

The second part of it is in terms of, you know, um, um, what do you do when we want try to bring about AI into enterprise? Think about, you know, this, 80% of the data has traditionally been, you know, hosted, um, through a zon server, zon cluster zon infrastructure, and X 86 CPU based clusters, because that's most efficient for doing database managements. Gen ai, as you saw that, you talked about this whole, you know, first wave, they, they're all being created on accelerators, GPUs, and so you're creating all these models that run pretty efficiently on gs, especially the large language models.

And so when customers are trying to connect these two, they're trying to figure out what's the best way to take data that is, you know, best hosted on Zion. But then I want to take the benefit of this gen ai. What's the best way to bring these two worlds together and do it at a very cost-effective and a scalable way?

Why do we wanna redo the whole follow the data, right? Why do we want to redo the whole data architecture and years and decades that they're spent on trying to get their, you, the, the data regulatory compliances done, um, and then move it all over to the side of the spectrum? So there's new capabilities like, you know, rag retrieval, augmented generation, and other techniques that are being now deployed.

Now we at Intel are trying to help accelerate that. And so when you look at from our zon, uh, you know, uh, you know, capabilities, we have a rich history of, uh, the, the data side of things. Um, and then, you know, with Zion plus an accelerator as a head node with, with Zion as a headnote and then Zion with our Gaudi, we are trying to help provide choice for our customers across all the three, um, and bring the best and the most effective way to, uh, you know, solve their use cases, uh, where things that run well on Zion, leave it there, things that run well on accelerators, keep it there and find a way to provide the compute needs to our customers on both fronts.

So Gaudi is the, you know, accelerator that competes with the, the, the GPUs in the space. And this is a lot has been asked about, and this is where, going back to your proof points that we're getting to, right? And so first let's look at the performance.

It's shows up at ML perf, uh, you know, we just recently announced some of the performance against H 100. You know, it gives 40% more time to train against H 100, which is the leading, uh, you know, GPA available in the market today. But the most important thing is perf per dollar.

3 x perf per dollar, uh, for inferencing throughput. This is what people care about. How many tokens can I process per second?

And, and at what cost is it gonna cost? How does it take to deploy it? And, and this, you know, and power, of course, how much power and power, of course, yes.

And being an accelerator, it's very power efficient as well. It's over two x uh, power efficiency for, uh, some of the critical workloads that yeah, you can go look at. Now, what is most important, like I said, is the value.

And we've actually going to be much more front footed on, on sharing that more publicly at, at Compute X. At Compute X. You'll see we are gonna disclose what are pricing guidance for customers to actually model their AI investments.

This has been a big problem. Customers don't know how much compute they need and how much it's gonna cost. And so we're gonna publicly share, in fact, we're gonna say that a Gaudi three, which comes in a kind of a UBB or it's like a form factor with eight cards in it, is gonna be at $125,000.

And a Gaudi two, which is already in market today, is at $65,000. So, you know, super Micro is basically selling a server today on gdy two at 90 K. Um, it's a amazing value, just kind of allows customers to be able to understand, you know, what kind of computer it takes, what per, per dollar you can get, and per, per dollar per what for those who really wanna look at it from a holistic view.

Right now, we've had a strong momentum of customers. So with Gaudi three, all the OEMs and OEMs you'll see at Compex are enabled, they will be available in volume, uh, by, you know, uh, uh, Q3 this year. And then as far as end customers, we worked with partners like Bosch, Airtel, na, one of the largest, uh, cloud service providers in, uh, Korea.

Um, you know, IFF it's international, uh, fragrances and, uh, fla, uh, flavors and fragrances. I mean, who would've thought there's actually protein, you know, AI models to generate, to generate, uh, food taste and, and, uh, perfumes and other kinds of things, right? Um, we have had, uh, significant traction with, uh, customer deployments.

We have our developer cloud where customers can come try Gaudi and then go decide, you know, how they want to go deploy. That's been overwhelmingly successful. And so, again, like it's a, it's part of a long journey.

We are at the early phase of it, uh, you know, from Gaudi two, we are getting to Gaudi three. This is a third generation of the Gaudi architecture, and we're super excited about, you know, what customers are saying about its, uh, you know, performance as well as its capabilities. Anil, I wish I could keep going.

I appreciate you breaking that down. I think those cost value propositions are gonna be very important. I think different architectures for different, uh, cases are gonna be significant.

I think people need to remember that a lot of AI can be done on A CPU, and of course, accelerators will be answering a lot of power challenges that we're gonna have. We are running outta power. So there are places where GPUs are absolutely the right architecture, and that will be determinist determined by the different workloads and use cases.

Anil, I'm gonna be followed up with you. We're gonna need to keep talking about this. This is not over, but congratulations on the progress work to be done.

Thank you. Yeah. And, And, uh, oh, I look forward to sitting back down and, and, and having this conversation in a year.

Yeah, thank you. And it's s been a fun discussion and really appreciate your time. Alright, everybody, thank you so much for joining us here for this six five summit session.

That was a great one. But stay with us 'cause we've got so much more. I'm gonna send it back to you in the studio.