AI-Optimized Cloud Services with Crusoe’s Nadav Eiron

ai video series. I'm your host, Mike Vizard, today with NAB Iran, senior Vice President of Engineering for Cruso Cloud. And we're gonna be talking about what it is that is different in terms of workload requirements in a cloud service that you wouldn't see somewhere else.

And this is in the wake of them raising, uh, that at least recently, an additional $600 billion in funding. But, um, not all workloads in the AI space are the same. So what do we need?

What do we need to think about? Nadav, welcome, Michelle. Thank you.

Thank you so much, Mike. Happy to be here. So walk us through this a little bit because, well, on one hand there's training and then there's inference, and then there's all these GPUs, different flavors of GPUs, and of course now there's all these alternative AI accelerators of lions and tigers of bears omi.

But walk us through this a little bit in terms of, well, what does Cruso do that's different from everybody else, and what do we need to think through in terms of infrastructure? Yeah. Okay.

That's, that, that's a wonderful question. I, I can take it, uh, one of several directions. Maybe, maybe I'll take it in more than one, because I think there's different ways to answer the question as well.

So first you start by talking about the workloads and, and obviously, um, ai, uh, or, or AI as we know it today, you know, post, uh, uh, GPT and, and generative ai, um, is a different kind of workload, primarily because of the requirements of compute that are kind of like, you know, several orders of magnitude different than what we know in classic compute. And so, what, when you ask what's unique for AI workload from the perspective of a cloud provider, obviously we, we are optimizing for that kind of consumption, meaning that, you know, we focus on accelerated compute that's designed for AI with the sophisticated network that goes with it, um, and with the software that allows you to deploy your workloads on it. Um, so that's kinda like what differentiates an AI cloud.

And, and Cruso Cruso Cloud's mission is, um, build the world's favorite AI cloud. So when we talk about an AI cloud, we're talking basically about an infrastructure, um, service where we know that AI is what matters. So we're focusing on those kind of workloads, you know, versus a database, a web frontend, et cetera, et cetera.

Those are not things that we're optimizing for. When you ask about the differentiation for Cruso Cloud, the thing that we talk about a lot is vertical integration. Um, cruso is a unique player in the sense that our vertical span in terms of the stack that we build, is a lot taller or, or deeper depending on where you stand, um, than than many of, uh, the other players in the market, right?

So we start with sourcing energy and building data centers and, you know, sourcing energy in a climate friendly way, et cetera, et cetera. And we wanna go all the way to providing really value to end users in the for of, uh, AI specific experiences that go beyond just the infrastructure. And at every layer of the stack, we're looking to do that through partnerships with, with people that we believe are best in class, have unique technologies, and we think what we bring to the table is A, the ability to build this amazing infrastructure.

B, the ability to bring together innovators up and down the stack into a single ecosystem. And c, the ability to run all this 24 by seven, scale it up, um, and really provide a service that, you know, the most demanding kind like enterprise class customers can and will be happy to use. We hear, of course, GPUs and AI are kind of joined at the hip and there's a scarcity of GPUs.

Is that a gonna get any better? And some of the folks I talked to were like, well, it doesn't even feel like a cloud service anymore. It feels like managed hosting.

'cause I gotta make a year long commitment to something to get access to the GPUs, and I don't even know if I can, 'cause I'm still experimenting with my AI workloads, so I'm not sure I can commit. How do we navigate all this? So, uh, another excellent question.

I think, you know, to to, to the point of whether it's gonna change, the answer is absolutely yes, right? Like this AI landscape is very nascent in its nature. If you think about it.

Um, you know, again, we're talking about technologies that, you know, two years ago, three years ago, uh, a select few PhD students were dealing with, and all of a sudden it's the talk of, uh, everybody and their cousin on the street. Um, so obviously this is a quickly changing landscape. Uh, every week there's, there's some other twist in the story.

So definitely there will be change. I think you're correct that, you know, uh, today's landscape, or maybe it's yesterday's landscape, is very much of focusing on the GPU, focusing on the hardware. In my mind, this is a reflection of this heritage of AI having been primarily a research, um, pursuit, uh, in, in the not too distant past.

Um, so if you look at a lot of the work that's being done today, that work is exploratory in nature, like you mentioned, you know, it's basically applied research being done. And when you do applied research, not only do you not know what will happen in a year, you need a very different kind of infrastructure than when you're trying to make money and change the world, uh, you know, physically by giving billions of people something new to do. So when, when you're in this research mode, you want to have full control of your infrastructure, you want to try things, so you don't want anything to be abstracted away.

You want to have access to the, the, the leading and bleeding edge of the technology. Um, when you switch more into a posture of like, I'm here to build a product, whatever that product would be, um, and AI is a technology that helps me make that product better, but I'm here about making the product, then you switch into a posture of like, okay, AI is a means towards an end. It's not an end in its own right.

I don't want to be on the leading bleeding edge because I don't want to have to pay for, you know, hundreds of PhDs who are super expensive, who like spends their time in labs. I wanna focus on my product and I want technology that's easier to use, more predictable, um, and easier to integrate with. So, so I, I think we're on that journey.

Uh, the infrastructure we have today, to a large extent reflects where we are, where we were in the, in the recent past, but I think we're quickly moving to towards that world where the focus is on delivering value through products. Um, and we have to move there because of the immense investments, right? Like, you know, all these billions that get poured into AI infrastructure, the investors are like, okay, where's my return on investment?

And that has to be with products that people use day in and day out. What are, are options gonna be going for it? 'cause right now everybody's kind of wrapped up around Nvidia, but a MD has some options out there, and there's all these other AI accelerated chip announcements.

Is there gonna be more diversity? And do I need to figure out which of these is optimized for what type of AI workload? Or are they all simple?

So I, I think, uh, my answer to that is, uh, uh, as part of that journey that I, I just talked about, uh, I want it to be my problem to figure out which one is the right one for, for each workload and not you. Right? So, uh, and, and if you think again, uh, of the journey that cloud has made in classic compute, we've gone through this journey, right?

Like nobody as a cloud consumer asks, like, is my website gonna run on a a MD chip or an intel chip or an arm chip for that matter? Nobody cares anymore. Uh, from, from that perspective, it's, it's fully commoditized.

All you care about is it's available, it's cheap, you know, it's reliable, right? Um, again, as we're moving away from doing research and people trying to twiddle the bits, you're gonna see this layer of abstraction. You already start seeing it, especially in inference.

A lot of, a lot of use comes through an API that's basically, you know, the OpenAI API has become a defacto industry standard. If you wanna do like LLM inference, that's, that's what you talk and nobody, you know, cares what kind of hardware is behind it. I think this is good both for the consumers and for the suppliers.

It's good for the suppliers because if you're trying to innovate on the chips, having a stack where, you know, you only need to change one place in order for your chip to now gain billion of users, that's, that's good for you. That means that there's a, a lower, uh, uh, a lower obstacle on front of the adoption of unit technology. It's also obviously good for the consumer because the consumer gets to choose whatever is cheapest and best today.

And, and, and we avoid the lockin. So as part of that journey of AI becoming more mature, I think this question will become less interesting for people and more interesting for people who build infrastructure. But those are select few, and we can, we can be the ones that really dig deep into it and provide that solution as a service.

There is of course, a lot of interest these days in ways to train and possibly deploy models at a much lower cost. We've seen this conversation around deep seek and, um, I think Alibaba made some claims, and similarly, but I'm curious, are we gonna be more efficient in the way that we train ai or can we use other classes of processors? 'cause I feel like it's been early days in the history of tech would suggest that we will get smarter about how we consume infrastructure and the cost of infrastructures will drop.

So are we on some curve that may be a little more accelerated, but it's the same curve we've always seen? Yeah, I, I, I think you're right. And I think I, I look at it from a different perspective, which is, uh, I'm a believer that, you know, AI is a big revolution.

It's gonna change the world. And for it to do that, it has to be more accessible, it has to be cheaper. Um, I think we're also seeing a shift away from focusing on large free training, you know, batch kind of workloads.

Like the, there's a trade off between how much work you do at training time and how much work you do, um, at, at inference or at use time. And I think we're, we're starting to see that balance change a little bit. And I think it's a good thing, a because I don't like it, or there's only one right answer to the question of like, how much effort you should use to train versus, uh, versus use.

Um, so having choice is a good thing. It's also good in my mind because it allows you, uh, more easily to tailor the technology to new use cases. And again, this is something that we know in the past from technology, when we build technology, we don't always know how it will be used.

And so allowing more people, um, to be able to experiment and try and see what works and see how it applies in the field, maybe, um, that the person who originally designed technology is not familiar with how that works, that that is a key component of successful innovation and, and the economic impact of those innovations. Again, it's, it's part of moving, moving the action out of a select few large labs with, you know, PhDs in, in, in white lab coats, into the hands of people that deliver the end product experience. And so I see that journey as a very positive one and one that we have to go through.

If AI is successful, as we kind of play with all this stuff, um, energy keeps coming up as part of the conversation and the cost of energy. And some folks are doubting whether or not we'll have enough energy for all this AI capacity by 2030. And, um, how, how do we solve that problem?

So I think there's, there's multiple ways to solve it. So again, bruso does have expertise in sourcing and developing energy sources. It's, it's not my personal, uh, part of the company, but I'm, I'm sure that we'll be happy to, if, if you and, and the viewers are interested, uh, Peria who's someone who actually works on how do we develop, uh, you know, more sustainable, uh, more available, uh, energy sources.

I think from my perspective as, as, as a technologist in, in the software side, I think part of the answer is, again, matching that energy consumption to the value that you generate to, right? So if you really have a way to make the world three times more productive than it has been without ai, then all the spending that we, that we spend on, on energy is justified and will easily have the resources to do it. Because, you know, GDP will be three x.

So as a percentage it'll be so much smaller. Um, so that's where my focus is, how do we make AI be a real game changer, um, in terms of bringing economic value into society? Um, and then you rightsize your investment depending on what you get out.

I think, again, the worry that we see today is a little bit, because we're at this nascent stage where we're putting investment into basically understanding the technology and how to extract value from it. But we're not yet seeing the full, you know, fruits of our labors there in terms of the, the value actually being generated. I wanted, like, AI is so beneficial to humanity that it's like, okay, we'll figure out the energy source.

'cause we have to, that's, that's, that's the future I'm working towards. So to that point though, in the short term, do we need to be smarter about what workloads we're prioritizing? 'cause sometimes I feel like, um, maybe the fact that I could write an email slightly better doesn't quite justify the cost of that thing versus standing in the way of somebody doing some major healthcare research.

I, I, I think that's a fair, that's a fair point. You know, um, with capitalism, we, we hope that that money reflects those decisions and, you know, you're not gonna be willing to pay for, uh, for, you know, improving your emails, uh, as much as, let's say a drug manufacturer would be for like inventing a new cancer drug. Um, so that, that's one instrument that we have, um, to, to control that.

And again, I think this is where variety comes in and having different entry points and different ways to consume helps, right? I think there's also the point of, as we scale up, um, the question of what is good enough, and nobody wants, nobody wants to kinda like, you know, use not cutting edge technology. But the fact of the matter is, the cutting edge is really expensive, either in direct monitoring investment or another trade offs that you make.

And for a lot of these products, you really want to be one wave behind the bleeding edge. And a again, we know from the history of technology that there's a huge cost, uh, you know, uh, a hockey stick curve right as you get to the edge, right? Like the, the, the latest and greatest is always a ton more expensive than what we had before.

As technology matures, the difference in what you get from one generation back versus the latest and greatest reduces. And so a lot of the use case can be, uh, uh, satisfied by not the latest and greatest, but just what's behind it, which will be a lot cheaper. And lastly, we talked a little bit about, you know, this abstraction of the physical infrastructure of the hardware, of the chips, et cetera, et cetera.

As we do that, we also allow more specialization in the AUG grid, right? Like, uh, today the majority of use cases use one architecture, right? Like a GPU 90 something percent of the market is Nvidia GPUs.

Everybody uses a hopper architecture, and next year everybody will use a black or architecture as the barrier to entry for having different kinds of hardware gets reduced because of what we talked about before. The hardware is abstracted, and you can, you, you know, you can have a, a path to the market with less investment. It'll become, it'll, it, it'll be more profitable to build chips that are narrowly focused at a particular part of the journey.

And that will gain us, uh, benefits as well, so we can specialize on the up and coming next thing. And we see that happening in the marketplace already. Last time I checked, I thought $600 million was a lot of money, but in the age of ai, it's hard to determine what is a lot of money these days.

But what is the plan for that spending and what are you guys looking at? So, uh, again, I'm, I'm, I'm here to talk about technology and not finance and, and I'm sure, uh, you want to talk to, to people who actually, uh, you know, uh, count the pennies on the dollars. Uh, we be happy to oblige to, um, we're investing in, like I said, building the world's favorite AI cloud.

Um, so that's where the investment is going. And, and as I touched on today, those clouds are very much about exposing all the gory details. I would say stay tuned in the future, you're gonna see more and more higher level services, things that are easier to consume, um, that are easier to translate into something that brings value to society here and now versus, you know, three layers removed from that.

That's where we're focused on, uh, you know, in terms of the software side of right folks, you heard it here. We may be in the Stanley steamer age of AI compared to what it's gonna be like in the near future, and it's gonna be a lot easier for all of us. Hey, naab, thanks for being on the show.

Thank you, Mike. Been pleasure. All right.

Thank you all for watching the latest episode of the Text Drawing AI series. You can catch this episode and others on our website. By all means, check them out.

Until then, we'll see you next time.