Optimizing AI & HPC Workloads with Nvidia Partnership

Hey everyone. Welcome back here. Text Drunk tv.

You know, I, I wanna introduce you to our next guest. It's his first time here on Text Drunk tv. So let's say hello to him.

His name is Bjorn Holin, and Bjorn is the, uh, president of CIQ Bjorn. I, I forgot if it was Holin or Holin, correct me, sir. If I'm wrong.

It, it's Holin. You're correct. Okay.

Bjorn, welcome to Text Drunk tv. It's great to have you on here. Um, as I mentioned, you're president of CIQ and from what I understand, that's kind of a recent development, uh, the new title, but you've been with CIQ, but give people kind of a sense of your journey, Bjorn.

Yeah, absolutely. First off, thank you for having Alan and, and great to meet you. Mm-hmm.

Um, so I've been, um, in and around Silicon Valley for about 15 years now. Um, I came to the Bay, um, and then I started a couple of, uh, tech companies early on and then, um, sold one of those and then ended up running product at a couple of larger companies. Um, worked at a, at a firm called Bowers and Wilkins out of the uk.

Uh, and then I moved into the investment side of the house. So I ended up running a family office for a, a high net worth individual out of Silicon Valley where I oversaw all of the media tech and, uh, and other investments for, for that individual. Um, and so through that, um, have done a lot of work in venture, um, worked, uh, in, in various capacities there.

And one of my early investments was at CIQ. And so I ended up, uh, joining CIQ about, uh, two and a half years ago, uh, full-time, um, with the goal of kind of scaling up, um, infrastructure for the AI era. Very cool.

Uh, a lot of people, well some people may have heard of CIQ and, and maybe they know it from the Rocky Linux distro that you guys do, but give us the big picture of CIQ Bjorn. Yeah, absolutely. Uh, so CIQ is focused on reinventing the enterprise IT stack for the age of ai.

Um, the core thesis of the company, and actually the reason that I invested early on was the idea that over the next decade, the vast majority of workloads are gonna be performant workloads, whether that's AI ml or traditional HPC and existing IT infrastructure is just fundamentally unprepared for that transition. And so we are focused on reinventing a stack from the bottom up and the top down. Um, and so at the bottom we are the major, uh, uh, founding sponsor of Rocky Linux, uh, which many of your viewers will be familiar with.

Um, and then we have optimized versions of Rocky, um, that we sell. Um, and then, so we have Rocky Linux from CIQ hardened, which is our, um, security focused Linux distribution. And then we have Rocky Linux from CIQ ai, um, which is our performance, uh, Linux distribution.

Um, we have a, a product called a sender, uh, which is built on top of a WX in the middle of the stack. And then, um, at the top we have a product called fuzzball, which does high performance compute orchestration taken together. Those products are focused on, um, reducing the friction for building and scaling AI and HBC workloads and kind of re and changing how organizations build and grow those pipelines.

Excellent. Um, so I, I just want to make sure our audience and our audience probably is familiar with this 'cause they're very open source savvy as well, right? So Rocky Linux is an open source Linux distro.

Yes. Right. And, and CIQ is the, was the main benefactor, if you will, of, of Rocky coming out there.

But CIQ has, I don't want to use the word proprietary, but commercial distros of Rocky that are geared towards certain, you know, uh, certain use cases such as a, a, you know, high security, kinda setting hearted and so forth like that. Does that make sense? It does, yeah.

And so to, so to, to expand on that a little bit. So Rocky Linux is maintained by the RESF, um, and it is mm-hmm. It is completely open source.

Um, and CIQ is one of the major founding sponsors of the RESF. Um, now CIQ has optimized versions built on top of the open source, um, community version. Now those are all still open source as well.

Um, they, CIQ is an open source for company. And so Rocky Linux from CIQ, which does contain modifications in enhancements, still is an open source project, but it is maintained by CIQ. Excellent.

All right. I think, I think we're got, we got that. Let me ask you a question.

Beyond Rocky though, beyond, you know, the Linux distro, what else is CIQ about B? There's an AI mission here, right? It's, you mentioned there, it's not just ai, though.

AI is obviously the topic of the day. Um, for us it's about making performant workloads accessible and optimized. Um, one thing that we're seeing in the market today, primarily driven by AI, is that productivity is, uh, skyrocketing, but costs are skyrocketing as well.

Um, and one of the big reasons for that is there's an enormous amount of waste and inefficiency in how systems get built, and you end up embedding that over time into your infrastructure. And so one of the things we talk about a lot internally is that today AI is very focused on human efficiency. How do you augment and enhance or in some cases replace the work that humans do?

Um, we believe that the next generation of that is going to be machine efficiency. How do you make infrastructure much more scalable and efficient? And that is really our core mission.

Got it. Um, if you don't mind, I want to kind of pivot and talk about some recent news Yeah. Outta CIQ around working with Nvidia.

Um, well, I, you know it better than I, why don't you tell, tell us what it's about. Um, yeah, so we, we recently announced, um, a, an expansion of our, our work with Nvidia. So CI Rocky Linux from CIQ is the first Linux distribution authorized to, in addition to redistributing, uh, the Cuda and the Cuda toolkit, um, the doca ed drivers from, from Nvidia.

And the, the goal of this partnership is, is really simple. It's to make it easier and faster to run and deploy, uh, machine workloads. And so we believe fundamentally that engineers want to focus on building their AI applications, not on debugging, uh, kernel integrations with the, with the operating system.

And so we are able to redistribute that entire stack and have a much faster time to first inference. We've seen as much as a nine x production in getting set up and running your first workloads, bringing hours down to minutes. Got it.

Um, your, I, so again, our audience, they've, they've unpacked enough Linux to know we're talking about these are packages, right? That did they come default now in, in the, uh, CIQ, Rocky Distro now the, these packages or they're still optional? No.

So you, you can choose not to install them, um, but if you, and, and not download them, um, at, at runtime, but yes, if you want them baked in natively, um, that is one click away. Let's talk a little bit about cruda. Right?

We, we spend a fair amount of time here talking about it. There are parts of it, and, and again, you probably know better than me here, brn, but there are parts of it that are open source that they clearly open sourced, but then there are parts that, I don't know how to say this. They, they're not licensed open source, though they may in fact be quote unquote open.

Do, do you follow what I'm saying? Yeah, I do. How, how do you explain that for our audience?

Like there's a distinction there For certain packages within the CUDA toolkit that are fully open and redistributable, and then there are others that require, um, explicit partnership and permission and licensing from Nvidia. Um, now I don't, I don't wanna speak on, on behalf of Nvidia. I imagine that's a, a user experience, um, decision.

They wanna make sure that they, that when you are using co cuda, it is done in a way that is in line with their expectations. And, you know, I think Nvidia has done a very good job of opening that up over the past several months. Um, uh, CIQ, Rocky, Rocky Linux from CIQ, uh, along with, uh, two other Linux distributions, uh, was able to redistribute, uh, CUDA in partnership with Nvidia.

We've now expanded that into doca ed and I think one of the major focuses from both organizations over the next, uh, couple of quarters is going to be how do you lower the barrier for use, right? How do you get up and running quicker? And also how do you make sure that what you're running at scale is validated, tested, and secure?

And so from that standpoint, that really can only come through deep integrations at the code level and at the package level. Got it. I mean, you know, the, the point I was looking to make there was that, 'cause someone out there may be saying, oh, but I heard Cooters open source, what, you know, what's the big deal about it being included in CIQ?

I could get it, but there were parts that you can't just download right from, from anywhere. Yes. Um, let, let's dive into doca ed a little bit.

How would you describe that to the audience? So Doca ED is, uh, NVIDIA's replacement for their previous melanox ed drivers. And so essentially this is the core component of their data infrastructure and networking component.

So it allows you to go deeper into the NVIDIA stack and program and work with their data center infrastructure. Got it. And you know what, just for the folks like me, a little slow on the uptake, what does offed stand for?

Bjorn? That Is a good question. It is, um, open fabrics enterprise distribution.

It is, it is, it is not often, uh, actually, uh, set up like the full, the full length. But yes, that is what it stands for. Excellent.

Caught you. I, you know, Hey, I, I thought you would, you know, if I didn't know it, you certainly would, and you did. Good for you.

Yeah. You know, let, let's do some housekeeping beyond, beyond CIQ. Like for people who maybe want to download or at least look at some of the CIQ download or, you know, distros and downloads and so forth.

What's the, is it a GitHub thing? Should they go to the CIQ website? If so, what is the CIQ website?

What, where, where should we go? com. com.

The main, the main site there. Yes. Go from there.

Very cool. Um, you know, there are other players in this AI market that I am trying to say this, uh, diplomatically that are a little nervous about NVIDIA's position vis-a-vis Cuda, while also being the main, uh, GPU provider. And so there's a lot of talk of other cuda, like, if you will, toolkits and stacks that, you know, will allow people to build on, whether it's from public cloud providers or, you know, some of the other layers here.

Any plans in, in terms of support for that with CIQ or Kuda is you're, who is the horse that got you here and that's what you're writing? Writing? No, we are very much, I'm, I'm glad you asked that.

We believe sim kind of similar to our open source roots. We believe in kind of meeting our customers where they are. Um, mm-hmm.

We are proud to partner with nvidia. Um, we are obviously excited to, uh, be able to talk soon about some of our other partnerships that we're working on. Um, but we believe that it is not necessarily a winner take all race in the future, and we wanna make sure to enable customers on whatever hardware and software stacks that they choose.

I love it. Bjorn, I want to thank you for coming here up on techstrong TV today and explaining this all to us. You know, some of these as, as I was, I was mentioning off camera, there's nuances to open source, there's nuances to distros and, and open versus open source and licensing.

com days, right? Yeah. So I, I appreciate it, continued success with CIQ and, and come back.

You, you, you know, you intimated, you have more partnership of announcements coming soon, so you've got an invite to come on and tell us about 'em. I appreciate that, Alan. Wonderful to meet you and thank you for having me on.

All righty. Bjorn Hovland, president of CIQ. Go check them out.

com. Um, and you know, Rocky Lennox is, is obviously something that's made their mark in a relatively short time in the industry, so they're doing something right there. We're gonna take a break here on Tech Trunk tv.

We'll be back in a minute.