Day 0: Designing your AI data center with Juniper Networks

Welcome everybody. I'm Kyle Baxter. I'm gonna be talking about how you design your AI ML fabric with ra.

And so many of you aware of probably challenges that, that are out there. Um, there's the AI space is moving so fast that there's a demand to deploy it now. And the challenge is, is it's the network, the, the, the industry is moving so fast that there's not enough adequate trained resources out there.

And when they are, there's hundreds and hundreds of pages of this documentation, how to configure this or that, or, or this port protocol. And there's no time to read up on that. Become an expert.

And that's where apps can come play and be your expert assistance to help you deploy. Um, and so when we're talking about deploying, we're, we're gonna be looking at rail optimized designs. We've covered this in some of the other sessions on why rail optimized designs.

This came from NVIDIA designs to, to make the most outta the network, to get the most, most throughput, reduce latency, reduce congestion, all of that, get all those benefits. But when designing it, you know, how many of you could design it and say, you know, gimme a, gimme a design and gimme a cable map right now. Right?

But with abstra, I can show you how to do it in five minutes or less. Who wants that? Yes, I do.

And so we start with a simple, simple template designer where with just a few basic inputs that, that, imagine as an architect you are looking at this, you're probably gonna know all these, you're gonna know how many servers I want to use, how many GPUs per server. That's gonna be eight, but maybe you don't wanna use all. Um, but that is configurable.

Um, how many servers per stripe? What is your GPU Nix fee? What is your over subscription ratio on?

So pretty simple things that any architect will know and they can put this in the input and it will tell them exactly what are your design options based on those inputs. And it could be different things. You can see there's some that are using, you know, PTX 10 K sixteens as as leafs, or do you wanna QFX 52 40?

It gives you options that say, depending on what devices you want to use or how many of them want, you can see some of 'em have, can be different quantities potentially. And so you can pick the devices that make the sense for you that match your design and you can preview that to see what is that gonna look like, how is that all gonna be connected? So they can say, yep, that makes sense.

That's the template design I want to use. And then once you save that design, you can have a list of saved designs. You can see I have different designs.

I have a thousand GPUA 4,000 GPU and 8,000 GPU, some collapsed fabric that are smaller, like 128 GPUs. I have different designs that I want to use that I can then take those in in seconds. I can deploy them and have them ready to go.

So it only takes me just a few minutes to to, to design it and deploy it. But then what do you need to do next? Now what?

Like that's great. You built me a template. Um, you know, you, you pick the devices I need, that's great.

But what's next? Well, the next thing you're probably gonna wanna do is you're gonna wanna have a cabling map to know how to connect all these things. Mm-hmm.

Because there's a lot of connections when you're talking about, you know, like a 4,000 GPU cluster. There's a lot of connections. And so what we can do in ASTRA is even before you have hardware that shows up, before you've ordered anything, you can, you can build that design in abstra without the hardware where you can have that cabling map and you can see what it is, you can export it whether you want it in JSON or CVS Cs CSV, not C-V-S-C-S-V, um, depending on how you want to use it.

And obviously it's gonna be a big table 'cause it's a lot of things. But it allows you to take that caly map and then give it to whoever's going to be plugging in the harder when it arrives. So you have that full design, but even if somebody, you know, it goes in, doesn't plug in Exactly Right.

Even afterwards, once it is connected, we can use LLDP to see did it actually get plugged in the way it was intended? 'cause we, that's the whole intent based networking design we have in RA is we validate and continuously validate that the running network is operating as intended, including cave map, including config drift, all those things we can validate and continuously validate so we can make sure that kayley map's. Right.

Will you also like put up a picture of the switch and highlight the port and say This one's wrong? Yes. Thank you.

Yes. We do that. We, we have the, the kayley map and we can show exactly which ones are wrong.

Perfect. Um, and show the mapping of hey, you know, this spine, this leaf or this leaf to these GPU servers. We can map all that and show all that.

Absolutely. Excellent. Yeah, great question.

So that cabling map is, is really nice as well. But the other thing we can do is we can look at the configuration for those that still love looking at CLI and don't trust that automation is gonna deploy what they expect. You can go look at the CLI and you can go look at the config and see everything that that is generated.

Um, and then back to our multi-vendor point is we can generate for multi vendors, so not only just Cisco, but we can generate it for other vendors as well, including, um, Cisco and Arista and sonic based devices. We can cover those. And so it allows you to see the cabling map, see the configuration all before you ever ordered a single piece of hardware.

So let's see a little demo this. So I'm gonna show you here in just a couple minutes how we take an input on how many GPUs we have, build a template, deploy that template, get the cabling map and we're done. I mean, that's amazing.

We can do that in five minutes. So let me go over here and I'm gonna play, uh, a little video so that way we can, um, walk through it. So I'm going to go to ra, I'm gonna go to templates and there's a create an AI cluster template.

And so just like we talked about earlier, I can have inputs and automatically updating on the top. You saw the number of GPUs, automat changes, I added different settings. And then I can see in here the different combinations.

So again, you can see some different combinations of, if I want to use, you know, one of the PTX chassis mm-hmm. I can reduce the number of devices, but if I don't want to buy a chassis, I could buy, uh, 52 40 from the QFX line and have those Question Yes. Uh, Brian Martin signal 65, um, you have servers per stripe.

What's a stripe? So, so simply a stripe is, is a collection of, of rails. So it's, it's how many do you want in that, in that stripe to make it up?

'cause that rail is, is a single combination of all the, the GPUs connected. Mm-hmm. And then a stripe is, is a combination of the rails.

Okay. So that's basically the number of ports that'll be used for a given rail. Mm-hmm.

We go with the servers. Yes. Okay.

Thank you. Yeah. Yeah.

Excellent. Thank you Brian. So now that when I decide I want, hey, I wanna use all 52 forties, I don't want to use a PTX for whatever reason, I maybe don't wanna buy a chassis.

Um, I can look and see here and I can hover over and see how everything's all connected. Um, it's a lot of connections 'cause we're talking about a thousand GPUs. So there's a bunch of servers in there that are connected, but we can see how it's all connected.

We can see then here the number of spines, the number of of rack types we have, we can see the devices. It's picked, it's 52 forties as we expected. So then all save it, give it a name and we'll save that template so that way we can use this to rubber stamp and just repeat the design over and over if I wanted to.

Um, and so we had that design saved. So now I'm gonna go to my blueprints and, and after terminology, a blueprint is your, your your design of what you want your data center to look like. So just like a blueprint of your house is the model of your house looks and gives you all the specs.

A blueprint here in the data center is a blueprint of how you want your fabric to look and operate. So I'm gonna give it a name and I'll pick my template. So like I said, in just a few clicks, we can deploy that template and look, it looks exactly like what we had in the template, no surprise.

Um, so it matches exactly what we built in our template. And we can now go quickly in just a simple button create and it's gonna go scaffold that whole design for us. It's gonna use some default pools of like IP addresses and ASNs.

You can configure that to however you want. Um, but it defaults to some default pools. So that way you have something to start with.

And then we can go right into our staged area. 'cause in apps terminology, we have the concept of stage and active as the names indicate active is what you actually have deployed. Mm-hmm.

It's what's active stage is to where you can play with the design, see what it's gonna look like. And this is where we can look at things like the cabling map and the configuration all before we even ever have a single piece of hardware. And so we'll then look at the links, we can see the, the cabling map.

Um, we can then export it like we already saw and talked about into CVS or JS files. We can see exactly what interfaces everything's connected on, what IP addresses they'd be used. So it gives basically an exact blueprint here that we built in just five minutes and say, Hey, I'm already ready for building, you know, a thousand GPU server.

Here's the cabling map. When you're ready to order the the hardware, order it, plug it in as I specified, it's just gonna work A question on the back backing up a second. Yeah, please.

Have I already made some decisions on the racks and all of the DGX systems that I need in this particular fabric, right? Because if this is from the network operator perspective. Yeah.

How have I ingested what systems I'm gonna use in this fabric? Or are you saying I build the fabric before I decide what DGX systems I'm gonna plug into it? Where's, where's the, the deciding factor of how I build the fabric?

Mm-hmm. Mm-hmm. So we're focused more on the, on the networking side obviously, or networking company.

But um, but we do model also, and we do have agents that live on the GPU servers and that's how when we get to it, we'll show some of the, the metrics we get on the, the nick and the GPU and, and we can monitor that. Um, so we, we model it as what we call like a generic system. So that way it's anything that that can, that can fit that kind of idea, which is needs to be, you know, some kind of, it's like all of them are like AUN two servers.

So the agent lives on that Ubuntu server and we can then connect to it. Um, so really what you're looking for is what is the nick speed that you need to get there? Um, beyond that, whatever, you know, DGX, you know, server you have or GPU server you have is is transparent to that, that process.

So there's some sort of management plane connectivity, right? Mm-hmm. To those systems that I'm ingesting here.

So I've got it like a data feed perspective. Yeah. So when you, when you actually get the hardware, you need to then assign it in, in, in ABSTRA and say, Hey, this for like the, the switches, this serial number matches this leaf that I want to use, corresponds to this point Correspond.

Yeah. And then the same thing with the GPU servers. You correspond, hey, this one, you know, goes to this location in, in that template design.

So there is obviously some assignment that you have. I've got The GPU operator design as well that would kind of mirror. Mm-hmm.

Perfect. Thank you. Yeah, No, good question.

So while you're configuring, are you also then choosing which of those many, uh, load balancing techniques we just learned about that automatically default to the, the RDMA aware one or what? Yes. Yes, we do.

And we'll get definitely more into that. Um, but, but yes, we do have load balancing configuration. Um, so the, this first section was really all about how you get the initial design built and then into a cabling map and be able to see the config.

So that way you're, you're ready to order and start, you know, plugging in. We'll get right into in the next session on, on how we do things like load balancing and how we do assigning virtual networks, all those things to, to actually get into some of the, what we call like the day one where you're starting to, um, get into not just the, the design but into kind of in more on the deployment base You mentioned. So go ahead.

Rinky. C, D, C, um, just curious are we're here basically looking, right? GPUs are the problem in this situation, right?

But are we also going to take a look at, well I've got this database server, I've got a plugin, I've got mm-hmm. Storage. Mm-hmm.

I've got other, you know, possibly a public mm-hmm. Network that I need to plug in. Uh, does this, does appra also handle the security aspects, you know, and, and, uh, things like that.

Yes. So, so, so back to my point earlier, for no matter what kind of network it is, we can, we can manage it. We're focused a lot today because it's AI infra field day Sure.

On how we would deploy, you know, the backend training, right? That's what's new. But, but we've been doing it for years of any other kind of network, like you said, storage network or even, you know, front end inference network, which follow a more traditional design.

Gotcha. Um, that's not, you know, not always the rail optimized, you know, kinds of designs. So absolutely we can do that.

And you can use it in the same single ster instance. Um, you can have multiple blueprints for like your, your backend, your front end, your storage, your, and so you have, and then you can move around just in your single UI and be able to see that and even do things like data center interconnects when that's necessary. That was my question.

Yeah. Yeah. And then on, on your point on security, you know, we have security policies that can limit, you know, what do you want to talk to what or not talk to what, and then we can even look at flow data to see, you know, is there potentially something that's deviating or looks malicious inside the network?

So we talked a lot in the past session about, you know, how do we, you know, prevent some of that stuff, you know, going in and outta the network. Okay. But in the network we can look at east west traffic and to see, you know, is there potentially something malicious going on to inspect that by looking at some of the, the flow data, like, like s flow packets and things like that.

Got It. Okay. Okay.

Does this, um, yeah. Take into account the specific type of nick you you're using or what your interfaces are going to be? Or is that just assumed that, you know, what that is In?

It does take into account some, we're not doing nick configuration at the moment. Um, so we're more looking at getting metrics and information from the NIC so we can look at things like congestion and at a sequence. And, and then also the health of the general server.

Um, NIC configuration is something we're, we're looking at in the future Okay. That we can get to. So, so in that when it gets to configuration yes.

Then that would obviously matter what kind of NIC it is. Um, but um, but when we're looking at, you know, all the, the NVIDIA GPUs, which is the, the big play in the, the space, um, we can get all the common GPU metrics and NIC metrics from the, the NVIDIA servers. Okay.

And the, the second question along those lines is if you're coming up with a, a cabling diagram mm-hmm. Um, if you actually know the layout of your data center, can you also come up with a cable links out of the configuration? We don't look at cable links.

Um, If you have a cabling diagram, yes. Next, next question you have is what are the lengths of the cables? Yes, yes.

That's a very valid question. Um, we, we don't have that yet in the product for like cable links. Um, but we have like where, what should plug into what, you know, interface this on this, on this interface on that one.

Okay. Um, but that's, that's a a great question 'cause obviously you're gonna need to look at, and you know how far cable links, especially when you start getting into some of these high speed ones that, that cable links can make a difference. Okay.

Well I, I figured it would be like the next progression of, of what, of the information you have would be show me how the data center is laid out. Mm-hmm. Yeah, yeah, yeah.

I mean we, we can show it from like the physical connections of what's connected, like I said, but not the, the cable links yet. I think there was another question or a couple over here that, yeah. Okay.

So quick question. So you mentioned that, uh, it sits on top of abuntu or you can connect to a BO two. Do you need to have versions for Red Hat and for SUSE and for all sorts of things?

Uh, our, our agent is, is agnostic to the That's all You. Yeah. Okay.

You just mentioned Ubuntu servers because that's what you're Using. 'cause that's, I mean, Ubuntu's typically what a lot of the invade GPUs or servers are running. Mm-hmm.

And they're, they're, I think they're all on Ubuntu at the moment. Um, but, but our agent is agnostic to, to any Linux version. It can be deployed on others.

Thanks. Mm-hmm. Okay.

You get, um, something that, uh, after you the, the design mm-hmm. There ization costs solved, what is connected at. So probably partially, uh, any answer It's okay.

Alright. Yeah. If it's there any validation process or, uh, after stage.

Yes. Yes. So we do lots of validation.

Um, I'm actually gonna show it in the, the next one on how we catch errors. 'cause the first one I'm gonna show is how we catch errors of things that we're like virtual networks aren't assigned. Okay.

Um, and so we'll see some of that in how we, we have validation in our stage before we push it to the active to that production. And we prevent you from pushing bad bad code to your production network. Oh yeah.

Gotcha. But yes, yes, we do all that validation. We can also do, um, commit checks.

Um, so if you're familiar with the Juno CLI, you can do commit checks on, on Junos commands. We can help you run all that, um, on the, the staged area before you deploy it. Okay.

So in addition to, you know, other validations that we can do, we can do things like commit checks and, and validate, you know, syntax and all that. Okay, Gotcha. Yeah.

You Got another one? What's under the rack tab? The racks tab?

The, oh, the racks Tab. Is that, is that where we figure out how many servers per rack is per rack? Yes.

Yes. Good. Exactly.

We can map it out physically. Yes. Yeah.

Yeah. Great. Yeah, yeah, yeah.

Yeah. We can map it out physically, just not the cable links. Yeah, yeah, yeah.

Okay. All right. So, um, so a couple more things that I wanted to point out that, um, we asked of what if I need some more flexibility.

Um, a common one we get asked on is what if my server account's low enough that, um, I don't want to have, um, or I don't wanna have multiple stripes and I do everything in one stripe. So meaning having leaves only and no spines. Um, so then everything connects and, and runs through those leaves and the NV link on the, the Nvidia side, and we can do that as well.

Um, so we can build that, um, in the template designer where you can have, um, 128 GPU servers to get to a thousand GPUs but only have eight leaves. And then you don't have to have the, the spine layer on top for additional connectivity. So it saves you someone hardware cost, optics, cost, all those things.

So if you have a small deployment, we can do things like that to give you that kind of flexibility. Kind of like a, a collapsed fabric in a way where we're collapsing everything in and reducing from a, from a a three stage to a, you know, a single, single Layer. Would that be the collapsed rail design?

Mm-hmm. Click button there? Yes.

Um, so you have L three cloths and collapsed rail, no. Straight rail or rail optimized, or is that what clauses is Rail Claw is Rail. Okay.

Yeah. So no straight cloths Yes. In the solution.

Okay. Yeah. Yeah.

Thanks. Mm-hmm. And then this gets back to the other point we're talking about is what if, you know, a lot of people ask what if I wanna do something else with my, you know, my, my, my front end, my storage network, all this.

So, you know, we can do your typical designs that, you know, aren't the, you know, the reoptimize that we're talking a lot about today. So we've had that in the product for, for years. So I just wanna make sure we cover that.

So any kind of network design you wanna do, whether it's re optimized or not, we can do that as well. So to summarize what, what we've, what we've seen is apte can be your expert to help you deploy faster. We saw how we could deploy, you know, from, from specs to design to cabling map in a matter of minutes.

And we were able to do that all without having a single piece of hardware where we could, we could look at how the cable one map, how the config is all gonna look. We can validate all that before the hardware arrives. So we know that when the hardware arrives, we're ready to go plug it in and let's start using the equipment and start building some, some exciting models.

Um, so any final questions on this section? I had one, Karen, those info advisors. Um, so you mentioned, I picked up on, you mentioned you're doing All this in the contextual graph database?

Yes. And is that something you've developed yourselves? Are you leveraging, uh, another product or No, it's proprietary.

We built, um, within, within Juniper. Mm-hmm. And Perfect.

Thank you. Yeah. And I might have missed this, but what type of a, uh, APIs and SDKs are needed for this?

And if so, do you have? So we have full rest based APIs. We also have Python SDKs.

Um, so whatever you wanna build, we have a lot of people that are doing integrations. Um, we also have a Terraform provider that's becoming very popular in kind of the, the networking world to, to simplify and make managing private clouds very similar to how you would manage public clouds. Mm-hmm.

Um, but then you have the advantage of, it's your private cloud, it's your infrastructure, but you have the same kind of operating capabilities. Um, we actually demoed that at a, a previous field day session. Um, so anybody that wants to go look at how Terraform works, we, we've done that before as well.

Or you can just simply, you know, search for a Juniper Apps or Terraform and you'll see the, the Terraform module out there and all the documentation out there with it. And Ansible support, is that coming? Yes.

Oh yes. Okay. We do have Ansible support as well.

Okay. Alright. Um, so thank you for bring that up.

We do have Ansible support as well for those that, uh, um, still want to use that All. Uh, one more question, uh, bar. Um, in the beginning you mentioned that there are typically three network.

There is the storage network, the front end network, and then the GPU network. Yes. Um, when we see all this now, uh, in that blueprint mm-hmm.

Is everything connected to the same switches or are these dedicated, uh, separate, uh, networks? So the GPU has their own dedicated switches mm-hmm. Or are those three networks all combined converged on the same switches?

They're, they're typically running on different switches, different networks, but they, they, there is sometimes connections between them, um, or they're sharing potentially GPU resources. Um, but they are, they're thought of and modeled with an appra as, as separate network, separate switches. Okay.

Mm-hmm. And so Appra is also aware of this mm-hmm. Of these three different networks?

Yes. Okay. Thanks.