Cisco Reference Architectures for AI Networking with the Nexus Dashboard

com, which you can refer to for different scale sizes, right from say just 96 GPU cluster, all the way up to 32,000 GPU cluster, how to design, how the bill of materials, components and everything as well. And we talk about Nvidia, but we also are vendor agnostic, right? So we have our reference designs with EMD and Intel as well, which are out there.

All right, so that was mainly from the systems reference architectures, how we are going ahead with customers with that. Now, how do we simplify the operations for customers and go ahead and make sure that right, from designing at scale, while maintaining the simplicity in terms of automation and end-to-end visibility is what we are going to look at in this operating model. Because we have after years of like talking with customers requirements and everything, try to have more and more venture platform with Nexus dashboard.

And that's what we will jump into in terms of demo and video. And for this also, what I will try to do is take example of customer. This is one of the largest pharmaceutical companies in the world based out of uk.

You can guess the name as well. They are running Nexus 9,000 series switches in the data center fabric. As of today.

They're leveraging Nexus dashboard for their traditional data centers that they have across. They have a requirement, again, building 512 GP clusters, scaling out 2024 GPU clusters in future. They're calling this AI superpower facility for all the research in terms of DNA computation and all those things that they're trying to do at the same time, drug discovery and everything.

And they want to make sure from a networking perspective, the network team can leverage the same principles that they're applying to manage their data center fabric or even this AI cluster in a seamless fashion. So, So, so Megan? Mm-hmm.

Consulting you effectively had two scalable units. One was a 512 GPU one and this other one was a 32 GPU. Do you have something in between 64?

Yeah, Yeah, absolutely. The reference, I just give examples of like, just to make that understanding clear, but if you go to those reference architectures, you would see the in between as well. So that's why I mentioned it's from 96, 1 28, all the way up to one K, 4K, like 30 2K GPUs you will get, It'll be a scalable unit.

Yeah. Yeah. And the scalable units will change based on the designs as well.

So because you, your whole purpose is to start small with say 32 GPUs. Your scalable units are tied to 32 GPUs. And if you are having that entire cluster with 512 and you are a new cloud vendor, right?

They are thinking in thousands like thousand G, P 2000 GP and So on. This customer you're talking about what's their size? Do you have an, yeah, so Right now they want to start with 512 GPUs.

They will expand to one 1024 as the budget permits and all those things. So it's still like starting with 512 GPUs. And this is where I'm on the create fabric workflow right over here.

And you can see that we have options to create new land fabric onboard existing fabrics and so on. Our scope is to look at AI fabric, right? And you can see that we are able to go ahead from this same dashboard to manage all the different fabrics that we have, but we will select just the AI over here.

And you can see that we have two options that come up, which is AI routed or AI VX line EVPN. Meaning that if you are a, say new cloud vendor who wants to have multi-tenancy at a network layer, you click on AI VX line, E-V-V-P-N, or in 90% of the cases AI BGP routed would be good enough for your backend network. And the only thing that I need to provide is the name lack of creativity over here.

I will name it backend only, and then I, we can assign A-B-G-P-S send number, select the license tier, and that's pretty much it. All the best practice configuration from a Cisco validated design perspective in terms of the QS profiles, E-C-N-P-F-C, all the lossless fabric configurations are done by default with this few clicks and you would have your fabric up and running. That's how simply, uh, simple.

We have made the process out there, and I have tested this out in the lab as well, it's pretty much straightforward, nothing to do, much more than these steps, but at the same time we talk about how for advanced set of customers, they require that congestion management protocols as well, which is one of the most complex challenging things to configure. And that's where once you click on advanced button, you will get all the nerdy knobs with the default value as well for you to tweak. But what we have done is once you click on advanced, you would see AI settings, you click on DLV or dynamic load Balancing Enable, and that's pretty much it.

That's all you need to do to select, and it'll make sure the entire fabric that you have for your backend as the DLB policies applied to it, you click on next, you will see all the advanced settings. You can keep it default. If you are a power user, you can go ahead, change, tweak all the BGP numbers and everything.

But at summary, all these things are extracted from a user perspective, but are being configured at a fabric level. And that's what I wanted to show here. So that's how, for this particular customer, they can manage how they're managing their traditional EVPN VX line fabric storage fabric, everything together in this seamless fashion for even their AI cluster that they're managing.

So, So I, I guess I didn't see it. How do you specify that, you know, let's say a port is, uh, non-blocking or, or, or, uh, so backend port versus a front end versus a storage port. So when you, uh, select that as AI routed, uh, means that you're toing it for lossless fabric configuration.

When you are trying to do converged sort of scenario, that's where in that DLB mode, you select policy-based mixed mode in the dropdown menu. That's pretty much it. And then making sure that whatever tags you want for your storage and like, uh, backend traffic and so on, you can, can just make sure that it works on that.

So this is how we have the fabric created, but now we need to add switches to this fabric. So once you order the switches, you rack them up, you power them up via DHCP server or static ip, you get the management IP addresses and that's when you can just go ahead and add the IP over here. Now you would see there are different approaches through it.

There can be people who will try to go ahead and tag each and every switch as a leaf spine and add it in a tag-based approach without any guardrails as part of this fabric. It might be being managed into other fabric. It can be anything that can be out there.

What we are trying to do is, in a leaf spine architecture, you just have two hops between say leaf one, two, leaf eight, right between a spine and so on. We, all we need to do is give like a seed IP over here. The credentials for this IP specify the hops by default.

In a leaf spine it'll be two. But if you're having three tier architecture and so on, you can give like three hops and so on, click on discover switches and then it'll discover all the connected switches which are out there. Now the key thing over here is you are seeing all those statuses red, right?

That is the guardrails we do because this switches based on the management IP and the connections that might be out there. They can be your frontend switches. They can be managed in a different fabric as well.

You want to make sure that you are just going ahead and selecting the switches, which are part of your backend fabric. And all you need to do is select the switches and you would have your backend fabric up and running. So once you have all the switches discovered as part of your uh, dashboard, you'll be able to see your inventory with all the switches.

You can see, uh, we have 48 switches as part of it. You have the interfaces more than 6,000 interfaces where we are collecting the stats and everything endpoints. You can see 2000 endpoints, meaning there are 2000 GPUs where it is connected to.

You can look at it from a topology perspective as well, so that now this scalable unit concept right, comes into picture that, hey, I have the scalable unit eight leaves connected to certain spines within that scalable unit I have UCS servers, which I have with different GPUs. I can expand on the servers and get more details about it as well. So that's how we are simplifying the configuration, assigning to scalable units and making sure that your fabric is up and running.

So what do you say you expand it, it means that you can actually drill down onto a particular one and actually see the, the exact configuration. Uh, uh, so for the servers and nicks, we show the visibility piece, not the configurations per se, but that is what I will try to show you in the upcoming demo and how we are able to visualize those GPU nix try to correlate any issues happening in the network across the board as Well. Okay.

That's what you're coming, Yeah. Yeah. That, that's, that's what I'm going towards.

So thank you for bringing that Down. Okay. Because I, I was interested to find out if you could actually like Yeah, yeah, absolutely.

Errors or whatever in them. Yes. If, uh, if things weren't exactly the same or if if things yeah, were exactly the same and they weren't supposed to be.

Yeah. And I see there is a delay also in this particular, uh, like video, but hang, hang in there with me. So this is another customer, one of the largest, uh, you can say automobile manufacturers in US Who is out there looking at running their HPC clusters or changing those HPC high performance clusters that they're with only limited amount of GPUs into more of deployments where they can start with 64 GPUs as of today because they are running more CPU jobs for their HPC cluster.

Combine them together and they're using slum, which is an open source workload manager tool, and they want to get the AI job visibility in terms of all the different jobs that they're running. Try to correlate the issues, which can happen not from a networking perspective, but from a Nick or a GB GPU visibility perspective. So within Nexus dashboard we have integrations page where we can integrate with vCenter, Kubernetes, all those things which are out there.

But what we have done from an AI perspective is added this integration with slum, which is your workload manager. All you need to do is give it a name, name for the integration itself, give it an IP address, whatever you want to have in terms of IP address or host name, give it username password, click next and you'll be able to get exactly that integration up and running. So now you'll be able to get data coming in from slum as well and you'll be able to visualize it on this dashboard.

So now this is the at glance view. You can see all the anomalies, advisories, network infrastructures, all fabrics, AI resources, your backend front end and everything the backend is showing some issue. We can click on it drill down.

Now you can see we have AI jobs. I don't know why it's moving so fast now, but you had that AI jobs, you know exactly the critical uh, anomalies and everything which are out there. You can see all the jobs running.

You have trained one eval bot, one, trained two, all these things which are executed by system admins. But you can see that I want to focus on why my train one is giving me suboptimal performance when I'm doing this distributed job and I'm a networking team. I don't want to go ahead and just blame it on compute team.

And compute team cannot go ahead and just blame it on network team. We want proof, right? And this is where we can select that particular job which is running understand what's the summary?

You had a scalable unit with uh, like 2000 uh, GPUs that we had across different scalable units. We are seeing that within this job it's just leveraging eight switches to server 16 GPUs. We are tying the anomalies from optics, understanding how the optics are performing, tying it with the GPUs, and then we are seeing that connectivity issues are happening from a linked perspective.

So that's how we are able to exactly pinpoint and we can even drill down further into the analytics of these anomalies which are happening and get even further granular if we want to. So once we click on analytics, you will be able to see the drops. You can see ethernet interface one which is connected to a particular nick on node one has exceeded the threshold, the psych CRC errors pertaining to it, the power module temperature and everything which is out there.

You can click on how many GPUs this part particular job is running. You can see exactly the GPU three within UCS server. One is having an issue.

You can click on optics and see what optics are being used as part of this job. You can click on topology. So this topology is actually different than what you saw.

This is a job specific topology, meaning this job is leveraging these many servers connected to this scalable unit. You can drill down into a particular server. You can see between your leaf one, GPU zero, the link is showing red, meaning there is an anomaly there.

You are looking at, uh, the GPU three within this particular server, which again has the anomaly. So you can drill down from here and it'll show you exactly that. Hey, trans, trans temperature has exceeded the maximum threshold.

You can click on it, you'll be exactly able to see what was the reason for it, what triggered the anomaly, what's the impact, and even how do I fix it? Yes, Fredic Van Herrin, hyphens consulting. So more of a, a practical question is can you also use command line to interact with the tool?

Uh, uh Yes. What I mean by that is, is instead of going through gui, is there a way to kind of start with an API? Yeah, APIs are there, and that is after this demo, A P is the thing.

So thank you for bringing that up. So at the same time, uh, the customers we have, we are seeing more and more customers want that centralized view. The view that we looked at was a job specific view view, but what we are trying to do is that show it at an AI resources level so that you can look at your entire cluster, the scalable units which are out there, uh, the number of switches, the number of servers as part of your inventory, look at it at a glance of the cluster, look at the GPUs, which are part of the entire servers that we have.

We can even click on any of the GPUs. Sometimes customer like this health view where they can get at a glance view of all the GPUs, flag out exactly which one has the anomaly. You click on that you'll be able to get even further like GPU matrix, which are out there in terms of temperature, utilization, power, and everything from this particular dashboard.

So that's how we have simplified it. When you say mm-hmm. You're saying power, you're talking about the networking switch, leaf nodes, those sorts of things.

Not the GPU. It's, It's, it's, it's the GPU power that we are getting it from the exporter. Okay.

So actually it's actually GP power temperature that we are trying to get. So not, not the network switch. So all these things are specific to your e AI cluster, right?

So this is including the networking and GPU components together. And the last piece of the demo, the quick demo that I wanted to showcase is about the API. So we have new cloud customers who want to not rely on this vendor specific dashboard to go ahead and do that.

They're running a multi-vendor environment. They have their automation workflows and everything which they want to integrate it part of. So within our Nexus dashboard help center, we have this, uh, different documentation guides and everything.

That's where you will see the API reference with DevNet community at the, at the same time we have the FPI reference via swagger interface as well, so that you can use it in product if you want. And for all your management orchestration. And this is again, not just for AI cluster, for your entire data center fabric that you're running, you can have all the different APIs and say for scalable units that we are talking about, if we want to just look at any of the scalable units view, how many switches are there as part of it, I can just search for scalable unit.

I can go look at the API specs, I can try it out right over here in the interface itself. Click on that view the cluster name if I want the scalable unit that I want. Or I can just give it the entire fabric to export all the scalable units that I have.

So I can just go ahead and type in, Hey, what is my S-J-C-I-T-E backend having in terms of the scalability units and switch? I click on execute and it'll just give me the server response in terms of Aus and everything. So Yeah.

Um, silver consulting. Mm-hmm. Today's world with agentic activities, do you have like a, I think it's called an MCP server that supports access to these sorts of APIs from, uh, I don't know, an LLM that I configure my network?

Yes. We, we, we are working on the MCP servers for Nexus dashboard. At the same time we have our platform called AI Canvas, which is our ops workflow dashboard where you can type the queries and you can even generate configurations, view configurations, create reports, charts and everything.

So there are multiple ways to do it. You can do it in an open source way with an MMCP server or you can use our Cisco, uh, AI canvas, which ties all the domains together. It's not data center specific, even your campus van.

All those things are part of that particular, and I want to spend, give time to the hyper fabric team. So that's why we covered day zero today in automation. 2, but we are also evolving based on customer requirements.

So there are more and more granular details in terms of interface temperature, cable planning, uh, the auto IP assignment of GPU Nix, everything that is coming up. By the way, the reference architectures that we talk about also go with this Cisco Nexus dashboard model. So you can manage all those AI clusters via this Nexus dashboard management portal.

So with that, thank you very much. We can address any other questions that we have and I would like to shout, give a shout out to Paris Gupta spe all from the engineering team of Nexus dashboard as well. We're doing fantastic job with this.

So thank you very much.