Unified Data Center Network Operations with Cisco Nexus Dashboard

For this next session, we're gonna talk about, uh, Cisco Nexus Dashboard. And, uh, my name is, uh, Sunil Guke. I'm a senior Director of product management in, uh, the Nexus team of what we call data Center Networking team.

And I have with me Joey, um, Joey is our, um, uh, technical marketing engineer. So he's going to compliment me with the demo after I finish my talk. So we're gonna split even 15 minutes of, uh, me talking and 15 minutes of, uh, Joey doing the demo.

Let's, let's target, target for that. How many of you're family with, uh, nexus product line in general? All of you?

Excellent. I'm gonna take a step back and give you context and, uh, talk about how we arrived to where we arrived today. And then we are gonna jump into the product and talk a little bit more in details and then do a demo.

Okay. We've been shipping Nexus nine K for good 10 years, um, 11 years now. Um, and with Nexus nine K, we had other products lines like Nexus seven K, nexus three K, but flagship is Nexus nine K, nexus seven K also got end of life with Nexus nine K.

We enable variety of, um, fabric deployments, a variety of data center network deployments. And for a long time we have done, uh, variety of, uh, two tier, three tier multi-tier architectures, uh, with Leaf spine, super spine, that kind of models for general purpose data center. We've been doing large scale HPC and AI ML clusters, also with Nexus nine K for our service provider customers, mobile operators.

We've been creating a lot of edge data centers. That's what you see here. And lastly, with all these data centers, there is a need to interconnect all of them.

And we are using Nexus nine K to interconnect very securely as well, right? So we have enabled all these various deployment options with Nexus nine K for us to be able to do that. We have created these two architectural options, okay?

Cisco NXOS, which carried forward from the Nexus three case, nexus seven case that, that, that kind of architecture with vxlan, with without vxlan, and also Cisco, a CI that we introduced with Nexus nine K. How many of you're familiar with a CI? Number of you?

Okay, just checking. Just checking. Um, well, Cisco and XOS and Cisco, I say, let me just, you know, put it in a topology format, right?

Like that is a little bit more visual. So with NXOS, you can run in a regular layer two, layer three type of network. Or also you can also do overlay with vxlan, with Nexus nine K with a CI.

It's by default is VXLAN under the hood, but it also comes with a controller called Epic Controller. Essentially, the architectures are bit different and they solved, um, uh, specific business outcomes and they evolved in these two tracks. But we are seeing that the customers have gotten to a situation where sometimes they have both models, uh, deployed in their environments, and we saw the need to bring them together and unify them basically, and make them sim make them simplify in terms of operations.

And, you know, simplification, in my opinion, is best achieved with unification. So what we started, uh, uh, we started on a journey of unifying the two architectures, and I think maybe six months ago, or maybe in Amsterdam, maybe last year, we talked about what we call Nexus one. Were any of you here at the time?

No. Okay. So Max Artica was, uh, in our team, uh, went through this thing, but the idea here is to bring these two architectures together and bring them to operate or produce the similar outcomes and the outcomes wise.

What we are talking about here is being able to provide that zero trust networking with microsegmentation policies based on endpoint groups or endpoint security groups, being able to do that advanced service chaining and, um, insertion of L four to L seven services, being able to interrupt with third party, using open protocols, being able to integrate very well into the CICD pipelines, administrative multitenancy, so on and so forth. So the idea here is to bring these two architectures to deliver these solution outcomes and also make them interoperate with each other, right? And we can do so by doing a lot of unification work.

And that unification work that we are doing is focused on all these various planes unifying at the data plane level, unifying at the control plane level, unifying at the policy plane and management plane, right? And for this, we have gone to IETF and proposed some standards on how we can do this unification work, right? And so that's what we are working on.

That's what we call Nexus one. And if you look at this architecture, the data plane lives in the switching infrastructure, so is control plane. And so is policy plane from Enforcement Point.

But the one that lives outside of this physical infrastructure is the management plane. And that's how we come to the unified management plane for Nexus. And we call it Nexus dashboard.

1. The first one, which is gonna come out in July, we have completely redefined reimagined rebuild Nexus dashboard so that we can enable this unification. Okay?

And in addition to enabling this unification with Nexus one, we have also changed architecturally on how Nexus Dashboard looks. Before it had various modules and services, and it was following an app store model where you had fabric controller as an option. We had, uh, insights as another option.

We had Orchestrator as another option. So we had all these various modules that were running on Access dashboard. What we did was very soon we realized that it is not actually a phone.

So it had to, it is an automation, uh, application, and it had to be very well integrated across all these various applications. So for that, um, we kind of removed these kinds of, um, uh, standalone applications that are on Nexus dashboard, rather, we put all of them together into a single application, very modular in nature, but, um, the capabilities are very nicely integrated into the whole, uh, uh, nexus dashboard. And that's what Joey will show you in action instead of me talking about it.

But that's kind of what we did with, um, nexus dashboard, evolve it to support the Nexus one architecture and also re-architect at the same time to integrate all the functions very well. Okay, now with that, with Nexus dashboard, now you can pretty much do the complete lifecycle automation of the Nexus product line and also MDS and also in certain cases our Catalyst product line. And let me talk a little bit more about that.

Uh, with Nexus, you can enable variety of architectures. I talked about NXOS, I talked about a CI, you know, in NXOS you can implement with vxlan, just classic ethernet. You can create fabrics for media, you can create fabrics for ai, ML, all these various types of fabrics can be automated.

A complete device lifecycle management with Nexus dashboard, in addition, in cases where our customers wanted to use vxlan, EVPN as the control plane, even on the Catalyst side in the campus. So we enhanced Nexus Dashboard to extend that VXLAN EVPN automation, even on the campus side, because we have been doing VX line EVPN automation on the data center side for a while now. So it was very easy for us to extend the same experience on the Catalyst side.

And we have always supported MDS product line to build the storage area networks. But, you know, regardless of the type of fabric, the idea for us is to provide that consistent operate operator experience with Nexus dashboard. Question, Josh, from Diversified, are you, are you saying the MDS and SAN architecture is going to be, or is part of Nexus dashboard, or is that going to continue to be separate?

I, I couldn't tell from your Comment. So it is, uh, the, at the time of, uh, uh, nexus dashboard installation, you would choose whether you're gonna manage LAN or whether you're gonna manage san. And it's the same, same image, same product, everything, same.

It's just that, um, uh, it's an installation option at the time installation Option. Okay. How would I talk about the capabilities of next dashboard?

So I would like you to think about this in these four categories, right? Provision, secure, manage, and analyze all these four capabilities. Very well integrated into Nexus dashboards, uh, user interface, and also very well integrated into Nexus dashboard APIs.

So the APIs are also organized in this type of, um, grouping, the type of fabrics that I mentioned about classic lan, right? Two tier three tier NX OS architecture. You could do vxlan, A VPN, not Nexus, and also Catalyst.

You could build AI ML clusters. You could build the, the DCI networks like the interconnectivity, um, the external fabrics, interfa net connectivity. You can build media fabrics, you can build BGP routed fabrics, all these.

And also not, uh, let me not forget the data broker, um, type. And, and we have always had a data bro, data, data broker offer, but we all ask, we always had a separate VM to manage that. Now we are bringing that also into the mix and then adding it to the NEX dashboard.

Basically developing the NEX dashboard to be the single stop shop for Nexus and MDS product lines, right? And this is where you see all the product types, like is screenshot of, uh, the, uh, new, new user interface. Now, let me briefly go into each of these areas because this is like complete network lifecycle management.

So from a provisioning perspective, you would be able to bootstrap, build your fabrics from scratch. There is a fabric builder for each fabric type, and it, you automatically, you select what is the role of the device, and whether it's a leaf or a spine or a border, it'll automatically generate the entire config for you using fabric builder. So that fabric builder is available for all the fabric types, right?

So you can build the fabrics from ground up and also continue to do your, um, incremental configuration. You want to add more networks, you want to add more works. You want to do, um, SNMP configuration and TP configuration.

So the complete provisioning is taken care by Nexus dashboard. Now, I'm not touching all the points because it's difficult. This, this 15 minute time that we set ourselves is not gonna give us to enough time to touch all of them.

But I'm gonna touch the secure part of it, right? It has compliance dashboards that allows our customers to be compliant with their policies, with their internal security policies. It gives them security advisory information.

It gives them end of life notifications. It gives them the capability to use. The Tetra agent that we acquired through is ENT acquisition so that we can do CVE mitigation, right?

This is something new that we are working on. It is not there yet, but that is in the plan. And also if we notice a, a, a, a critical bug in a, in, in a, in a, in a, in a, uh, specific customer environment, we understand that we digitize that and then we can make, we can alert the customers of the applicability in their environment.

So we can do all of that from compliance perspective. Obviously there is a lot of audit logging happening, audit logging that for the entire fabric is stored in the Nexus dashboard. And also it can be shipped out to Splunk, you know, for, uh, uh, Splunk integration.

And not to forget the segmentation. We have always implemented, um, uh, various microsegmentation microsegmentation policies. Now, all of that also you can achieve from Nexus dashboard perspective, okay?

That's from the secure side. Now, manage the most important ones. Fabric upgrades, being able to do the hitless upgrades, being able to do free upgrade and post upgrade snapshots and being able to compare them.

So you can do all of that with Nexus dashboard and the next dashboard lifecycle itself. You know, being able to upgrade the, um, the, the application, being able to do the in place upgrade so that you don't lose, uh, data. And also being able to do a backup and restore to something else.

So we're looking at this from complete Nexus dashboard lifecycle point of view. And let's not forget the change control. So when you do any configuration changes, any provisioning changes, we have implemented the ability to, uh, for the actual, the person who is doing this change, being able to submit this change for an approval and then get that approved and then continue with, um, pushing out this change.

And this can be integrated into, uh, ServiceNow framework as well. So, uh, that from a complete lifecycle management point of view for manage. And lastly, let me cover the, an analysis portion of it, which is analyzed gives you like tons of visibility into your network, not only from topology perspective, fabric views, switch views, port views, link views, you name it.

So we have, uh, complete drill down, starting from a global view to until the link level. And we have recently started implementing ai ML job visibility. We have always implemented when for a ML fabrics, um, the congestion hotspots and that kind of things we implemented.

But now we are taking this to the next level, which is being able to correlate between the AI jobs that are running in the system, to what is happening in the network and for a specific AI job, which GPUs are involved, what is the GPU Health and how is the traffic flowing from this GPU to that GPU, which path it is taking. So we pretty much are able to see all of that from Nexus dashboard and troubleshooting wise connectivity analysis from point A to point B, how is the connectivity, which path it is taking, are there any hotspots? Are there any acls?

Are there any Q drops? All of that you can see from connectivity analysis. Uh, Delta Delta analysis is more about snapshots, like, you know, this is my snapshot of the system on time.

T one, uh, time T two one more snapshot. Compare them both and see what is the differences. Deltas, right?

Uh, that is from troubleshooting perspective and analytics. Traffic analytics is something that we implemented. We used to implement something called flow analytics, which was dependent on every flow record, but we modified it to go based on more, um, what we call a net flow type of analytics.

Now you get, um, you, you get to scale a lot more, right? So we implemented that. Now with that you can really see top talkers and et cetera, et cetera.

But also the application type of, um, uh, traffic analytics, which application is really taking up, um, a lot of bandwidth, et cetera. Those kinds of things. You can see from traffic analytics I talked about data broker.

This is gonna very nicely integrate into the entire Nexus dashboard framework, and you will see that come through very soon. And also, good amount of insights into san. What I will do is to briefly show you the new thing that we did from A IML.

And from A A ML perspective. Um, a lot of work has been done in creating, uh, a very advanced load balancing mechanisms, right? To, to manage this traffic very well.

And these load balancing mechanisms, we are, uh, we are, we are branding under Cisco intelligent packet flow. It comes with lot of dynamic load balancing capabilities. But how do we have a good understanding of what is happening in your AI ML cluster?

Which jobs are taking lot of bandwidth, which GPU is talking to which GPU, what path is it taking? How congested is that path? All of that visibility you can get from Nexus dashboard just by importing this AI ML fabric into Nexus dashboard.

Um, so that's kind of what I have from Nexus dashboard. As a quick, quick rundown of what we are doing with this product, um, you know, in general, I mean, it's, it, it, it gives you so many capabilities to accelerate customer's innovation, increase business value, and minimize downtime. Those are kind of the business values that, uh, business business outcomes from Nexus dashboard perspective.

Before I hand it over to Joey, um, anything that, uh, I can answer, do you have any questions? All good. You wanna see the product first and then we can, we can talk more.

Okay. Come on Jay. Thank you.

Good intro, Sunil. My name's Joey. I'm a technical marketing leader in our DCM business.

And as Sunil just described, we've been spending about the last six months, uh, really rebuilding Nexus dashboard in a really good way to try and unify a lot of these services. So I want to do a brief demonstration today just to show you kinda what we're working on. 1.

So this is software that we expect to ship in August. Uh, it's going to be our FCS release for what we call the unified product. Um, very briefly, what you'll see when you log in is the anomalies and advisories.

That's some of the telemetry that he alluded to. I'll go through a few use cases for that. But you'll also have a map of all the fabrics that you have deployed.

What you can do with this system is you can actually deploy localized clusters that can manage different fabrics. A use case for that would be, you know, you have multi-site architectures, you have fabrics in different locations, so you can deploy separate clusters to manage those. And then what we do is we federate them.

So we have the control plan using out-of-band management and all the clusters can actually talk to each other and pull data from each other. So we do support kind of having clusters all over the globe per se, but still having a centralized place to manage them. If you go to the fabric view, I just wanna call a couple things.

Uh, we support many different fabric types. Now, uh, on the same Nexus dashboard, uh, appliance, you can see I have a couple a CI fabrics here, uh, in a multi-site configuration. And we do expose some of the underlying feature sets that we would typically be deploying as a separate application, right?

If you're familiar with Nexus dashboard, in the previous releases, we had this dropdown up here, and you would go back and forth, Hey, I want to go here. I want to configure NXOS, I wanna go here, I wanna look at telemetry. Okay?

So what Sunil was alluding to was, we've unified all that and now we just look at all of that natively within the fabric view. Okay? So we support many different fabric types.

A CII have a, a VX lt, EVPN fabric here. And I also even have just a pair of NXOS switches. I use these to generate traffic in my lab, do layer three routing and also act as an I-P-N-I-S-N network for my a CI multi-site, right?

So we can manage all these things now from the same place and we can also then start to build on these use cases, expand that one fabric ecosystem, and be able to build workflows between the different architectures to deploy and ultimately manage these fabrics. Question, yes, Strom diversified. Yeah.

Uh, would another fabric instance type, could you also have the IP media fabric as well? Yes. Type, because before it was a installation selection, right?

Yes. From a decent NN perspective. But now it looks like that could be another fabric type.

Yes. Yep. That would be a new fabric type.

So we are, um, yes, the, this will allow you to do an IP fabric for media right here in the same place. In the same place. Yep.

And this just creates the template. You'd go and add the switches and then we'd push that templated configure for that. Okay.

Yep. That's another thing we've added in, uh, in this as well. So if you were to go into, let's say, A-A-N-X-O-S fabric, for example, what you'll notice is that we have the full inventory of the fabric configuration state, whether or not we're exporting telemetry, all of the security segmentation, all of the networking related config, and then also all of the connectivity related information.

So when we say unified, one example is that we're able to do configuration and then see the impact of that configuration from the telemetry all in the same place. So again, no, no more going back and forth between different things. And we've also added in the segmentation and security, the security groups construct, right?

We talked about, uh, the excellent GPO and how in a CI, we also have the contract model, the policy model with an A CI. So we now are exposing a lot of this unification work within Nexus dashboard to automate these security policies and make it so that we have a cohesive policy model across the two architectures. Once we deploy configuration, we can see the impact of that configuration by going and looking at different areas of the environment.

I apologize, I'm my lab's an RTP, just gimme one second for the, for it to load here. But you can see things like my layer three neighbors, Hey, I created a layer three connection. Did that come up?

What's the status of that? What, what's the routing that's being learned on that? And we automatically categorize and do a change summary based on the last two hours.

So you can very easily do a health check of the environment. Hey, did any routes change in the last two hours, right? If so, what changed?

Which VRF was it? In? Which prefix was it on?

Which node was it? Right? So we can get really granular as far as the, the routing and also the endpoints.

So if we look at endpoints, let's say you push a network change, whether it's a CI or NXOS, now that VLAN gets trunked endpoints start getting learned, we want to see what the, um, you know, what the result of that was. The other thing that we do with Nexus dashboard is we start building what we call anomalies. So we can raise anomalies based on anomalies behavior that we would see as these things come into the network.

And as that telemetry gets ultimately sent to nd so we automatically sort all of these components based on the anomaly score. So as an administrator, you can very easily just go in and say, Hey, are any of my endpoints unhealthy? Are any of my routes unhealthy?

Right? And you can go in and then see exactly what might be happening, right? So you can go in, you can look at the anomalies page, oh, you can see exactly what is going on.

It looks like I have a duplicate IP address on this. It also looks like I'm buffering something. So there's some congested port that this endpoint sits behind that's having some problem.

So we're really trying to just be complimentary with the config, the telemetry, and the orchestration of all these fabric types. Another thing that we have that Sunil alluded to is a feature called traffic analytics. So if you're familiar with ND in previous releases we released as part of the cloud scale ASIC family, the concept of flow telemetry.

Alright? Flow telemetry is very powerful 'cause our asics actually have hardware tables that they can actually use to capture flow data and export that flow data to nd there were some scale implications in doing so. 'cause you have packets coming into a switch, those packets are being sent immediately to a cluster every one second.

So what we decided to do instead was build, take this, uh, flow telemetry functionality, but make it a little bit more persistent and more pervasive monitoring. So with traffic analytics, what we do is we do that same level of exporting of that flow data. We remove the source ports, which gives us a little bit more scale.

And we also can categorize each flow based on latency scores, congestion scores, and drop scores. So if we look at what this would look like, we start seeing all of the flows in our data center and we start categorizing them based on what we call service endpoints. So if there's an application that's running on a server and that server has an IP address, we say that's a service endpoint.

That's service is exposing port 80 for example, or 49 4 4 3. And we're also able to see what the health of that application is based on client to server communication, incorporating the drops, the latency and the congestion stats. Question on that, you, you mentioned that you are, you're truncating or you're dropping some of the flow data for scale purposes.

Is that happening before the data's ingested? Is it happening at the switch? Where is that absolutely popping happening?

Yep. And we have the, the capability stored all? Absolutely.

It's a great question. So what we did was a couple things, but the primary thing was instead of sending everything to Nexus dashboard and having ND process everything locally, we actually take that flow table in hardware and we flush it to software on the switch every 60 seconds. So what happens there is that the switches actually take a little bit of responsibility to ingest that data, enrich it a little bit, and then send it to nd.

So by the time Nexus dashboard gets it, the amount amount of processing ND has to do is very minimal compared to what we were doing with Flow Telemetry. So yes, we, we did re yeah, it, it does, there's aggregation in some enrichment happening on every switch before it gets sent to nd Yep. And uh, there's no implication on the CPU 'cause again, the hardware's capturing that.

It's just flushing it to software, doing a little bit of processing and then exporting to nd And what's nice about this too, personally, is you can kind of get on the application developer's view. You know, a lot of the times when I was in TAC or in an escalation role troubleshooting, we always ask what's the source? What's the destination ip?

Well, when you're talking to an application team, you sometimes wanna say, Hey, what port is your application using? Or what type of application is it? And so from the network administration perspective, for example, with SSH here you can say, okay, these are all the devices that are sshd into that one particular server.

And then we also have statistics per flow for everything that's happening there. We also have the ability to trigger a flow collection. And what that will do is that will start actually doing that same five tuple export that we were doing with Flow telemetry, building that path for you and also isolating exactly if there was a drop or a latent link within that path.

Um, one other thing I would like to show, because it was talked about and it's something that I uh, feel passionate about is, uh, connectivity analysis. So oftentimes when troubleshooting various issues in the data center, you want to know exactly how the flow goes across the fabric, but you also want to know what type of traffic was captured and what the switch was doing with it. So we can trigger these workflows through traffic analytics, but we can also leverage a technology called EL a m, the embedded logical analyzer module on the Nexus 9,000.

And we can actually automate the capturing of these packets in the hardware and see exactly what's happening. And because Nexus dashboard is now unified and also supports all of the services, here's an example of an a CI multi-site fabric. Okay?

So we can even do, we can do it per fabric or we can do it across multi-site as well. What you would do is you en enter a source in a destination, you can get granular on the IP protocol if you want, or if you want to capture just a specific source and destination port based on your troubleshooting use case. And then you can go, I don't wanna see the log.

Then you can go and it will build a topology for you based on the telemetry, where the endpoints exist and how the IPN is constructed. And then it will start running the Elam M capture across every device to figure out exactly what the active path is. Okay.

And this could be handy 'cause you might have traffic analytics data saying this client can't connect, but this one can. You can go build a connectivity analysis job and isolate exactly how that flows, how the ECMP hashing works, and then also compliment that with if there's drops or congestion, uh, via EAM. Quick example of how this works, if you just click on the details, it will tell you the data plan information, the path, and all of this is derived from the actual packet header data that we see go through the asic.

So we are actually setting a trigger in the hardware saying if you see this frame trigger, tell me what the metadata looks like, what the headers look like, and also the full report saying what port it went out on, what drop code there was, or any analysis that might be useful to troubleshoot. Okay. Yes.

Uh, the question again is that, is that across different fabric types? You showing here that it's two a CI multi-site? Yes.

Could be a CI or a VXLAN fabric. And you can do the same, it will be Activity analysis. Yes.

The, the, the building and the automation of the VXLAN border gateway piece to N-N-X-O-S environment, building that together, we are working on automation to do the end-to-end config for that. Mm-hmm. 3 timeframe.

So towards the end of this year. 2, the connectivity analysis piece will also work across architecture as well. Okay.

So we'll be able to build in, again, anything where we want to unify what that looks like or deploy or troubleshoot, that's what you should be expecting out of nd going forward better. Do you have anything in the demo for an AI ML fabric? Um, I don't have Run in or a backend fabric.

I don't have it in this setup. We do have some record. Yeah, we have some recordings.

Um, for sure. Actually we have multiple booths too. If you go to the world of solutions, we have multiple demos we can show you there.

I I could pull them up, but yeah. Um, yes, and we have, uh, some live fabrics and also some demonstrations of what's coming in the future for the visibility on GPU workloads from server to switches. Yeah.

That gets Into rail alignment analysis and stuff, so, yep, That's fine. Yep. And we have the blueprint already and the templates to go and automate all the QOS configuration for priority flow control and for ECN, um, that all gets automated through ND and then the telemetry comes in after the configs pushed.

Yes. In terms of, um, vxlan, it's a standards based mm-hmm. Protocol.

Is there anything on the roadmap to pull, um, third party vendor VXLAN fabrics into, uh, nexus dashboard for a unified the experience? Yes. Try.

Oh Yes. The short answer is, um, yes. Um, we plan to support other vendors as well with, uh, nexus dashboard and especially once we start implementing the VXLAN inter, uh, interrupt, like, uh, remember I mentioned about enhancing the ITF standards to provide the policy piece, uh, of vxlan.

Now if once we start to implement across the other vendors and other vendors are also implementing, I think we will be able to have that, um, managed by an access dashboard as well. Powerful. Yeah.

And nd also supports what we call freeform. So basically if you wanna put CLI into a file, right, we can, we can automate that. So as the first iteration of this would likely be some type of freeform template where we would just control based on what your user input is.

But then expanding the ecosystem from there is a, is an obvious decision. And also the, the extension to the third party, um, support not only takes care of the provisioning part, but we would like to address the visibility and analytics piece too. Uh, as long as they support, for example, uh, NetFlow and, you know, we have a capability to ingest NetFlow.

Mm-hmm. And because we built it in a generic fashion, mainly with the intent that at some point we will be able to support the third party devices.