94. AI Has Escaped Your Datacenter Presented by Cisco – Tech Field Day Podcast Spotlight

AI generative AI escaped out into the internet. Now, AI is escaping from your data center. Is your network ready for it?

Does your network have AI for the network? Does AI AI exist in your network? Join me on the Tech Field Day podcast as we follow up with Cisco and find out all about networking and ai.

Welcome to the Tech Field Day podcast, where we bring together a group of IT technical experts to discuss a single idea about key concepts in the industry. This podcast features a variety of perspectives from members of the Tech Field Day Delegate community is often record in association with one of our events, our tech field as part of the FU group. And this podcast has also published us to company Site Textron tv.

On this episode presented by Cisco, we're discussing how AI has escaped your data center and is invading your entire network. Before the discussion, let's meet who's on the panel today. Hi, uh, I'm Andy Banta.

I'm currently doing work with, uh, Magnes io and have been a longtime person in the tech industry doing storage networking. And I'm Jack Poller. I am an industry analyst with Paradigm Technica, focused on the intersection of cybersecurity, uh, and artificial intelligence.

And I'm Lee Peterson, the VP of the Secure and Product Portfolio at Cisco. A big fan of Tech Field Day, really, really for honored to be here. And of course I'm alist k an event lead here at Tech Field Day, the event lead for AI infrastructure field day four, where Jack and Andy attended, and also colleagues of Lee's.

Uh, it struck us as we're going through the, all of the presentations, but particularly this, the Cisco enterprise networking presentations that AI is both a workload, but it's also an enabling technology for the network. And as we're seeing a bit of maturity in how companies are using aid ai, moving from just some beginnings thoughts of what they might do to actually doing useful work with ai, uh, the network has become one of those things where there's features that we might want to use. There's also, uh, much more complexity in managing that work network as a, as a whole.

So, um, we definitely saw the importance of having both AI for your network, but also your network for ai. And Andy, I wondered, wanted you to have some space on particularly that topic, because I think it was one of the things you, you saw maybe a bit of confusion amongst some of the, the messages we saw at a infrastructure. Well, Absolutely, and I mean, it's, uh, we saw some presentations from companies other than Cisco networking companies other than Cisco as well, but Cisco, uh, presented a a huge amount of material and the topics kept switching where whether they were talking about the infrastructure necessary to carry ai, and that was, that, uh, included like bigger, faster switches as well as more secure routing and better transport of the data, as well as, uh, as using AI tools to manage your network or, or to, uh, um, configure, debug, troubleshoot, configure your network, as well as tools for, um, you know, AI tools for doing things like using, uh, location with, uh, wireless devices.

So there was a lot of, um, it was, it sometimes you needed like a scorecard to be keep track of whether we were talking about the infrastructure necessary for AI or how, uh, Cisco was presenting this infrastructure to enable it to, to enable their own AI for their own purposes. And it was, uh, it was just a, a very interesting mix of topics and it, it really goes to demonstrate what AI infrastructure means, uh, to these field day events where it's, it's both enabling the AI and the AI is enabling the infrastructure. I think what makes it really interesting in addition to that is it's now becoming a self-reinforcing positive feedback loop where you enhance your infrastructure to enable AI development, and then that enables you to develop, uh, more complex AI models that allows you to both fine tune your network as well as to manage it, troubleshoot it, and build even better and faster, uh, infrastructure for your AI development.

So there's a lot going on that Cisco presented. The other part that I thought was interesting is this, what, you know, sort of the premise here that we're talking about is AI escaping the data center. And traditionally we think about AI and giant data centers and training on the data centers, and there's not as many people are gonna be doing that as they are going to be doing inference and particularly inference at the edge where you have devices that are going to be generating a lot of data, moving a lot of data around and wanting to do analysis and AI with that data.

But you can't transport that back to a core data center to do your analysis because it takes too long and it costs too much time and bits moving down the wire. So you wanna move your AI infrastructure to the edge and all of the various different technologies that Cisco's bringing you to bear to support that, both as networking infrastructure for ai and again, the ability to apply AI to help you build that type of infrastructure. I think what's interesting with that latter point that Jack, is that if you think about four or five years ago, most of the CIO CTOs were saying, we're gonna just move everything to the cloud.

We're gonna put it all in, uh, relying on the, on the, uh, hyperscalers that run these workloads. For us, what we're seeing more and more is now it's much more federated. So obviously the, the, the AWSs and the Azures and the Google clouds, the world are going nowhere.

There's still a very important element, but we're seeing more of a mix where also investing in the, in, uh, our customers in their own data centers, investing in colo facilities. And then of course that idea of what can I do at the edge, particularly when it comes to those, those heavy DPU workloads that are maybe from an interesting perspective, more, uh, effective to do closer to where I need that done, where I need to be able to actually make a decision on something. Think of things like video applications, think of things like small language models from a chat perspective, much more effectiveness doing those at the edge, both from a cost perspective, but also just from a, an outcome perspective of what those customers trying to drive.

Yeah, I think, uh, one of the things that I've seen over the course of Tech Field Day and other presentations is that, you know, we tend to think about AI right now as we're hyperfocused on large language models and the chat GPT type things. There's a lot of AI applications that are go beyond that or analysis of real-time analysis of data. For instance, every time you make a credit card purchase that goes back to somewhere where there's a decision made, whether that's a fraudulent charge or a valid charge, and there's a lot of data that's collected in order to make that decision.

But you wanna make that in real time. You don't want somebody to put their credit card on the reader and then wait a minute for an answer to come back whether this is good, uh, valid or inval charge. Um, we've also seen things where you're doing sports analysis or applying AI to motion tracking and all of that has to happen as close to real time as possible.

So the, the lag of moving that data into the core, whether it's on premises, uh, core data center, or whether it's moving to the cloud, sometimes that can be too much. So there's a technology driver and a motivator towards, uh, putting stuff in your own compute infrastructure where, where, regardless of where that is versus renting that infrastructure from the hyperscalers. But there's also use cases, we, reding, hyperscaler you can take advantage of their economies of scale and it does make sense still.

So, and I mean, a couple of the examples Jack just talked about are talking about, uh, potentially data centers sized, uh, you know, models that you need to work with, whereas there was, uh, some discussion last week on a couple of the, um, places where you don't need data centerized models. Uh, one of the examples I remember was controlling robotics where you, uh, you would, you basically need a factory sized model to figure out what's going on. Uh, and the other one that would impressed me was the, the ability to, um, feed in information about your enterprise network to the AI assistant and have the AI assistant use both the, the source of knowledge from Cisco as well as information about your local network to be able to better answer questions about your own network.

So these are, these are two examples of really edge based systems where it's using the local data rather than drawing from a huge Bottle. And, and just to sort of take both of those points together, if we think of the, the robotics use case, the, the, the most of the factories that are doing things like loading and moving things around using robotics now have a rule that if any one of those robots goes missing for more than 250 milliseconds, every piece of robotics in that, that location shuts down because the, just the risk of, of human life, the risk of damage and things like that. And so that's really what it, it needs to be done at the edge, uh, to the second point around, uh, then how do we extend some of these AI approaches and apply them to, to the technology itself.

Uh, the MCP server approach is very interesting. So, uh, a lot of the investments we're making in, in AI within Cisco, we've developed a steep network learning model. So we know networks better than say a general purpose.

L LM knows networks and we've been able to take, we have the largest data lake from a networking perspective of anyone in the industry. We've been able to use that to go train these models. We been able to take what we know from 30 years of running CCIE courses and all that knowledge.

We've gotta take every support case it's ever been entered. We've been able to put that all into the soup and produce a model that's much more effective in terms of a analyzing a network and, and understanding what's going on with it and making a sensible set of recommendations to further that out, though the idea of using MCP to further enrich that because there is gonna be data that our customers have in their networks, customers have about their own applications that need to be, uh, sort of correlated somewhat with what we know about the network in order to be able to actually produce a, a, an actionable outcome to be able to make these networks better, faster, more reliable, and ultimately in service of driving employee satisfaction and customer satisfaction to let these businesses do a better job. This is where I think it gets to be a little bit of a blurring of that line between networking for AI and AI for networking where there's more of a feedback loop between those two parts where the actual data that, or that the AI that knows about how the network operates can be published out through CP so that the AI that's delivering some business value that might have some dependency on that network can talk to the, the network, the AI for your application and can talk to the AI for your networking.

And that's, that's becomes a bit of a blurring of where you've got just pure infrastructure that is there to host your ai and where there's an AI in that infrastructure that's assisting the AI that's doing something for business. And this is where the whole idea of agent to agent communication and, and the CP standards and some of this emerging cooperation between different parts of the, your AI estate become really crucial. One of the things we had joked about, um, during the presentation was do could the, uh, Cisco's LLM pass the CCIE exam?

And we were joking about it, but in some sense it's a really important point in that there is a tremendous amount of, um, tribal knowledge that has been developed over decades that Cisco has, uh, both just from the telemetry Cisco's collected and just the tribal knowledge of the CCI and all the different people involved in this that that you're able to bring to. And when with the ability to have an agent query and get the very quickly, get the information it needs and apply, uh, changes as quickly, then you, that that sort of automation and feedback loop that Alistair was talking about becomes very powerful. And that's I think where Andy and I are very interested in, that's that application of AI for networking that isn't just, hey, we're, you know, it's not, it's, it's, uh, not a variation of, um, the traditional Silicon Valley of let's do the, the next version bigger, better, faster.

We've got, you know, we've gone from a 400 G to an 800 G network, look at our switch, right? It's a lot more than that. These things are actually, are actually starting to have value above and beyond just the ability to move packets back and forth.

And I think that's very interesting and very valuable to, and to AI and even to non-AI data centers. Right? And, and I mean, regu, when the, the question came up about passing CIE uh, even pointed out that the, the exam itself includes lots of questions and, and just learning the answer to the questions is one thing, but one of the tests for CIE is that you actually need to build out a network.

And it would be very interesting to, to, you know, see AI assisted get to the point where it could actually build out a network. And, uh, you know, this, this gets into the idea of AI feeding ai. Where, uh, could the, uh, could the AI assistant start figuring out that its own demands on the network were increasing the load on the network and therefore would need to expand the network?

I think there's an interesting sort of, um, parallel to this. You know, they can build an LLM that can pass the bar exam. You can build an LLM that can pass any sort of medical exams.

I don't want an AI lawyer and I probably don't want an AI doctor. And so we think about this the same way we ask ourselves the question, if A-C-C-I-E was standing in front of this screen right now, seeing what I'm seeing, what would they do based on using all that knowledge? They're very much designed around how do we do human and loop?

You still need to have some level of hands on the wheel to make sure that we don't set these things loose and have it make changes and make adverse effects. The difference is though, that where this is gonna enhance the workflow for most of our operators is that that engine is able, that agent is able to look at these things at AI scale. So it's ability to look across large data sets and infer the interesting parts of that.

A lot of the time when you're doing this troubleshooting, 95% of the things you look at are, are dead ends, being able to sort of jump straight to the root cause and go, this is the problem, and if you do this, we'll fix that. And as we get more confidence over time, there'll be certain things that are routine and mundane that we say, Hey, you know what? AI agent, you make that change for me, monitor that change and roll that change back if it has an adverse effect that we didn't expect.

Right? And I mean it regular and pointed out that the, there's the trust factor and right now it simply makes recommendations without actually acting on them. So it it can help, so it can build the trust.

Yep. And part of the trust is, hey, I've seen this. I, I as the agent, I've seen this before in another network and I've made this change and this was the, this, you know, based on the, the millions of, of pieces of knowledge I've got about networks and the effect of making these sort of changes, I believe this is the best course of action for you.

Right? And you will over time build that trust as you see that, do that and enhance that workflow. There's, there's another aspect to this, which I think we're somewhat alluding to, which is, and you know, and I'm a reformer reform software and hardware engineer and you know, I no longer code and 99% of the people no longer code in assembly because we trust compilers to do that job for us.

And in fact, most people no longer code in C because we trust a high level compiler to do it in a very high level language. And one way to look at the LLMs is we're, we're interacting with this complex machinery at a very high level instead of at the very low level now. And it allows us to go from the pain and effort of having to manually tweak every single knob in the environment to, and to look at every single configurable item to see exactly what the current state is, to have the AI do a lot of that heavy lifting for us so that the human in the loop is involved in, I think this is where you're driving at the human in the loop is making a higher level decision making.

It's not saying tweak it. I want to tweak this particular command line, do this command line to make this parameter on this particular interface change. Right?

It's basically saying the AI is saying, I'm applying, it changes, and this is the list of changes. Does that look good to you? Just review it rather than thinking about it all.

And in fact, at some point it should be able to say, I'm going to make a change to make the network do this at a very high level instead of change interface one, interface two, and have a thousand changes. All of that's gonna be hidden by an higher level language. And that the value of that is tremendous in terms of changing the workload on the limited, the most limited resource we have in the enterprise is the humans, right?

We, we have infinite number of compute cycles relative to humans. And, and I mean, I I think that if I was a, a network administrator, I might want both. I might have, I might want something that says, we're gonna apply this set of commands to these switches and this is what this, this set of commands does.

Yep. And where, where the, the, the element of this is that, that it guess it's a true intent based networking. We've talked about it for a long time.

My ability to express in English what I would like the network to do and have it implement it doesn't mean that you, you can, I can have my 15-year-old son walk in there and with no networking knowledge and do that, you still need to know what you're doing. But a great example of that is we built into, uh, there's this approach we're taking within my product, uh, portfolio called Unified Branch, and the ability to deploy a full stack of networking in a repeatable fashion across thousands up to, you know, 10,000 plus locations. And it starts with going into the assistant and saying, please build me a branch design based on a Cisco validated design for unified branch.

And because it's a validated design, because we've tested it, we pull forward what we believe the right, uh, defaults are, and you, you manually inspect those, you know, you can tweak some things and change some things and then when you press go, that agent goes in the background as the logical deployment associated with that network. And what you're really doing then is yeah, reducing the amount of time upfront, reducing the risk of error. And when it comes time to do that install, you're reducing the technical capability of the person going on site to do that install because it effectively becomes plug and play all the design work and all the configurations done, send somebody out there that understands low voltage cabling and how to, how to physically wire things up and physically mount things.

And, uh, they don't need to know DHCP or DNS or any of the things that, you know, a network engineer would need to know to go configure that from scratch. If I, if I can mention another topic here, the, one of the other things that kind of fascinated me from this discussion was the define time management, uh, aspect of the wireless networking where, uh, it, the way it was described, it could almost paint a picture of where the wireless devices are in an environment without having any cameras. And that, that I thought was kind of cool.

The idea that just to define time management, it can actually locate in a, in a room or in a, a building where all the wireless devices are and essentially draw you a map to them. I think our ability to match the physical world and the digital world together in an interesting way. And sometimes that means physical sensors, cameras and what have you, but we can infer a lot from that digital footprint the devices leave behind in order to be able to paint a pretty rich picture there.

Uh, wireless portfolio is super interesting. They've been doing AI before it was really, you know, the, the thing Azure Radio resource management, having a, a, a model that understands making changes. 'cause no, you're typically not tuning things like channel selection channel with someone have you on the fly manually in a network.

So the ability, again, using that, those large data sets to say, look at all the radio environments that I understand and what's, you know, almost like a digital twin approach, what's similar about this one to other networks I've seen? And if I make this change, this is the expected result. And it's able to read that out to you, but it's not gonna come to you and say, Hey, I suggest we we turn DFS off on this channel, or we move from this channel to this channel, or we, we lower this channel with, that's not the way our radios work.

They've tended to self optimize for a time, but AI has allowed that to get much more effective and, and much more better understanding causality. I make this change, I see this result, and if that result is good, I leave it. If it's not, I roll it back and try something different.

What's interesting also is that the wireless has gotten to a point now where it's not either a secondary communication channel or just for users and laptops, right? We're now using wireless, as you said in robotics, right? And ai, we are moving enough data across the, the wireless networks that it can be a backbone, particularly at the edge for your AI information that's moving back and forth and what you're doing with ai ai.

And I think that that's, you know, we have to change our perspective and, you know, on, on what the, the communication channel really is and why we're using it. And, you know, why would you drag a wire any everywhere if you, if wireless is capable of doing all of these things. Um, some of the things that wa Cisco presented was about the, and I don't remember the name off the top of my head, but the, the, the multiple antennas, and I know there's m mi, MO and, but there's just, you have so many antennas and the ability to do so much with that, that gets you much better, uh, interference, uh, capabilities and, uh, uh, was it, uh, reliable wireless networking?

I think ultra reliable. Yeah, ultra reliable wireless back. It's a super interesting product because I mentioned those robots don't have time to roam from one access point to another access point the way a traditional client would.

And so we've built the ability to run standard wifi your, your standard, uh, IT environment combined with your OT environment in the same hardware unit, and be able to manage those through the same domain. And the robots are gonna use the, the ability to connect to multiple access points at once and your standard device, you know, your, your, your, um, tablets and things you might be using as a human in that, that factory space or in that warehouse space, continue to use standard wifi. So really, really interesting, uh, evolution of how we do things.

And again, it just means we're getting richer and deeper sets of data about what goes on in those environments. And our ability to do that inferencing and be able to make intelligent decisions about that is what's gonna drive a higher quality network for our customers. And it should largely be invisible.

I, I joke that the, the networks like oxygen, you only notice that if you're not getting enough, right? It should really be in the background there supporting the things you're trying to do and be able to, to do that in a very intelligent way. Well, I think that's, that's actually a very interesting perspective in looking at it for ai, right?

And thinking about the AI network, and right now, I don't think we can look at it as oxygen. When you think about in, you know, a lot of what we're talking about both AI for networking and networking for ai, but specifically networking for AI is we are placing such heavy demands on the network that we can't, we're not at the stage right now where we can treat it for as, as oxygen where it's just in the background. It is a critical component that if it is not, not architected correctly, the network itself isn't designed correctly, then the AI is not going to work the way we want it to.

Yeah. And that plays both in, in data center, but also in, in the wider, you know, across the multiple presentations we had from different business units within Cisco AI and infrastructure four, we saw that, as I say, the, this is, no, not yet a place where you can say a standard network design is gonna be ample for everything I need. I'm not gonna about, I'm constrained by both data center and and beyond out to edge.

And we already know that networking out to, to cloud is also a significant impact for us. Um, yeah, it's, it's a long way before we can just assume the network is good enough because it's there. And I think that's, that's a path that, that Cisco is, is definitely on.

And I think within the Cisco teams as the, this skating well ahead of, of, of where the puck is for most customers at the moment, because these technologies do take time to get out. We, we are seeing, uh, a bit of a rush amongst customers to, to realize that they're networking inside their own organization needs to get better to deploy AI as a production workload, as something that's delivering business value. And yeah, it'll be a little while before that.

It's, it's an oxygen kind of supply, Right? Yeah. And, and I mean, one of the things that Cisco brought out was the fact that they're merging essentially their on premises data center or, uh, enterprise networking tools with their, uh, cloud networking tools.

And they talked about the features that these actually can also look at software defined wins well, and be able to manage those the same way as well. One of the aspects we really haven't talked about here, I'm sure Jack would love to talk about, is all the additional security that this, these things bring up. Uh, we talked some about secure routing last week and all of the, the additional features that were needed for secure routing.

And I don't think that we really have paid enough attention to how much additional, uh, infrastructure and horsepower is needed to actually do the level of security that we are gonna need in an AI ready world. Yeah. One of, one of the things that Cisco is, I think, very proud of and rightly so, is, uh, the inclusion of, uh, PQT or post quantum cryptography in the hardware, uh, you know, in the devices, and particularly having hardware enablement for it, because as Andy said, um, doing cryptography and alone is very, uh, is computationally expensive.

So the ability to have hardware that accelerates it and have, uh, the post quantum algorithms involved means that we're already prepping for it. We run have a rate, great risk right now of what's called harvest now, decrypt later, where somebody grabs the data, holds onto it for a while until that data can be decrypted. And a lot of the data that we have is very ephemeral.

We don't really care about it after a couple of minutes, but there's a lot of critical data that crosses a network that is actually very long lived. That is, should somebody get a hold of it, it becomes very, you know, and a lot of that we think about things like PII like our, our personal identifier, social security numbers, or whatever the equivalent is in the European countries. Should somebody get a hold of that now, then, you know, at some future date they can decrypt that, get access to it, that opens up a whole ball of wax that we don't wanna, you know, we don't want people to get access to that data.

So the ability to have a quantum safe encryption of that type of data is very critical, and we need to do that now rather than tomorrow. I was gonna say, some of the ephemeral data that you talk about, Jack, is, uh, is ephemeral to the source, source and destination of it right then, but if it's information that can be held onto and decrypted later, it can be used to do predictive, predictive analysis of what's gonna happen. And think about this in terms of robots, where if you're able to gather the information that's being fed to a robot, uh, you, you can probably say, the robot did this, and therefore if we wanna interfere with that robot's operation, uh, we know that the robot's gonna be doing this at this time so that we can interfere with it there.

And so it's one of those things where it's not just personal identification, but it's also being able to pick out patterns and use them, uh, against both people. And, and, um, robots, Before we dive to get too far into this, before we get too far into this rabbit hole, I'm gonna have to put an end to this conversation because as always, when I get a group of my delegates together and particularly bring along some expert from within one of our, our sponsoring companies, the conversation could go on for hours. And I know we're already about type, so if people do want to continue this conversation after they've, uh, listened to this podcast, where can people connect with you to continue that?

I answer every DM on LinkedIn, so my LinkedIn is open looking me up on, on LinkedIn, not hard to find. Um, that's the best way to start the conversation. I'm, I'm more active there than I am on, on other social channels.

And, uh, you can also find me on LinkedIn as well through dms or at, uh, paradigm technica com. And you can find me on LinkedIn social, and, uh, on my periodic blog Analyst. You can find me on tech field day com.

You can also find me on LinkedIn and across all of your favorite social media, the same as you can find all of the Tech Field Day team on all of the social medias. Also watch for us on Techstrong TV and across a bunch of the other places that we work with the wider future and groups. So thank you so much for to this episode of The Field Day podcast, and please subscribe on YouTube or in your favorite podcast application.

So don't miss an episode. Make to give us a, a nice high rating, nice review as well, so other people can find us. And this podcast was brought to you by Cisco Tech, the home IT experts from across the enterprise of events.

More episode podcast on. Thanks. Listen, we see.