Managing Edge Devices and AI with Esper’s Sudhir Reddy

This is Textron tv. Hey, everybody, have a great pleasure having a very cool conversation. I'd like to introduce you to Shir.

Ready. Shir is CTO with Esper. Welcome.

Thank you, Mitch. It's a pleasure, and I'm super excited about this discussion. Uh, it's a great topic.

We'll get to the topic. I would like you to introduce yourself, um, and tell us about Esper for folks that may not know the company. Yeah, I'll start with Esper.

So Esper is in the world of, uh, device management, and we do it in a unique way, which is really called DevOps for devices. And the idea is to bring a lot of the great practices we have in the DevOps world, where it's pretty common in the cloud and, uh, infrastructure to bring that to devices and enable our customers to really manage a large fleet of devices or a small one with, uh, with, uh, with a lot of ease. And we'll talk about some of the concepts, uh, as we go along here, but that's what Esper does.

And I am the CTO here. I run the engineering organization as well as what I call the customer delight organization, which is really, uh, all the touch points with the customer on support and services and all of those things. And, uh, that's what we do here.

Seems like a really great connection. Having the technical role that you get a lot of connection to the customer. It's, uh, that's gotta help inform your, your other job.

Yeah, it's great because I get to talk to customers. I get a lot of energy out of it, and a lot of understanding of what our customers want to do, or even sometimes problems that they don't know can be solved. And, uh, we help solve that.

So it's a, it's a unique way of running things. Very good. Well, I'm excited to get to the DevOps and, uh, device idea conversation.

Maybe we should talk first about set things context up for us, because we're talking about devices that are edge devices, right? Mm-Hmm. Um, and that can mean a lot, a lot of things.

What is the edge? What is it? Is it smartphones and computers?

Are we talking about, you know, teller terminals and, you know, um, you know, uh, checkout devices in a, in a shopping center, whatever it might be. Give us some context for when we talk about devices at the edge. Yeah.

You know, everybody, uh, we talk to 10 people and they have 10 definitions for what the edge looks like. But what, when we are talking about edge devices here, we're talking about anything that is mission critical that businesses deploy at various sites that are distributed across, uh, across their business locations. So think of anywhere from kiosks that are in airports that are displaying in an advertisement or are interacting with passengers all the way to, uh, restaurants, having their, uh, ordering systems or payment systems to back of the house, kitchen display systems and things like that.

And if you talk about the medical world, it could be anywhere from a, uh, uh, a patient interaction, a patient survey tablet that, uh, customers have all the way to, people don't think of these as devices, but MRI machines and all of those things are really devices that are all interconnected and doing various things for, uh, for customers. And, um, and not to forget the IOT world where there's a ton of IO iot devices that then connect to a central server in the location that then could talk to the cloud and things. So we encompass all of these things into what we term as edge devices, and we're in the business of managing all of those in a very efficient way In Interesting.

You brought up a couple examples, and it makes me think about even, even kiosks are now becoming, look, they look like signage. You sometimes can't tell that that's a digital sign, that a menu that I'm looking at a restaurant or something in an airport, like, oh, actually that's digital. It's not, not not a yeah.

Printed thing. It, it's amazing how the, uh, the difference between them are, are somehow times hard to tell. And that's intentional a lot of times because, uh, the same device may d display advertisement when there's low traffic and maybe do something more interactive when, uh, when there are people passing by or people stopped in front of it.

And that's where, by the way, when we talk about AI and things, it's those decision making, uh, processes. It's then how does the device interact with the, with the human user. All of those things are really, uh, the power of what devices bring to, to businesses.

Very good. So talk about edge device management then. Um, you know, that, that too has, you know, kind of a loaded, maybe overloaded term could mean lots of different things.

Give us some some definition for that. Yeah. So when we talk about managing these devices, there used to be a time when these devices where there was a central server and you would upload an application to these devices and you got what you need and you, you move on.

But the world has become much more sophisticated than that and complicated. I, I might add, and you have to think about managing these large fleets at scale. So we have customers that run 50, 60,000 devices in their fleet all doing where, think of large restaurant chains where every single one of their franchisees has 10 to 12 devices in a, in a restaurant that need to be managed.

And now the, the problem becomes, you have two things here. It's all about the application. There is an application such as a menu system or in the case of advertising content that needs to get on these devices.

And you need to be able to manage them at scale without having to worry about you. You want to manage the forest, uh, in, in that sense, but also get down to the tree when you need to get to the tree. Uh, so let's say one device was compromised for security, you need to be able to get to that device and do the things that you need to, uh, with, with that device that's tighten up the security or install a new application on it.

That's what we talk about when we talk about, uh, managing these devices. It's getting applications onto the device, but also being able to lock down all of the settings and the, and the various things that, uh, could be, uh, and when I tell say settings, it could be anywhere from brightness to wall to things like that. But also you may want to shut off the USB uh, connector on that device because somebody could plug a USB stick in and do things that, uh, you don't want them to do.

Well, we know that would never happen, but yes, I'll start that in there. I Have never been, it absolutely will happen if it's possible. I've never been found guilty of trying that.

Okay. At least we won't discuss it here if you have. Oh, yeah.

So, so you, you, we started this talking about, you know, applying some of the principles of DevOps to devices at the edge device management, and you're talking to a, you know, one of my favorite topics on, on DevOps. Yeah. It's interesting how DevOps is more than just about creating software.

Those principles have shown up into, into security, into platform engineering, into SRE, into finops, and, you know, kind of X ops, right? Talk about how it applies in this context. Yeah, it's fascinating because the DevOps world, and I, I've been a long, uh, practitioner of DevOps.

I, I used to run engineering at Chef Software before I came to Esper. Uh, so DevOps is in my blood, so to speak. And the interesting thing is, uh, as much as it is about software, like you said, it's also what the mindset and the way you do things and how you think about solving things, concepts like shift left and, uh, things, uh, come in.

The device world has not, uh, really advanced in that, uh, space to the extent where the rest of the, uh, the worlds have, uh, like cloud management is now DevOps. There is pretty much a, you don't even ask if that's another a question you ask, whereas in, uh, in the device world, it's really been very traditional ways of doing things. And we're revolutionizing that with bringing in, uh, DevOps concepts and these concepts to be specific for our DevOps.

Uh, for people who know DevOps are, we talk about desired state management, where you really describe the desired state for a device, all the things that you want on that device, you just describe it in, in a software, uh, artifact. And then as per or other tooling can take care of making sure that device not only gets to that desired state, but also stays in that desired state. Uh, that's where concepts like the drift management come into play where, um, hey, your desired state is X and your devices actually in y somebody went and installed an app they shouldn't be installing, uh, and things like that.

And, uh, you can automatically bring those devices back into that thing. Now, when we talk about all of that, it's also important that compliance is a huge part of that, right? A lot of companies have compliance requirements, whether it's mandated by a regulatory authority or whether it's a self-imposed.

Uh, every company has the policies they need to, uh, ensure that also is encapsulated in this desired state. And now your auditing becomes so much quicker and so much easier to do. And, uh, you can do the continuous loop of, uh, bringing in your auditors and things there.

So I'm meandering a little bit around there, but that's, you know, bringing all of those concepts of DevOps into what, uh, you could do with managing devices is The, it's interesting, uh, not, not meandering at all, because you've described so many things that are fundamental to DevOps. One of them is automation, right? Mm-Hmm.

So in doing things in a declarative way, you're not writing tons of scripts for every variation and everything in the world, whether it's A-C-I-C-D system or, or end points, you know, point of sale devices versus, you know, menuing systems. You, you also talked about operating at scale. Mm-Hmm.

But also being able to handle an individual item, uh, or issue when you need to set the context of that. And, and also applying some policy. We don't necessarily talk about DevOps as policy, but when you build pipelines and you build processes and flow of work, the same thing happens in, I imagine in device work, a device management, right?

It's about deploying it, prepping it to get it into the state that you want. Maintaining that state reporting for compliance reasons, right? Just like we have to run vulnerability scans on software in a DevOps lifecycle, we need to provide compliance reporting to say, yeah, here's these devices.

These are the policies, they are provisioned. That way we know they've stayed that way, you know, which isn't been out there plugging in USBs, or if he has it doesn't matter. 'cause that's all locked down.

Yep. Um, so you're talking about, oh, well, it is end to end kind of a lifecycle conversation when you talk about DevOps. Am I, am I summarizing this correctly?

That's absolutely right. And, uh, and all of those concepts are very nascent in the device world, and that's what, um, our mission here is to revolutionize that world. Interesting.

So talk a little bit about, um, since you're so work close to working with customers, how are they adopting that idea? I don't know if you're directly describing it as DevOps, you know, for endpoints, but thinking of some of those principles and things, are those things that come naturally to, okay, I I get why we do it this way and, and some of the benefits of that? Yeah.

You find, you know, that's a very interesting question. You find a range of awareness of, uh, of these concepts here. And before I go further, one of the things that is a great way to, to occur to, uh, to help our customers understand what we're doing is to think of, uh, what happens with the cd, the continuous delivery or deployment of your applications.

Remember, for these customers, it's all about their mission critical applications running on these devices. And they have a very sophisticated engineering operation in the, in the house where they're building up the, the application and all the backend stuff that goes with it. They have all kinds of, uh, uh, the continuous integration and all of that.

And finally, they have an artifact, which is the application that needs to now get deployed onto these devices, the tens of thousands of devices. And, uh, obviously you don't want to go on mass and, uh, deploy to all of them at, uh, at one go. In fact, one of the recent big worldwide incidents that happened was because the, it wasn't deployed in a, in a systematic way.

And that's where the power of, uh, continuous delivery takes over. And that's what we enable, which is now you have the, a way to take that artifact and deploy to a certain number of, uh, devices, first to run your test cycles, make sure everything's working fine. And then one of our, going to your question, one of our large customers, the way they do it is they, this is a restaurant chain I'm talking about, and they deploy to their test environment, of course, but then they, they roll it out to a city first and make sure that everything works in that city for, for a week or so, and then they roll it out to a region and then nationwide.

Mm-Hmm. So we enable all of that power to, uh, to, to the customers. And when you explain it to them in those terms of here, Hey, here's how you could be doing it, and this is all automated, by the way.

You don't have to have a, uh, along with the fleet of devices, you don't need a fleet of people to do this work. Um, they get it, they usually get it. Now the question becomes, now that you've gotten it, how do you put that into practice?

And that's where we help a lot of our customers, uh, understand what are the steps to get on that, what we call the DevOps maturity curve as, uh, you might have heard before also. Mm-Hmm. It, it, it makes a lot of sense because you can also connect the front end creation of software, right?

And that process using DevOps with the deployment that you talked about. And, you know, we, I spent some time in iot and one of the things I quickly learned is there isn't just one version of that device out there. Mm-Hmm.

There are thousands, potentially thousand, hundreds variations in certificates expire on these, but not on those or whatever the issue might be that you have to do with managing the device. So the permutations and the complexity is the idea of throwing an artifact over the wall, or here's five or six of them to go to out into the network, have fun, you know, operations is, you know, totally impractical. So the more you can connect the two together, so you've got our true flow workflow Mm-Hmm.

Because you need to do updates as you need to do security policy, right? Whatever it might be. Um, it isn't a disconnected and, uh, out of context thing that you've gotta reassemble things in a way that makes sense of how you gotta deploy it.

You can set up your test environment, you can set up your variation, customer unique environments, right? Mm-Hmm. If you have some special, uh, context that you have to test for and deployment strategies, right?

One company may be we're healthcare, we're gonna be very measured and controlled about how that goes out. Um, we're more retail. We could be a little more flexible and aggressive in some markets and, uh, Denver's our test market or whatever it might be, right?

That's typically how they do this kind of things. Yeah. Very cool.

Um, so you brought up the word, I gotta go there talking about ai, you know, you would brought it up. I would've, so, so give us that. How does that fit into this picture?

Uh, it's a fascinating world out there. So, uh, obviously the, the state of play of AI is, uh, there's a lot of, uh, it is on a hype cycle in terms of there's a lot of problems that customers are facing that need to be solved, but also there's a lot of research and, uh, new ways of using AI that's in the world. But where we come in is AI is, you can think of it as a running on a central cloud infrastructure somewhere, or you can think of it as running on the edge device or a combination of those.

And more and more customers and more and more practitioners are finding that it is much more convenient and useful to run these models on edge devices, think of loyalty programs in a restaurant, or think of, uh, medical records right in front of the doctor and getting, uh, insights into what the doctor should be doing right on the device that they're working on. So the point is, more and more of these AI workloads are actually being run on the edge at this point. And, uh, that that is only increasing.

And where we come into play is how do you then, just like an application, these AI models are not static models. They're models that, uh, uh, are continuously refined in a, in an engineering shop and need to then be, uh, delivered to these devices on a regular basis. Not only that, the amount of data that is collected by these, uh, by these algorithms really then needs to be synthesized locally, uh, but also transferred relevant data needs to be transferred to the cloud.

And, uh, then process for a larger aggregated sort of, uh, uh, analysis and things. That's where, uh, you know, a lot of times when customers spend energy and time on building out AI models, they don't think about how do I then deliver this in an efficient manner to my devices? How does it run on these devices?

And the data then goes back and forth. That's where I think the power of DevOps and what we do really also helps with, with that. Uh, if you think of an AI model, at the very core, if you take off all the algorithmic stuff in there, it's an artifact that needs to get delivered to the, to the device along with some runtime environments and, and things.

And, uh, we have all the capabilities to help do that efficiently as well. So that's, uh, that's where the AI world is going with, uh, especially when it comes to iot and edge devices and, uh, and these things. And, uh, we're uniquely, uh, positioned to support our customers on that.

You know, I, I concur, and I think it's also an area where there's a lot of learning for folks to, uh, you know, how do you incorporate n ml, uh, algorithms and model and data representation that it needs to be in and packaging the data with it that needs to go there as well. What as well as what's gonna be on the device, you know, it has its own kind of workflow feature set development and things like that that they talk about in the AI world, but it has to fit in with a flow, with a, you know, with a workflow in DevOps world, including getting onto the device and securing it and you know, all the requirements that you're gonna have there. So I would imagine having a pa, a partner like Esper who is living the dream, you'll living in that world with your customers, you know, as you're li living, as you're learning on macro, working with multi multiple customers, that benefits for folks that maybe you're a little bit more new into delivering AI to the edge.

Yeah. Yeah. Uh, and, and, you know, all of this is also coming unique new, uh, uh, ideas of how AI could be solving the, the medical, uh, situation that I was talking about is if a doctor is sitting there and diagnosing their patient and right in front of them is all the information that doesn't, uh, oftentimes because of the latencies associated with talking to the cloud and then getting analysis back, et cetera, all of the processing is happening locally and you instantly have insights and analysis and, uh, thing that help the doctor make a better diagnosis or a better decision, et cetera.

So those are things that are really it, you know, uh, one of the things that really gets me excited when I look at the world today is all the possibilities that we could be solving that, that are pains right now, to be honest. Yeah. Fantastic.

Well, um, you have a webinar coming up on, uh, September 24th, 3:00 PM Eastern Time. com. You'll see a webinars menu and either go to upcoming or if this, you're watching this after the 24th, uh, go to on demand and you'll quickly find it, uh, beyond 2024, preparing, uh, your device fleet for what's next, kind of what we've been talking about, at least a good part of it.

Um, what, what kind of things do you hope folks will get out of, uh, being part of that webinar with you? Yeah, Uh, thank you for pitching the webinar and, uh, it's going to be really useful and exciting for our customers. I'll be hosting the webinar and I'll be on there talking about things and, uh, ju just recently I wrote a book called The Practical Guide to Preparing Edge Device Fleets for the Future.

And that's really what it's about, is how do you prepare your device fleets to, to not only be ready for the workloads and things that you're trying to do today, but also future proof them and future proof your processes and practices and, uh, mindset. So it'll be a exciting conversation about that. And, uh, and it'll be engaging and interactive, so it'll be great, uh, great experience and learning for everyone and, uh, I want to learn some things out of that webinar too.

Great. So looking forward to that. Hope you have a lot of audience interaction with that too on September 24th, 3:00 PM Eastern Time.

io is your website. Correct. What kind of resources things might be helpful to folks there?

io is our website. Uh, it has a ton of obviously all the salesy stuff, but also there's information there on how to get started with device fleet management, uh, about specifics on Android devices and iOS devices and things. And, uh, the blatant plug for my own, uh, blog post there, there's a ton of, uh, information that I've written, uh, I'd love to share with, uh, with all the listeners and, uh, would love to get feedback on that too.

Very good. I've enjoyed reading your blog as well, so you Sure. And check that out.

Uh, well, Shir, it has been fantastic talking with you and hope you have a great event on the 24th and beyond. We've been talking with, uh, Shir Reddy, who's CTO and author with Esper about, uh, device management at the Edge, AI and everything else, uh, today and beyond. So be sure to check out the webinar that's coming up.

Thanks. Great talking with you, man. Thank you, Mitch.

This is fantastic. You bet. Thanks everybody for listening in.