09×06: AI Agents are Coming to the Real World with Olivier Blanchard of the Futurum Group

We are surrounded by intelligent devices, and these increasingly use various AI interfaces and processing capabilities to support the needs of users. This episode of Utilizing Tech focuses on Edge AI and AI assistance and agents with Olivier Blanchard from the Futurum Group. Welcome to Utilizing Tech, the podcast about emerging technology from Tech Field Day part of the Futurum Group.

This brand new season focuses on practical applications of ag, agentic, ai, and other related innovations in artificial intelligence. I'm your host, Steven Foskett, organizer of Tech Field Day events for rum. And joining me this week as my co-host is Mr.

Frederick Van Hern. Frederick, welcome to the show as always. Yeah, thanks.

Glad to be here. So, I'm Frederick Van Hern. I'm the founder and CTO of ens.

It's an HPC and AI consulting company and services, You know, Frederick, um, you've been involved in, uh, AI for a long, long time. Uh, you worked with a lot of, uh, well, I guess what people thought were AI before chat, GPT came out, those AI assistance, uh, like the S word and the a word that I'm not gonna say 'cause I don't wanna activate everybody's assistance here. Um, you know, those are also AI agents, right?

Yeah, to a certain degree. There are agents. I mean, there are kind of the basic elements of agentic ai.

So Agentic AI consists of agents, you know, there's multimodal one of speech, there are other components to it, but speech is the way we interact with humans, and that's, that's a very important interface. And increasingly, um, well actually even today and, and increasingly in the future, those, um, types of applications are interacting with other elements on the back end. So for example, if you ask the, uh, s lady for a sports score, she may actually tell you the sports score.

And if you ask the a lady to turn on and off your lights, she may do it for you. Um, if those things are connected, um, I could see a crossover between these technologies. Could you?

Yeah, definitely. I mean, it's, it's, uh, it's, it's such an important piece to see those technologies advance and getting kind of, let's call 'em smarter or maybe be able to reflect. And that's really where AI is going, right?

It's the ability to be a little bit smarter than the previous generation. And I think a agentic AI is, is a tool that can help us to get there. So when we think of these AI agents, uh, of course we think of, uh, smart PCs and mobile devices and all these other AI power devices in our lives.

And so that's why I am thrilled to welcome on this show today. Uh, somebody from the Futurum Group who focuses on that exact area, Olivier Blanchard, welcome to the show. Hey, thanks for having me.

Uh, yes, I should probably introduce myself real quickly. So I'm Olivia Blanchard, first time on the show. Um, I'm a research director with futurum, and my focus is, uh, intelligent devices.

So basically from the hardware full stack all the way to the, uh, user experience. And so, um, to your point just a moment ago, that includes everything that's on the edge. So wearables like, uh, smart watches, uh, xr, hearables, like, uh, uh, like Smart buds, the, the Smart Rings, mobile devices, tablets, PCs, smart tv, smart speakers, the whole smart home thing, all the way out to, uh, smart vehicles.

Uh, and that's on connectivity, a DAS and all of the intelligent solutions packed into it, uh, and all of the iot and iot. So, um, part of my focus at, uh, at Futurum is helping answer questions around the edge, uh, edge AI and how assistance and AG agentic AI work at the Edge as part of a bigger ecosystem of sort of like cloud to edge AI and agentic. And so I, I sort of own that piece of it, uh, which is still sort of a, a work in progress, not for me, but for the industry.

A absolutely it is. And, um, I think that a lot of people got a little frustrated with those agents early on, but frankly, I, I, I have to admit, I never did. I remained enthusiastic for the, the, the Apple S lady since I very first, you know, since my very first try.

Uh, simply because it is so remarkable to be able to interact with a computing device and with online services in such a natural way by just talking to it. Now sometimes it doesn't understand what I'm saying, and sometimes it can't do those things. Um, but it's pretty impressive what's happened behind the scenes.

I think people don't really give Apple and especially Google and Microsoft credit for the development that they've done on those assistants to really make them intelligent. Is that what you've seen over the years? Are you asking me?

Yeah, yeah. I, I agree. Um, I, I would add Alexa to this.

Amazon has done just a fantastic job of, of creating a, um, a natural language first assistant, an age agentic platform, as opposed to some of the other ones that are sort of like either, uh, you know, typed or tapped on a screen or language. And so, um, I think it was, it was a little bit harder for Amazon to do what it, what it does. And also it created this, this ecosystem that's much more contained where Gemini, Siri, um, and, uh, and others are sort of like out pretty much anywhere, whatever the device is on your phone, on your pc, uh, Amazon decided to do something that's more tied into the home or tied into a vehicle, uh, cabin.

Uh, and, and the use cases for that are, seem like there would be a lot of overlap, but they're actually very different, uh, and very unique. And so, uh, I I feel like we already have some differentiation and some, some different sort of trajectories, uh, for AgTech and, and AI assistance in, in the ecosystem that that's visible and that's already in existence today. So what do you see being AgTech?

What does that mean to you? I mean, you we're talking about intelligence and, and at the eng. What does that mean for you at Gentech ai?

For me, it, it means essentially having, uh, the, the ability, whether it's in the cloud or on the device, uh, and, and hypothetically, it, it shouldn't matter to the user where the processing does or happens. Um, but it's, it's the ability to be able to assign a task, uh, to a specific app or through a specific app, or not at all, just basically just by, by creating a prompt, calling up an agent or an assistant and saying, Hey, this is what I'm trying to do, um, you know, make reservations for three at restaurant X for this day, uh, this time. Or, Hey, I'm looking for a gift, uh, for my cousin and they really like horses and chihuahuas.

What should I shop for? Or, uh, you know, setting up an appointment or moving an appointment on my calendar, reading out an email because I'm multitasking and I need to be hands free. It's all of these different tasks that if you had a human assistant holding your devices next to you, they would be able to do this for you.

Uh, but having it sort of ubiquitous, uh, wherever you happen to be, uh, and just be picked up by microphone on a device and, uh, taken care of either in the cloud or on the device, or a combination of both. So I, I think that some people might, um, I don't know, take, you know, issue and say, wait a second, wait a second. You know, Siri is not a Gentech ai.

Um, but here's the thing. So we've been, we've been talking a lot here on this season about what is and isn't agentic ai, and I think you could make a case that essentially these, let's call them, um, uh, AI assistance are coming at the problem of agentic ai, just like enterprise applications are coming at the problem of ag agentic ai. And just like some of the more academic examples that we've seen, and it seems to me that they are all colliding on the same set of frameworks, essentially, that you have a variety of, um, sources that input information, uh, they trigger actions, they communicate with different data services and different platforms in a standardized way.

I mean, sure, um, Alexa may not have used MCP before, but really what was happening underneath, you know, with Apple and HomeKit and things like that was kind of addressing the same need in much the same way that we are hearing about ag agentic AI in, um, sort of more it circles. Um, does, does that sound to you like, like the direction that things are happening? And also do you see a real collision of, you know, agent to agent protocol and a model, you know, context protocol as, as supporting these, uh, AI assistance as well?

I, I think, yes, there there's gonna be some friction, uh, just because, you know, you have some walled gardens. Let's, let's talk about Apple, for instance. Um, and, and Siri and, and Apple's entire stack is, is fairly walled in and, and, and built up in, in a very specific way, uh, where Android and so Gemini is very different.

And then Amazon devices, uh, or Amazon, uh, Alexa with devices and, and licensing to sort of Alexa capable devices is, is a, a third model altogether. Um, I think that what we need to think about, 'cause we can have conversations about is, is something an assistant or an agent, uh, and it's an interesting conversation to have, and it's, it's fun to have that conversation and, and think about what the differences and similarities are, where the overlap ends and begins. Um, but I think that ultimately it's a conversation by hierarchies.

I, I think, I, I don't think of assistance versus agents, uh, in terms of where we are today, where we're still trying to figure out how to make this all work with all these different, um, sort of competing but overlapping ecosystems. I think about where we want to end up, which is sort of like ubiquitous agents and assistants where we don't, we don't have those conversations anymore. It's all part of the same thing.

And the way that I look at it is ultimately, um, what we may end up with is a scenario in which we have one overarching assistant, and that could be the s lady as you call her. Uh, it could be Alexa, it could be something else depending on your environment, right? Alexa might be home, Siri might be your phone, or Gemini might be your phone because your Android in your car, it might be something totally different.

It might be copilot on your pc, who knows? You might be moving around between assistance based on the, the overall use case or your immediate environment. But underneath those assistants that are sort of like the, the, the entryway, like the, the, the butler in chief of, of all of these different agents, the rest of your digital staff, you have these agents that are increasingly specialized.

So you might have agents that are only working for you specifically at your job. There might be some agents that are assigned to your entire business unit, your, uh, that everybody has access to in the business unit. There might be other agents in your company that you don't have access to, or you might only have temporary or limited access to.

And in your home, uh, or in your day-to-day, you might have agents that are specifically designed for certain types of tasks. So I imagine a, an environment where you have an overarching butler or assistance, um, and you might even be able to personalize them by giving them their, your whatever name you want. So it's no longer Gemini or Siri.

It's, you know, it's Martha or Billy Bob. Um, and you're assigning a name and a personality to them. You can customize them any way you want.

And then underneath that, you have agents that specifically or perform specific tasks. So you might have an agent for shopping, you might have an agent for calendar management for the whole family, you, the spouse and the kids. Um, you might have an agent that only takes care of your travel because you love to travel.

You might have an agent that only focuses on your stock portfolio and your financial decisions and your banking, uh, healthcare, all of that. And, and so you create this ecosystem around yourself of assistance and agents. Uh, that might be sort of like very general oversight, uh, type roles and some that might play very specific specialized roles that are very limited in scope.

Um, and, and, and a mix of that might be in the cloud for sort of general use, uh, assistance and agents. And you might have some that are hyper-local and behind a firewall because they're so hyper-personalized and they focus on, uh, tasks that are so sensitive in terms of your data and your privacy, that they're not necessarily safe or you don't feel comfortable having them in the cloud. They're going to be housed in like an AI hub where they're gonna be on your device locked down, uh, so that nobody can harvest your data, nobody can intercept your data or those, uh, those interactions that you have with those, uh, those agents.

Yeah. So when I think about a Gentech ai, I kind of see two tracks. So the first track is, is what you mentioned, you know, apple and Google, who are kind of improving their assistance behind the scenes.

Uh, and then the second track I see is the fact that with ent AI there has been an effort to standardize on the agents. In other words, it's the ability to reuse agents across the board, and it opens up a market for end users to use agen AI without too much knowledge about what's happening, building up, uh, agen ai. Do you see some use cases that are useful for end users with agen ai?

Oh, absolutely. I think another way to think about it is, is right now we still live in an app ecosystem, right? Everything's an app.

And so you have to open the app, you have to tap on your screen or, or do whatever you're doing, whatever the device is. Um, and, and you have to sort of like learn how to navigate an app, whether it's a banking app or work or Slack, whatever it is, even Gmail. Um, you have to find things, you have to know how to, you have to know your word, your way around.

Uh, and generally it's not hands free. I, I think of an agent as sort of like the next interface with apps where apps have an agent or an agentic layer where you don't necessarily have to open the app, um, but it, it's going to understand the, the complete chain of events of tasks that it needs to perform in order to arrive at the task that you want. So instead of opening the app, finding the right screen on the menu, tapping on the thing, sliding the sliders, getting to where, where you want to go, it's basically like, Hey, transfer $500 from account A at this bank to account B at this other bank, uh, on September the 13th.

Or, you know, pay my mortgage, or, you know, do some of those things, um, that normally would take more actions from you. And so I think that that how we're gonna start using the first, we're seeing the first use cases of Ag agentic is through this, this natural language, natural conversational, um, sort of, uh, final task being given to the agents and the agent being essentially sort of like the, the automated task layer of step one, step two, step three, step four, um, this sort of binary, is this the right, is this the right thing or is this the wrong thing? And finding its way to the outcome to deliver the, the, the, the order, the the task order that was given by, by the user.

Um, I already see stuff like this, uh, with, you know, Gemini or OpenAI, for instance. Whether you're creating an image through generative AI or, um, having your, your, whatever, your assistant of choice give you a summary of a movie or making a book recommendation, um, or finding a movie for you on your, uh, you know, Alexa enabled, you know, fire tv. Um, I already see all of those things happening.

Like every morning I wake up and the first thing I do when I'm making my coffee is I, I speak to my, I can't say the name 'cause it might activate half the speakers in, in my office, but I speak to my, uh, a lady device, uh, and ask for, uh, a news summary except excluding sports. I, I'm weird that way. I don't care about sports or not the sports that I would be getting, you know, scores about.

Um, and it does that, and, and I don't have to touch anything. It's really simple. And I think that those are the types of use cases that people are already comfortable with now that they already experienced.

Um, I think they're going to get not just marginally, but significantly better in, in the next year. Um, and we're gonna start seeing that being implemented in more sort of regular productivity apps, uh, whether it's, uh, Google Docs or Microsoft Word, or Microsoft Excel, being able to speak commands about this is what I, this is what I, what I want to end up with, make that happen. Uh, and, and these agents and assistants getting better and better at understanding where they need to get, what the outcome needs to be and how to deliver it, um, in the best way possible, where it's not something that's, you know, 30% baked, and then the user has to do a lot of work on the backend to make it, to edit it, make it right, um, but to get to it as close to a perfect outcome as possible.

And, um, and we're gonna see, we're gonna see some, some significant improvements in that user experience in the next 12 to to 18 months. And, and when it comes to truly ent, um, applications, I know that people have already done this. Uh, I was talking to somebody, uh, who was telling me that they had, uh, they used the Alexa plus ai, SDK, and they built a, um, basically a skill to connect to MCP, and they were able to do basically anything in the universe with Alexa through that skill.

Um, I am also, you know, involved, uh, for example, in the home assistant community, they are a hundred percent focused on basically making their voice assistant be a true agentic platform for home automation and I ot. Um, and, and they're not joking around. I mean, it's not just, you know, we're using that term wrong.

They are a hundred percent, um, trying to end up in the same place that enterprise applications need to be. And, and that's why, you know, I'm most excited about what's happening here with AgTech ai because one of the coolest aspects of it, and one of the coolest kind of, I guess, side effects of MCP is that we have a standard way to specify services and capabilities and data and a standard way for, uh, AI agents to interact, but not just with other AI agents. They can interact with all sorts of things as long as it can be specified that way.

Um, again, you're right that Amazon in my mind was kind of the pioneer of this by allowing people to build their own Alexa skills and connect Alexa to all sorts of amazing things. That's definitely where Google is going. That's definitely where Microsoft is going, uh, as well.

And it's pretty exciting to see where this could end up, but it's also a little bit worrying to see where this could end up. Do I really want to say like, uh, okay, Google delete my production database. Um, I'm not sure I want that.

Uh, what do we say about, uh, protection of these systems? Yeah, well, I mean, obviously they're the, the, you need to have some guardrails in place, and I think that's, um, it's the same if, if you give anybody in, in your organization access to the database, if they could just delete it, uh, at the stroke of a, a, a finger, that would be bad. So, so having the same permissions in the same guardrails for, for ai, uh, is probably a good thing.

Just like even in, in, you know, um, consumer facing applications, having parental controls and having different, you know, like sliders, Hey, you can do this, this, this, and that, but you don't have the ability to do this without some kind of prompt that says, Hey, how are you sure you wanna do this? 'cause this is final. So we're, I think we're used to having this level of sort of natural protection, uh, against bad decisions or in malicious actions.

Um, I don't think they, they change radically, uh, with the use of, of agents. All we're doing really, if you really think about it, is switching from doing things with our fingertips to doing things with voice. Um, but where I think things might get really interesting is as the interface can change, because we're, we're seeing definitely an acceleration in voice interfaces, um, and they can be relatively private, even if you're at a public place by using smart earbuds.

Um, I think that we're going to see smart glasses become much more of a common interface for the, this next phase of, uh, assistant n agentic experiences where I, I don't see the phone disappearing anytime soon. We're not getting rid of screens, but I think that the ability to speak and hear, um, speak with an assistant or an agent and and hear prompts back or responses back in a way that doesn't require us to, to talk into a watch or, or carry a microphone or do something with our phone, um, so that we can do it anywhere, anytime, is, is going to be very high value. And I think the, the utility of, of smart glasses specifically to be able to interact with agents verbally is, uh, it feels natural to me.

And I think that's gonna accelerate adoption of, uh, of that particular type of device. Yeah, we, we, we rely a lot more on conversational agents today than, than ever. I mean, one of the risks is voice clothing, right?

I mean, you're already seeing it where voice cloning used to be like so difficult art nowadays it's a lot easier. So that would be one of my concerns regarding to, um, security and so on. The have been long enough in speech at some point, you know, decades ago we asked ourselves the following question, at what time will speech technology be so good that there is no need for a keyboard?

Do you feel that agen AI is getting there? I mean, it's all conversational. The keyboard is really not a necessary necessarily item anymore, but do you feel that agent AI is getting us closer?

Um, yes and no. I mean, the, yes, the, the quality of, um, speech recognition and, and speech to text is already outstanding and it's only gonna get better. It's not quite there all the time.

Uh, but it can, it's, it's so close. I think within, you know, 16 to 18 months, I think we'll be as close as we need to get with it. Um, the issue isn't so much to replacing keyboards.

I think it replaces keyboards in some instances. I mean, if, if you're trying to record your thoughts or record a conversation and just, you know, work, like essentially have it work as a transcriptionist, transcription works great, it's fantastic. Um, but if you're writing something, whether you're a creative, right, you're writing a script or you're writing a report or, um, you're a student and you need to write an essay on something you're writing poetry, whatever it is, I feel like whether it's a keyboard or even a tablet or you're writing things you know, by hand and then it, it turns your, your handwritten notes into text that can then be exported to a Google Doc or whatever.

I think the act of writing and the pace of putting your thoughts onto a device, whether it's a piece of paper or a screen, is still important. And I don't think that ag agentic capabilities and conversational sort of speech to speech to text capabilities replace that. Um, so my take on that, and it, it might be a generational thing, but I don't really see a, a a time when keyboards disappear completely just because we have, we have voice to task capabilities.

Um, we, we tend to think differently when we speak than when we're actually writing something. And, uh, I don't see that going away anytime soon. I think that's a really good point.

And, and for me at least, um, I don't remember what, who it, who, it was one of the people that was saying it, I think it was John Gruber was saying, uh, that writing is thinking. Yeah. And I completely agree with that.

It's hard for me to truly comprehend something unless I have written it, unless I've written about it. Mm-hmm. And, um, and I think that that's one of the, the challenges of using, uh, text to or speech to text in for me is that I don't feel like it's, it's thinking for me yet, uh, I need to really kind of absorb it.

But, um, there's another aspect of this that I think is important as well. One of the big criticisms that generative AI has faced is the environmental impact, the incredible power and, uh, cooling and footprint of these systems. Well, when we think about AI assistance, they run on a much lighter weight platform.

Um, they are designed to run, as you say, on mobile devices, on smart glasses, on, um, earbuds. And as, as you also, you know, wisely pointed out transparently between your device and the cloud. So you don't need to know or care whether it's running on your earbud, on your phone or on the cloud.

It just works for you in a way that's transparent. Um, I think there's a lot of hope there that, you know, potentially a lot of these concerns about, uh, you know, power and cooling and, and floor space and everything, uh, could be wiped away as we get smaller models that kind of team up in an agentic fashion to meet our needs instead of trying to build sort of this super intelligent super brain. So I guess, do you see that crossover between mobile devices and AI assistance maybe kind of solving some of the problems of AI generally?

I do actually. And, and I wanna draw a distinction between generative AI of, you know, like the, the, the amount of processing power that it takes to from a prompt create entire videos, right? Um, that are, that are very GPU intensive and doing that at scale and having an agent perform a task on a device, right?

Those are very different things. So I think that that generative AI and, and huge large scale use cases, um, are probably driving this, this environmental, this ne potentially negative impact, uh, uh, on, on the environments with the data centers. But as the processors, the, the systems on chip become much more high performance generation over generation, and the models themselves, the software, um, becomes also more efficient generation over generation.

You're starting to see models that were only, that you could only run on the cloud a year ago, um, be compatible. You can run 'em on, on, on pocket devices today that's going to continue to evolve. Um, I, I think that definitely, absolutely, yes, moving a lot of the, a lot of the inference, especially, uh, some of the trading, but mostly inference from the cloud to devices, uh, and including having maybe a, an AI hub at home, something with a GB 10 or even a GB 300 from Nvidia just running on a, on a desktop that's essentially doing, performing a lot of that, those processes, whether it's the, the training of models, training of agents, or the inference of, of doing things locally for at its speed, net security, um, those things can take a lot of that load outta the cloud and move it locally, whether it's, you know, in your pocket on the desktop, in your car, in different, like devices around you.

2 years from now, maybe three years from now, where the orchestration on device where essentially there's the, the processor on device decides, okay, is this something that I could run locally or is this a request that needs to go to the cloud? Or are there portions that I could run locally and portions that need to be run in the cloud, we'll get a lot better. Uh, I think it'll be a lot more mature and, and we will start seeing this sort of orchestrated workload, um, handoff of anything that can be run locally for speed and, and, uh, and security will be run locally and everything else can be pushed to the cloud.

Um, so hopefully that, that, there's a different conversation there about data center buildouts and how the efficiency of, um, device chip sets of models being, becoming more efficient and being able to run more efficiently on, on smaller chips and the ability, once the software ecosystem catches up to run a lot of this stuff locally, might actually deflate the need for as much cloud, uh, and data center build out for AI that we're, we're talking about today. Uh, I, and I think it'll be good for the environment. I think it'll be good for user experiences as well, but it, it might end up sort of capping off demands, uh, for, uh, for the big GPUs and the big CPUs for the data centers.

Yeah. What does Agen AI mean for the AI pc? Can you talk a little bit about that?

Yeah, Gentech AI for, for PC specifically to me, um, there, there're sort of like three sort of major value propositions there. One is security. Uh, I think that running a lot of sensitive, uh, workloads, whether it's the prompts themselves or the data collection or, or even what you're doing, uh, the, the finished product, if you will, being able to run locally or behind a firewall can protect an organization and a user from snooping or from that data ending up in the cloud somewhere.

So I think that, um, if you're a CIO, you want some of those workloads, uh, to be, to be kept inside of, uh, of your organization, whether it's on a copilot plus pc or if it's on a, a more advanced professional level, like, you know, GB 300 powered, uh, a high, high-end A IPC for professionals. So there's, there's that element of it. The other one is productivity.

Um, if you can run models locally, if you can do most of the things that normally you would need a cloud-based agent to do on your device, it doesn't matter if you have bandwidth or not, it doesn't matter if you're at 35,000 feet and the wifi isn't working. You can work from anywhere at all times. You can connect to the internet, you decide you don't wanna connect to the internet, your agenta capabilities are on device.

Um, and the third thing is just speed. The, the, there is lag between, you know, putting out a prompt and waiting for a response. So the workloads happening in the cloud to come back, uh, to your device.

And whether it's a, it's a bandwidth issue or it's, it's another issue, it might be a bandwidth for you in terms of the network. It might be a bandwidth issue with the servers and the cloud, whether it's, you know, sort of like semi-local or it's a, a public cloud somewhere. Um, have the, having the ability to performent tasks directly on device is gonna be a lot faster.

Um, even your interactions with the agents that they're verbal are going to be a lot snappier and they're gonna feel more natural than if you have to wait for a response to like, basically your prompt to go out, a response to come back. Um, all that stuff can happen in semi real time in a way that feels organic. And so you don't, you don't waste time waiting for the processing to, to move back and forth and all the data packets to, to go out and come back in.

Yeah. So it's, it's security, um, it's productivity and, uh, it's, it's, uh, bandwidth. So thank you so much for joining us.

Um, it was great to talk to you about this. It's great to finally have you on one of the shows. I look forward to getting you to an AI Field Day event at some point.

Before we go, uh, where can people continue this conversation with you, Olivia? Well, anywhere they want and anywhere they can, but mostly you can Google me, uh, just know that there's another Olivia Blanchard who is an economist. com.

You can read all of my stuff there. You can also find me on LinkedIn and on x at OA Blanchard. And that, that should be a good start.

com, and later this week you can also see me live as a delegates at AI Field Day. tv for, uh, live video from AI Field Day. So thank you both for joining us, and thank you everyone for listening to this episode of Utilizing Tech.

You can find this pod podcast in your favorite podcast applications as well as on YouTube. If you enjoyed the discussion, please consider leaving us a rating or a nice review. Uh, this podcast was brought to you by Tech Field Day, which is part of the Futurum Group.

For show notes and more episodes though, head over to our dedicated website, which is utilizing tech Do com or find us on Twitter, blue Sky MAs at Utilizing Tech. Thanks for listening, and we will catch.