AI’s Next Frontier: Why Inferencing at the Edge Could Reshape the Industry | Utilizing AI Episode 31

A consensus emerged over the last few years that AI compute would be centralized in massive data centers, but this is being challenged as the industry evolves in 2026. This episode of "Utilizing AI" focuses on the transition from training to inferencing and data centers to the edge, featuring Futurum research directors Olivier Blanchard, Guy Currier, and myself, Steven Foskett. Welcome to "Utilizing AI," the podcast focused on practical applications of artificial intelligence from the Futurum Group.

Every Wednesday, we explore news and use cases of the ways in which AI is transforming enterprise IT and the industries it serves. I'm your host, Steven Foskett, president of the Tech Field Day business unit here at the Futurum Group. And before we dive into the discussion, let's meet who's on the panel today.

I'm Olivier Blanchard. I'm research director with the Futurum Group, and I focus on edge and intelligent devices. I'm Guy Currier.

I'm also an analyst at the Futurum Group, and I'm the chief analyst at the Futurum Group division, Visible Impact. And as I said, I'm the head of the Tech Field Day business unit. We just hosted our AI Field Day event, and we've got an AI Infrastructure Field Day event going on as well.

com. So in the process of kicking off and recording this podcast, something funny kind of occurred, and I think this happens quite a lot when it comes to predictions. The greatest thing about predictions is you don't have to guarantee that they're right.

Well, everyone kind of knew what was going to happen in 2027 and 2028 with AI. Things were going to increasingly standardize on Nvidia. We were going to build out massive data centers.

The Neoclouds were going to kick in. They were going to continue to buy up everything they possibly could, and we were going to invest trillions of dollars, perhaps, in AI infrastructure to continue the AI build-out as companies like OpenAI, Anthropic, Microsoft, Google, et cetera, found ways of monetizing AI. Well, something funny happened in the last few months, and it's probably quite predictable.

Things aren't quite going according to plan. Now, Olivier, I want to start with you on this because I know you've got your thumb on the pulse of edge AI. Talk to us a little bit about how this whole centralized vision isn't really working out the way people had expected.

Yeah, we could talk about this for a long time, and we don't have a lot of time, so let me see if I can make it quick. I think that the initial sort of sales pitch that data centers and AI were going to be able to handle most of the load of all the AI training and all the AI inference especially, was silly on its face to begin with. We were always going to need a more sort of federated hybrid approach, where data centers do handle a lot of the things that data centers do well, but we were going to have to distribute a lot of that compute.

Some of the training, but definitely most of the inference, was going to have to move out towards the edge. So the way that you could think about this is you have your data centers doing some of the services. Doing what data centers do already with other types of compute, but you would also need more localized stations or hubs of compute to handle some of that inference.

And also, whatever you could do to make the inference happen as close to the data as possible, which is at the device level. Your smartphone, your computer, your vehicle, your smart speaker, your smart camera, your smart glasses. The more intelligence and the more AI you can put in those devices, the more immediate and just efficient some of that AI experience or AI-enabled experience can happen.

And then somewhere in the middle, you need a sort of unifying layer that's essentially an orchestration layer, which is the hardest part to do, where essentially some compute in between all of those things, between the cloud, between those sort of intermediate compute areas, and the devices themselves, decides and dictates which packets go where and how it all works together. In a way, it's very similar to how 4G and 5G networks share information packets back and forth, and control all of that at the right speed at the right time. So there's a layer of complexity there that investments in data centers alone didn't really take into account or didn't articulate well for investors, for the public, for a lot of people who are involved in this industry.

And also, I think sort of gave people the wrong idea about the pace and scale of how AI would come into our lives and produce outcomes, like better productivity, or some of the experiences that we were promised on AI PCs, for instance, that just haven't materialized yet. And so here we are in this weird thing where we-- I think the industry overpromised and over-indexed on data centers, and also over-indexed on training, and didn't really lay the groundwork for the inference that's actually probably 4X, at the very least, the amount of compute that we're going to need for AI to actually do anything useful for all of us. Well, I have a little bit of a different take about things not going as planned.

I do come out of more of a data center orientation, but thanks to you, Olivier, and to my Futurum colleagues, along with the experience that we've had for the past few years I've noticed, everyone has noticed, how much AI, which has been at the edge for, let's admit, a very long time, since long before the ChatGPT moment in late 2022, how that is becoming almost a critical foundational application for edge at this point. So I actually gave a talk at AI Field Day based on some of that Futurum research and talking about how edge-based inference has sort of pulled a lot of applications, not out of the core, but towards the edge. My take on how things are not going as planned is, and I'm going to come back to that edge versus core topic, because I think that's a really good one, but my take is I think that people are starting to realize that a whole lot of what their colleagues, other enterprises, software companies, everybody else are claiming they're doing with AI is not actually being done so much and so effectively with AI.

The stories that we hear, it's like the biggest case of a global FOMO ever, which is everybody seems to know somebody or have seen stories about somebody else who has done something incredible with AI. They're running their life on AI. They are running their company on AI with one-third the employees and all this other sort of stuff.

Certainly, the executives in tech, at least seem to have taken this to heart with waves of rolling layouts across the industry, especially in software. But if you talk about this with middle managers, or with software engineers, just to pick on the software business, you find out that a whole lot of the use is the same AI slop-oriented or producing co-pilot and chatbot stuff, and it's not master software engineers orchestrating 100 agents to build applications in 30 days. It's just not.

And we've talked about this in this forum and other forums for quite some time. The stories seem to far outpace the reality. And I have a research background, and I have to tell you, the way you ask people questions in research studies and the way you interpret the results, you can make a lot of headlines out of them when the reality on the ground may not be quite there.

So the way I bring this back into this edge versus core conversation is we are still seemingly as human beings, and certainly as software developers, oriented on nodes, on places, on hosts, on things we can imagine are individual units, even if they are made up of collections of units. And that's how we tend to develop. And Olivia, I know you and I have talked about this before.

The future of the application, whether AI or UX or some combination, whatever it might be, is distributed. It's highly distributed, it's mobile, but we are resisting that move because we are still humans trying to direct all of this stuff. And so the big comedown of 2026, which we predicted for maybe six to nine months, is going to be a lot about all of this stuff turning out to be not quite so true after all.

Yeah. Steven, before you jump in, I'd like to just point something out here. I think the resistance that Guy is talking about isn't just natural human resistance to change.

I think it's very revenue-driven. Right now, we're talking about AI, right? We're talking primarily in terms of spend.

And for some reason, economics be damned, the more spends, the more value. And the equation is basically, if you spend a billion dollars, then you're worth a billion dollars or more, right? It's like this weird sort of cost equals revenue inference when revenue has been not that great in terms of the AI- Well- ...

revenue generation, right? Not the hardware. Obviously, you look, and we're going to talk about this, I'm sure, at some point.

If you're NVIDIA, if you're Dell, if you're the companies that are actually building this out, there's also a lot of revenue there that's working. But in terms of AI, in terms of the tokens, if you're an OpenAI, for instance, you're not really making your money back. And the only way to make your money back that I can see so far is by having a toll booth and basically charging for the service of AI.

Whether it's trading or inference. And the thing is, if you centralize all of your AI compute in the cloud, which you own, which you operate. You're the gatekeeper.

You can charge a fee. You charge a service fee. You are a toll booth.

You are a token meter, and you're charging per token, per meter. If you start moving things to the edge and the inference work, the AI workloads, the compute is happening on your phone or a device that you own and operate, you own your own token machine. You don't have to pay anybody for that.

And so I think that the resistance that we're seeing in terms of bringing AI to the edge is structural, and it's sort of deliberate because there's not a revenue model to capture revenue at the edge for those AI companies that are spending all of this money on essentially building a revenue machine for the cloud, not for the edge. And so I think it's- I would agree and disagree with you, if I may- ... it's an economic problem, not a Oh, okay I would agree and disagree.

I just want to say, you do have a point, but that play would not work without the human psychology behind it. That's all. Well, the human psychology of, I'm not even going to say greed, but just ROI, right?

No, that's the supply side. They're still running. Everybody's running a business, right?

That's the supply side. The demand side is, oh yeah, that makes sense. I can just do everything in the cloud, and it's just going to automatically work everywhere because that's just easier for me to comprehend- No, that's true ...

than trying to figure out some kind of web of services where AI sometimes happens here, always happens here, but only under these circumstances that I've defined with my declarative YAML, and it just becomes complicated. So just screw it. I'm just going to do it all on this platform because I want to show value, and I'm not going to screw around spending three months designing.

Don't have to do that anymore. Well, happily, it is all about people still, because people are the ones making the decisions throughout this. I think that you all are bringing up a really interesting point, though, this question of supply versus demand, the pressures on the supply side.

Olivier, one of the things you talked about there was business model, basically to sell AI, but you qualified it. I noticed you qualified it to say it's not that the business model, generally, for AI isn't working, it's that the business model that we expected for the massive centralized AI data center machines is, let's say, slow to materialize. And in fact, I would even go a little bit further and say that, as it has materialized, as these companies have turned on the cash machine, there has been increasingly a pushback, which is, I think, what I'm hearing Guy say as well, from people to want to pay that toll.

And yet, if you look at what's been announced at all of these events, by the time this is published, we will have had WWDC with Apple as well as Computex, and I can confidently predict that Computex and WWDC and Google, whatever they're going to announce, and whatever MediaTek and Broadcom and Qualcomm and all these companies are announcing, is going to further allow us, as you pointed out, Olivier, to have our own token machine in our own pockets. And there is a business model for that. I would remind all of our listeners who are blinded by the trillions of dollars that are being promised by the SpaceX IPO and the Anthropic and OpenAI IPO, that companies are already making billions of dollars selling AI hardware, and not just Nvidia.

Apple is increasingly reliant on AI hardware revenue in the form of phones and laptops and iPads and so on. So is Qualcomm, so is MediaTek, so is all these makers of portable edge-type equipment. And all of that actually does have a compelling business model, and I think that as the spigots are turned on in the centralized thing, AI world, people are going to start looking for ways of pulling that back to the edge and in their own control from a financial perspective, but also from an operational, organizational governance-type perspective.

And I think that it does bode well to see that these things are going to be moving to the edge. Another topic, Guy, that we talked about at AI Field Day was the emergence of smaller, more tuned, specific models that are good for specific tasks. I've recently seen the emergence of a whole class of models that can do tool use, tool calling in agentic situations locally much, much better, which is something that we desperately need, because right now it's just impossible to do agentic AI with local models unless you have really honking expensive hardware.

But that's going to change, and suddenly we're going to see AI agents running locally. Do you concur here, Olivier? Am I off base?

No, I concur. And I think it's not even a hard problem to solve, right? So if you're not charging for tokens because you're using your own tokens locally and just creating them out of thin air by virtue of owning your AI PC or your DGX Spark or your superior smartphone, you can still charge in other ways, right?

Essentially what we're talking about is software at that level, and software can have basic bronze, silver, gold, premium, extra premium features that you can unlock. You can also include fees or subscriptions for extra storage, right? It's nothing new.

There is a revenue model there. It just hasn't really materialized there. It hasn't been sort of put out or agreed upon, and implemented yet.

It will come, though. There is absolutely no way that I can imagine in this day and age, that technology companies, the AI companies especially, the software companies that are trying to monetize this and have a vested interest in having distributed AI become ubiquitous and essentially become something that we're all addicted to, whether you're an enterprise, a small business, or a consumer, that they're not going to find a way to capitalize on it. And if you're a big platform company like an Amazon or a Google, there might be a business model for you where you don't need to do that because you're going to, or Apple even, by keeping people in your ecosystem, you will make the money down the line somewhere else.

And so by giving the AI for free or more AI for free than your competitors, there are benefits there. The ROI is more scalable But for smaller companies that might have a very unique product that has a moat around it, it offers something very differentiated, but they don't necessarily have that huge ecosystem around it. They might be able to charge a premium or somewhere between basic and premium to monetize those AI features that you can't find anywhere else.

So I think that the tokenization of AI in the cloud makes sense, but the tokenization of AI is not the only revenue model for AI software outside of hardware sales. Even Guy can't disagree with that, I don't think. I'm stunned in disbelief.

AI for free? Olivier, did you say that? AI for free.

It's been a little bit like that, right? I think that a growing story, and I was able to publish a little bit at Futurum about this recently, working with Keith Patrick and a couple of the other analysts over there. Keith runs the enterprise software practice at Futurum.

And software companies incorporating AI have to pay for that AI. And for the most part, they have, even the born-in-the-cloud ones or the born-in-the-AI ones or whatever, they are founding their business model on cloud-based hosting, clouds that they don't own. I don't even think Salesforce owns its cloud, although it may be out of date on that one.

So those tokens they're paying for are not their own. They're buying them. And so as we are starting to restrict the supply because of hostility towards new data centers and because of the great expense of new infrastructure, and it must be said, the over-engineering of the frontier models that all of this relies on, general purpose models built with titanic datasets.

But Steven, your point on smaller models and more svelte AI forms is well taken. But to take it back to our earlier topic, there's a sophistication and nuance to doing this that the human beings participating in this market, in this world, have not reached yet. Microsoft has reached it as a corporate entity.

They've been doing some really amazing work on their enterprise software. A lot of the software companies understand this very deeply. But the ultimate source of all of this, human beings doing stuff, we don't even really understand how the frontier models work to begin with.

Even the engineers who created the frontier models don't really understand what they've created. I've said this over and over again. And so the human time factor of when humans are up to date on all of this means that there's a lot of stuff being built, there's this big demand for stuff, and there's this huge potential for inefficiency, disappointment, and all that other sort of thing.

And what's been built so far is massively expensive. Expensive models, expensive hardware, expensive power that is not in full supply. That infrastructure takes time to build out.

So that's how we're entering into this potential era of a cash crunch or a supply-demand inversion. And so that's my reality. I'm deliberately being unfair to you, Olivier.

Your analysis is nuanced itself and deep. But I think that that's maybe the most basic way that this is not working out as planned, which is that machines are not just taking over and doing everything at lightning speed. People still need to catch up.

And the minute a market goes into something expecting one thing and gets another, that's friction, that's problems that can create that inversion. So over my shoulder right here is a book that Daniel Newman, CEO of Futurum, and I wrote. I've heard of that guy.

Yes. The Daniel Newman. The real Daniel Newman.

We wrote this book, I think it came out just before COVID, and so it didn't do as well as it probably could have. In his book, we're Human/Machine, we were talking about an AI revolution that had not happened yet. And I deliberately opted to not call it AI.

We thought about it in broader terms. I think a lot of what we call AI today is just its marketing, but it's not actual artificial intelligence. Totally separate discussion.

But the argument that we made in the book, which we could have made in one page, but we just wrote several hundred pages to sort of illustrate the point, was that there were essentially two models for AI and for the intelligent machines that we would have, whether it's compute in the cloud or humanoid robots walking around and everything in between, is that the two models were human replacement and human enhancement. Right? So, the argument that we made was that there was a lot more value in surrounding humans with what we now talk about as agents, with assistants, with robots, whether physical robots or virtual robots, to help us with tasks so that we can focus on the things that matter, things that human brains can do better than machines, and then let machines do a lot of the menial, repetitive tasks that are sort of like we don't really want to waste our time on.

And that a worker, no matter what the industry, or a student, or an individual just trying to improve their lives, would be able to do so and be more creative and be more productive and do whatever they want to do better if they surrounded themselves with all these little smart technologies that would just handle things for them. And what we're seeing is this tension still today that we've kind of called out in the book, where a lot of companies are thinking, "All right, how can we lower costs? How can we improve efficiency?

How can we improve outcomes? " And there's still very much a conversation inside of the enterprise, inside of corporate, which is about replacement. Human replacement by AI technology, human replacement by physical robots.

How much money can we save? How much more profitable can we be? How much happier can our investors get if we reduce our human headcounts, and replace them with these efficient machines that can do their jobs?

The disappointments, I think, in that that's coming, that's already happening, but I think that will come in the next few years, especially in the enterprise, will come from decision-makers and managers who really bought into the idea that humans could be taken out of the loop and just replaced by AI. To a certain extent, yes, I think that we can expect some human replacement, like we do with every disruptive cycle of technology, whether it's farming, manufacturing, whatever. It's going to happen.

Some jobs are going to be lost. But I don't think that the expectation that machines can do the things that we do and do it well, is really going to materialize. And I think that we're going to have to circle back and adjust to the argument that we made in the book, which is that the best use of this technology is to have a human at the center and have a bunch of these agents working around and between and for human specialists or human generalists even, to help them be more productive, either do more in the eight hours that they have, or five hours, or 10 hours, or 16 hours, or do the same amount of work that they would've done in a shift or a 16-hour day if you're an analyst, but be able to do it in five or six hours so that they can just go focus on other things and have a better quality of their life, for instance, or work on a passion project while their work is still being done.

That human enhancements by AI, I think, is the more realistic model for these technologies than human replacement. And the disappointments that we're starting to experience or see since in the enterprise will have to, I think, shift strategies back to this human enhancement. And the entire design of the architecture, of the infrastructure of AI is going to have to follow that.

And currently, I don't think that's necessarily how we're thinking about it. I don't think that's how it's being built. I don't think that's how the investments are being directed.

And companies that don't adjust back to the human enhancements model are going to pay a penalty. The companies that transition more quickly to "Let's help our humans do better with AI" will come out of this hole faster. That's my hypothesis.

It is interesting, isn't it, that things don't always go as planned. And I guess that was the hypothesis of this whole program. Once the rubber meets the road and these companies have to figure out how to monetize the things they've built, it isn't going to be the way that they thought that is.

But it's not 100% not going to be the way they thought it is either. I think some people for years now have been pointing out that companies like Microsoft and Google have been very savvy in the way that they've invested in AI. Even though they have thrown huge amounts of money into these pots, they have been left with a very good situation if things don't turn out the way that was expected.

And I think that that's an interesting aspect here as well, and we're starting to see that come to reality, I think. Even though they continue to invest, I think that there's an understanding that Microsoft or Google or other companies like them could turn a corner when they need to in order to try to make these things more profitable and come out looking well. Also, some of the hardware providers.

NVIDIA just keeps reporting huge numbers and I saw a post by Michael Dell on social media that they just deployed the first liquid-cooled Vera Rubin rack in CoreWeave. That doesn't sound like a company that's on the wrong side of the trend. Olivier, are they?

Is it entirely not working out as expected, or are maybe things going pretty well still for these companies? No. That's a great question.

I think if you're in hardware, you're good. AI has obviously been a boon for all the hardware companies, right? Oh, yes.

Everybody's aware of NVIDIA, right? Yes. Exit the commodity hardware- Yeah ...

toot, or at least for the moment. Because everything's- Yeah ... got to be souped up and yeah.

Yeah. Dell had, I think, the best quarter it's ever had, I think. The year-over-year growth this last earnings, which was last week or two weeks ago, was insane.

And especially the data center products, the backlog is massive. And so it's a good time to be Dell, and I think they've done a lot of the right kinds of investments at the right time. And the thing about a company like Dell, and by the way, Lenovo as well.

Lenovo is also doing extremely well. And what differentiates these companies from a lot of other ones is they have the full stack. They have the devices.

Lenovo and Dell make PCs, they make all the peripherals. So they're at the workstation and collaboration space. They have the software.

They have their AI factories and their TruScale. And then they also have everything that goes into the clouds, the racks Whether you're on-prem, whatever. And all of this is even uplifting things that aren't necessarily traditionally AI related.

Dell's not data center focused products are also in very high demand, more so than they were last year. And we've even seen, to Guy's point, technology hardware that used to be very much a commodity, like memory and storage, also just shoot through the roof because there's a very high demand for it as well. So if you're on the hardware side of things, business should be good for a while now.

Even timing chips, we work with a company that I've been looking at for a while called SiTime that makes timing chips that are better than cords for a lot of reasons that I won't get into now. It's just smaller, faster, better, more precise. And all of that timing for those data packets that have to move back and forth super fast between all these GPU clusters, that's really important because that gives you more GPU utilization, more efficiency.

So even down to that level of minutia where you have a highly specific part that's a very sort of relatively cheap part compared to an NVIDIA GPU, all of these hardware manufacturers, all these hardware vendors are absolutely just struggling to keep up with their supply chains because the demand is so high. And I don't see that stopping anytime soon. Regardless of how much opposition we see against new data centers, that's a separate story.

The demand for the hardware is going to keep going very strong into the next disruption, which will probably have something to do with physical robots. But again, topic for another day. Well, I think that to put this doom and gloom scenario in perspective, that what you and Steven are talking about is one element of it, which is the hardware sales continue hand over fist.

AMD is also doing quite well with its AI accelerators and actually has a broader portfolio than that, data processing units, and so forth. The neo clouds themselves, CoreWeave is one, but there's Nebulous, there's Vulture, there's others. Sales are robust, and we just have to ask ourselves if this is another Akamai situation where Akamai, if people remember, towards the end of the dot com boom, built out tons and tons of fiber to carry all of this internet traffic that then only started to reach capacity within the past five years or something like that.

And I actually think that, again, to counter the doom and gloom thinking, there's I think a lot of tolerance in the market for mistakes and problems in AI. I think it's a fun story for me personally to tell that the market's going to turn, the other shoe's going to drop, and all this other shoe stuff. And I think that's true.

I'm not saying because I don't think it's true, but I think that the possibility of dark data centers or neo clouds or the big hyperscalers not utilizing a good deal of all these new build-outs, I think that likelihood is relatively low because I do think that there is productive AI, very productive AI is going on right now that will continue to rise. I think there's a ton of waste, but I don't think all that waste will get sloughed off and we're going to enter a mini dark age of AI. AI, the benefits are so evident, even if we misplace them sometimes.

And the ability for really create-- I think the biggest AI company probably hasn't even been founded yet. The possibilities are going in so many dimensions to do things better using AI and more intelligently and in a more distributed way, and in a more this way and that way, all these things that we're complaining about. I think that kind of development can happen very quickly.

And I think, Steven, you made the point that the downturns and the negatives are also learning points where we learn how to do things better. And actually, probably Olivier, you made that point in a different book six years ago. So I think that, yeah, things are not going as planned, but we could probably say that that's a good thing.

Yeah. It's funny that these things, the more you look, as I said, no plan, what is it they say? No plan can survive in the face of the enemy or...

I- Yeah ... murdered that, but- Yeah ... the point is we had these plans.

It doesn't mean that everything's going to collapse. In fact, to the point that we're seeing, I think, it means that people are pivoting, people are changing what they're doing, and I think that's smart. " I also want to mention, Guy just mentioned the hyperscalers and the neo clouds and the hardware providers.

" And we're actually going to be talking to some of those neo cloud providers and the providers of hardware there to learn really what's behind the door of these data centers. com or "Utilizing Tech" in your favorite podcast application for that one. Before we run, tell us a little bit more, Olivier, and then Guy, how can people continue this conversation with you, and where can they find your work?

Well, obviously, at the Futurum Group, and our websites, which everyone already knows. I'm occasionally on X in short bursts. I'll disappear for a couple of weeks and just all of a sudden show up in rapid fire.

Occasionally on LinkedIn as well. And you can also catch me on the Schwab network and other places where I occasionally talk about earnings and technology. Well, you can find me at some tech field days from time to time.

It's always fun. So that's one thing I get to do. com is a good place for my more thoughtful, let's say, published work.

And I'm in LinkedIn quite a lot. social. I'm over there too, and Guy and I actually are Bluesky buddies.

So do check out sfosket on Bluesky. Heck, I'm on Mastodon, I'm on LinkedIn quite a lot, so it'd be great to hear from folks. Thank you so much for listening to this episode of the "Utilizing AI" podcast.

If you enjoyed this discussion, please do subscribe on YouTube or in your favorite podcast application. And please give us a rating, a review, and maybe a comment. Give us a shout-out.

The podcast was brought to you by the analysts and experts from the Futurum Group, Guy, Olivier, and the rest, where insights meet AI. ai, the "Utilizing AI" YouTube channel, or the Techstrong TV app. Thanks for listening, and we'll catch you next week.