AI ROI Demands Better Business Case Discipline

AI Leadership Insight Series. I'm your host, Mike Vizard. Today we're with Nithin Mummaneni, CEO of Infinity Loop, and we're having a little chat about, well, ROI, return on investment in AI.

It's hard to figure out. Nithin, welcome to the show. Thanks for having me, Mike.

Everybody's talking about it, but it's one of those things where it feels like the blind man trying to describe the elephant. Nobody knows what part of this thing they got. What is, in your mind, the fundamental challenge we seem to be all having when it comes to determining what the ROI is for AI?

I think it starts with the experimentation phase of not just planning the right use cases to test AI with, because a lot of the times, we're being pressured from boards and CEOs to actually go and implement AI in organizations, but people don't know where to start, or oftentimes they pick the wrong solution. So it really comes down to use case prioritization and just making sure that how you're actually implementing AI is right for your organization and that there's an actual measurable outcome. How do I determine that?

Because one of the concerns, or at least when I talk to folks that they're discussing, is that, let's say I have a great idea, and I come up with this thing for AI. Well, AI is so fast that my competition's going to come up with the same idea and implement it a few days later. So how do I justify all my time and effort when really it's just like table stakes?

Yeah. And so it comes back to how organizations have adopted to the internet as an example, 20 something years ago or 30 years ago. It was this new technological revolution similar to how AI is today, but it's fundamentally going to change how they work.

Right? " I think that framework is extremely important that not a lot of people actually follow and then stick to it through implementation and beyond. So that's one of the things that we recommend to our customers, not just prioritizing us, but also other solutions in the space, too.

One of the things you also hear folks struggling with is the overall cost. And as these more advanced models become available, they also get a little more costly to use and so on and so forth. Are we going to have to make some decisions here about what models to use for what use cases because the costs are not as low as they once were, and certainly the providers and the models can't subsidize them forever.

I think it comes down to, and what we're seeing, the build versus buy decision, internally at our organization. So whether they have Anthropic or an AI implemented at their organization and/or actual providers that utilize those platforms, to build on top of them to provide that application layer that's going to help them. So whether they want to do it in-house or actually go external, I think is an important decision, especially with the rising token costs that we're seeing.

It's going to be instrumental in them determining do they want to dedicate resources, because it's not just a token cost, it's also resourcing costs, it's also dataset costs that you'll have to tag on and maybe even implementation costs from consulting firms to actually go and do this. Or you could just go buy that good or service externally and not have to deal with any of that internally. So I think that type of framework is what we're seeing consistently, especially over the last six months with even our customers, is that we're going to have to justify that build versus buy decision with that increase in token cost, which we've been able to successfully do mainly because of the value add that we provide.

It comes back to that framework I was talking about earlier. Are there some decisions where I might go into some sort of self-hosting environment and build and deploy it myself, even though there's costs involved in that, and they are considerable, but maybe there's specific use cases or my intellectual property is so core that I don't want to have that with a third party. What drives that decision?

It comes down to the strategy of that organization. A lot of the times, depending on what industry you're in, et cetera, it has to come from the top, right? And it's about the executive teams and their folks that they work with to decide upon is it that critical for us to adopt AI in this way, or can that be outsourced to an application layer, as an example?

And how does that impact the competitive advantage within a particular industry? So it's a strategy question at the end of the day that businesses are now making, that will dictate how competitive they will be in the next five to 10 years. So it's very important currently.

Some folks are concerned that they are setting themselves up for a new type of competition. And while the providers of the AI models may not be training their models on the data that the customer shares with them, they are certainly gaining exposure to the metadata, to the code, and the workflows. " It started out with books, and then the next thing you know, they were everywhere.

And will the same thing play out with the model providers? I think at the end of the day, it comes down to the data, right? The big model providers themselves, I think are great, but they're going to have a hard time kind of doing everything.

And mainly because of specific datasets that cannot be exposed to those models, especially with the architecture that, not just companies like ours, but internal organizations are building themselves. So they have agreements with these companies. They have specific architecture set up so that the models don't get trained on that proprietary information.

So I think as long as that's in place, it's going to be very hard for those models to replicate and/or grow into replacing and becoming the one system for everything. I think we're well away from that. And that's part of the reason why you're seeing the self-hosting phenomenon taking place, especially over the last three months.

And it's no surprise as well that token costs are increasing as well. What do you see folks doing out there that just makes you shake your head a little bit and say, "You know what, folks? We might want to be a little bit smarter about this than necessarily going down that particular rabbit hole"?

" And it's the managers that willy-nilly spend a bunch of money, whether it be internal resources pulling together to create their own systems or buying externally from providers like us, without a game plan. And what I mean by game plan is specific kind of measurable outcomes that you expect to achieve on those various use cases. So we're seeing that sometimes, a lot of the time, things are stuck in the pilot phase and don't go into full implementation because they were not able to prove that measurable outcome in the first place.

And that's the mistake. Again, it's just lighting cash on fire at that point. So people just need to plan a little bit better and tie it back to an outcome.

There's also concerns out there that we will not have enough data center capacity for one of 20 different reasons, but the end result is do we wind up in a scenario where maybe we have to start rationing our usage of infrastructure to prioritize things that add more value to the business versus, I don't know, throw in GPU cycles at creating a social media meme? I don't think it's that big of a concern. I've always said the US is a place where if you wanted to make something happen, it's a great place for innovation.

That comes from constraints. So, if at some point in the future that you run into these constraints, I think the only thing that will get impacted is the price, obviously, which will deter a lot of people. So that's what you'll likely see rather than the other way around.

What is your sense of on a global marketplace, what are you seeing outside of the US? Are we running up against new levels of competition? And I ask the question because one of the things about AI is we have a lot of labor, and there are other places where there are not a lot of labor, and AI levels the playing field.

So is the nature of competition going to be fundamentally different in the age of AI? I think the US is still the best place in the world for AI innovation. And a large part of that is because these frontier labs are based in the United States.

And when you look at the competition, the only other frontier labs are coming from China. And based on that itself, we've barely scratched the surface in terms of application layer implementation across the company. So you may see companies across the globe utilize these frontier labs, like an Anthropic, like an OpenAI, to create application layers that they can sell into the US.

But again, I still think in the US, it's still the bedrock of innovation here. And all of the latest innovation will come from that because the frontier labs are based here, and they're the most powerful models. We have also seen, though, that there's a lot more interest in open-weight models and whether they come from China or somewhere else.

Is that a viable strategy, or is that something that is fraught with its own issues? Has its own issues with that as well. If you go to the open source route, which you totally can, you're going to have to deal with even more heightened data privacy concerns on how you actually implement that.

And then if it is offline models, how you're going to enable that in your own environment, that's another issue as well. So I think you should definitely evaluate the different ways you can implement AI, but it depends on a company's risk tolerance at the end of the day. Of course, everybody and his brother's talking about AI agents, and some of them have gone rogue, as you well know.

Are we prepared, and do we really understand what it's going to take to govern AI agents and, more importantly, I think, secure whatever it is that they are trying to access because, well, AI agents are AI agents, and they're going to go find data one way or another, right? I think because we're at the cutting edge, we don't know. That's the unfortunate thing.

There's only certain things you can do from a predictability standpoint to get ahead of these things. But again, if AGI is truly on the rise, we don't know the inherent risks that it could cause, not just to privately traded companies, but also public ones as well. So I think the risk is unknown.

If you did know it, you'd be a very rich person. So all you can do is predict it at this point and try and get ahead of it and be proactive. But I don't think there's any true way to really gauge the risk of utilizing AI.

Some people are also suggesting that when it comes to ROI of AI, that maybe tokens aren't the right thing to measure or the right thing to use for tracking consumption. Is there another way to think about this? I think it comes down to how you utilize the system.

There's probably a level deeper that you can dig into, not just based on token usage, but also impact in certain ways, just to normalize what that token usage could be. So it's probably a derivative of token usage is probably the right metric to look at, depending on how you implement it. Do you think also the way we are currently structured around various vertical industry segments will hold up, or are we likely to see companies will leverage data and AI to jump into new marketplaces where previously they have not been before?

So I'll just surmise that somebody who's in one manufacturing space may decide that, well, with a little help from AI, they can be in the transportation business, if you follow me. Yeah, I think every company has its own strategy on their future five to 10 years. I think the ones that will get ahead are the ones that adopt AI in some fashion.

Again, AI is an enabler. It's not the end result. So I think it'll help optimize a lot of companies and speed up their roadmaps in the next five to 10 years so they can potentially acquire more businesses, the ones that don't accept AI or expand into new markets.

Again, it ties back to that company's business strategy and how willing they are to adopt AI to compete and eventually grow and outrun competition. Some of the more established companies are looking over their shoulder with a raised eyebrow because they have noticed that the providers of the AI models have very aggressive startup programs, and some of those startups are receiving discounts and benefits to go create new applications that will compete with some established player who may also be a customer of one of those AI model providers. But it seems like the relationship between those companies is a little bit fraught, shall we say.

So who's friend and who's foe? How do I figure? I think the biggest thing is, look, the rate of innovation is definitely going to increase even more with the adoption of AI.

Companies have a war chest. They have millions, sometimes billions of dollars to spend, and it's up to them how they want to allocate the capital. " So I think they've got to keep their war chest open in those types of situations.

But again, it comes down to that business strategy. So if they're bold and they see a threat, if I were them, I would just buy them out. You hear the term moat a lot lately.

So what is the definition of a defensible moat going to be in the age of AI? Because I think a lot of organizations that may have previously thought that they had a moat are going to discover that that moat is not as strong as it used to be. So where do I go run to, to create my next moat?

I think the biggest thing is the data. The AI is only as good as the data sources that you're able to feed it, but also it's that business knowledge that is able to translate that data into actionable insights that somebody can then take to actually run the business. So it's twofold in that way.

Data, by far, is the most important, but you also need to have people in charge that know what to do with that data at the end of the day. So you don't outsource that completely. So I think those are the two key levers customers or companies should look at.

Do they have the right people, and do they have the right data at the end of the day? Because that will ultimately create that moat if they didn't already have or help sustain it. " They're basically trying to increase a productivity number on an existing process versus maybe thinking up something entirely new and different.

I think it's a slippery slope. So this comes down to that people equation, and then tie it back to the data, because you got to be able to database all of the relevant things within your company and industry to make sense of it. If you don't have a good data set that you're working from, it's going to be very hard for you to implement AI in that way.

So that's the precursor. Now, also comes down to the people that I mentioned. It can't be one person making this decision.

There needs to be a collective within an organization that deeply understands how that data structure works within their company that they have built, and then later on, the business strategy side of things on how it can actually help them grow into the future. So you got to have some type of collective. It can't just be the CEO.

" So it needs to be a collective front in that regard. All right, folks, you heard it here. We definitely need to look harder before we leap because, well, ROI is all dependent upon what you want for that return and then matching it up to the actual investment.

Nathan, thanks for being on the show. Appreciate it. Thank you, Mike, for having me.

AI Leadership Insight Series. You can find this episode and others on our website. We invite you to check all those out.

Till then, we'll see you next time.