Enterprise AI Adoption Moves Beyond Bubble Thinking

AI Leadership Insight Series. I'm your host, Mike Bizberg, and today we're with Roman Stanek, who's the CEO of Good Data AI. And we're having a little chat about, well, this bubble that we keep talking about in AI, and some people argue it's not a bubble, and others would, but Roman has some interesting opinions about the whole matter.

Roman, welcome to the show. Thank you. It's good to meet you.

It's good to see you again. Good to see you, my friend. So we are talking about this bubble almost every day now, and others will argue that the investment is justified, and others say, "Well, it's too much, and maybe the whole thing is a Ponzi scheme," and who knows?

But ultimately, you might be saying, "Well, maybe if we pop this bubble, it would be a good thing, because maybe there's more money available for enterprise organizations in AI," or what's your take on what's going on here? So, I lived through multiple bubbles. In the long term, bubbles are always good.

Bubbles are always good. We have fiber today because of the first bubble, and we have overbuilt things during the bubbles that we always find useful. So I see this kind of overbuilding of hardware and data centers and so on as positive.

Is it going to be used by people doing ChatGPT, the recipes, and so on? I don't think so. But for enterprises, it's going to be extremely important.

It's going to take much longer than people expect. One of the things we see, how slow enterprises are at deploying AI. And AI is a different beast.

AI is actually kind of unnatural for most enterprise IT organizations. So a topic you and I know really well is enterprise IT, and I would say that AI is something that they've never encountered, and they don't know how we would actually approach it. If there was a bubble that burst, how would that impact enterprise IT organizations in a positive way?

Cheaper hardware, inference is cheaper, everything. Again, it's the same analogy. We had no fiber before the first bubble, and then the bubble burst and then everyone had million miles of free fiber.

So I think that's kind of the analogy. That's what I'm pointing to. All right.

I think that that's a fair assessment. To your point about enterprises and why they're struggling a little bit with agentic AI especially, is a lot of the workflows that they do, their businesses are deterministic, and they need to be done pretty much the same way every time. And I've got a bunch of AI agents that are probabilistic, and they never do the same thing the same way twice, and they're pretty aggressive about accomplishing their mission at any means possible.

And that's not quite what we're set up for, because we don't really have the governance and the controls in place to limit that activity. So how long do you think it's going to take enterprises to wrap their heads around agentic AI? I actually think the problem is actually deeper.

What you just described, that's kind of the top of the problem. That's top of the pyramid. I think the bigger problem is enterprise IT was never known for agility, and AI enables extreme agility.

You have teams of 20 people that are as productive as teams of 2000 a couple of years ago. So do we need to scale enterprise IT down in terms of number of people and make them more productive and more agile, and how do we actually combine it with what you said, the governance and so on? But I would say deterministic and non-deterministic agents are not the biggest problem today.

The bigger problem is who does what and how do you actually start AI? I'm working with a large enterprise clients where the innovation committee means once a quarter, and the models change once a day. So that's the impedance mismatch I'm talking about.

Mm-hmm. " Yeah. But I would actually start with AI native.

I'm looking even at Good Data. The amount of data we are producing and the way we could actually improve our own operations, and when I look at companies that are 100 times, 1,000 times larger, the opportunity is there. But as you said, it's not about I have my existing process, and I will put some agent, and that process will get only messier.

It's like, how do we actually rethink the operations? How do we think about AI native enterprise? And again, how do you actually do it in a situations where most people know how to buy and implement SAP, they've never done complete transformation with AI.

How much digital transformation are we going to be looking at here? I know we used to bandy this phrase about 20 years ago. Yeah.

But it feels like we're almost on the cusp of coming full circle here, but it's not clear to me that people-- Are we just having a lack of imagination about what to do with this stuff at this point, and maybe we need to just sit down and that innovation committee that you referred to maybe needs to meet every day? Or that needs to be different. There needs to be no committee.

There should be no innovation committees. But you're absolutely correct. I actually think that, I think about it a lot, the whole digital transformation.

How do we blow it up and how do we actually let small teams innovate, and how do we put some sort of basic guardrails so that things don't blow up? But it cannot be done top-down. It cannot be done through some, again, innovation committee.

Things are changing way too quickly. There are new models, new harnesses, new approaches every single day, and it's just impossible. There was actually a article in "The Economist" a couple of months ago, and they claim that AI is too weird for enterprise IT.

And I actually think that's a good perspective. I actually think that's the best thing I read about AI in a long time, is that it's just people who, and organizations, who know how to deploy, as you said, not just deterministic agents, but everything is kind of deterministic, layers and applications known for decades, and now it's all different. Yeah.

We've seen this drama play out before, but what I see is that, well, AI kind of requires a lot of data. The enterprises have a lot of data, but they're a little slow to pull all that together. Yep.

But then on the other side of it, theoretically, there's going to be all these startups that will challenge them, and maybe a lot of the old guard will fall away. But the startups don't have any data. So, how do I get on either side of this equation and drive innovation?

Yeah. No, again, that's the dilemma. And that's what we do as a company.

We help large companies to adopt and organize data with AI, and as you said, most companies have decades and decades of data that no one ever processed. And I actually believe, one thing I believe that 2010s were lost decade in data. That while we had some progress around Hadoop and visualizations and so on, the rate of change was not fast enough to manage all the growth of data.

And now it's changing. Now we have the opposite. Now we have the abundance of intelligence.

We just don't know how to actually deploy, what's the best architecture, how do we put all the semantic models and everything in place, and who's going to do it, and how do we make sure that it actually, again, is predictable and secure and so on. But it's a fundamentally different environment than 10 years ago. Many of those enterprise organizations that we're talking about, well, probably none of them would ever get a Good Housekeeping Seal of Approval for the way they've been managing data for the last two decades.

So do we need to just revisit the fundamentals of data management, and everybody's going to have to go deal with some issues that they've been ignoring for the last two decades? Yeah, absolutely. I'm spending, if you spend time with me, I'm spending most of my days in like talking about iceberg tables and transformations and so on, just to get data ready for iceberg, for agents.

It's not some sort of agentic kind of-- I spend almost no time talking about kind of agentic processing at this time yet, because the fundamental data sets or data structures and semantics and metadata are not in place yet. So will this be the thing that kind of results in this bubble popping as we were talking about earlier? Because at some point, somebody's going to wake up and say, "Well, we're not realizing the ROI on all this AI investment fast enough," and then the investors will get a little antsy, and then that's where that pop in the bubble will be, and then we'll get on with the rest of our lives in the business of AI.

I don't know. I think that most of the AI today, the most successful AI today is still kind of consumer AI. It is kind of every app you open has some AI inference, every kind of application.

Most people I know live their lives through ChatGPT and kind of organize their lives around that. So I do believe that enterprise AI is actually, as much as it used to be a big chunk of the IT spend, I think that the consumer stuff is actually much more important at this time. And you have so many exciting opportunities.

And not only that, you have the whole defense spend, people spending money on defense AI and so on. So I think that enterprise AI, we have to sort out, but I don't think it's going to be the thing that will actually burst the bubble. All right.

So what may lead to the bubble bursting then, do you think? Will it just be some sort of, we all collectively have some sort of emotional crisis on Wall Street and investors just go somewhere else, and it's a herd mentality, or is there something that goes beyond that that's actually real? I don't know.

No one knows how bubbles burst. But one thing that this whole thing is kind of predicated on constant innovation. How that every model will be twice as fast and twice as intelligent.

And every time-- So the whole investment is actually based not on today's performance. It's actually kind of modeled on continued improvements in the capacities or capabilities of the models that actually double every 200 days. So imagine that for a minute, imagine that we stop, that the next models will not be better.

Imagine that somehow we hit a wall in model performance and people say, "Well, that's it. We don't need more models. This is actually good enough," and so on.

So I think that's the biggest risk for the bubble is that the main kind of frontier labs will not be able to maintain kind of the innovation velocity they promised to investors that there will be always demand for better models, better hardware, and so on, because they will be more capable. And then there are folks like Mark Zuckerberg, who are suggesting that maybe we're looking at the wrong end of the AI horse, and that a lot of this is going to be driven by smaller models that seem to run on our phones and local devices, and that's where the action's going to be, and we don't need these massive data centers. And that's actually, surprisingly, that may actually be better solution for the enterprise.

One thing that we are investing as Good Data is kind of the small model inference. We do our own kind of hosting for our clients and the price performance is better than the large models because the use case is different. Enterprise agents have certain task capabilities that you can actually map to smaller models.

So it actually makes more sense. We are not solving Riemann's hypothesis. We are actually doing some basic e-commerce optimization.

So maybe a smaller model will be enough. But again, I don't think that that's where kind of the main battles are. It's not in enterprise.

The consumer Apple models on this, like being able to do like a local inferences and so on, that is probably much more kind of a potential for bubble bursting more than enterprise AI. When I talk to some enterprises, though, they're having the challenge gaining access to GPUs. So hopefully there's other things we can run these models on.

But it almost feels like, to your earlier point, a lot of this circular financing that's going on is creating demand for GPUs. People are hoarding the GPUs. If I can't get access to the GPUs, then I can't develop the AI agents, and then suddenly we have this kind of self-fulfilling prophecy.

So, I mean, where does this kind of logjam break? I think it's getting better already. Like as I said, we are doing local inferences for our clients, and we have our supplier.

And about a month ago, six weeks ago, I went to their kind of availability dashboard. There was zero GPU available. I was scared.

" It was scary. Six weeks later, we can buy all the way, everything from the smallest CGPUs to large GPUs and so on. And that's one thing.

The second thing is, don't forget that today we all depend on NVIDIA, and you have all these big players like Google and Amazon trying to kind of build their own hardware and compete at NVIDIA on their own game, like with the GPU chips. And I think that the main thing that keeps actually NVIDIA kind of in the lead is no longer the hardware, it's actually the software layer, the CUDA and so on that everyone, it's the only layer we know how to actually use models. Yeah.

So how should enterprise IT leaders be looking at all of this? Because on the one hand, if the bubble does burst and they get cheaper access to infrastructure, that's good for their company. It might not be so good for their 401 s, though.

Yeah. I know. And again, it's kind of almost like unsolvable problem, because again, this is so new, this kind of innovation capabilities and new tooling and productivity gains and so on.

So I think this is-- I wouldn't worry about GPUs. I would worry about my own organization and do I need 2,000 developers, and do I need to have innovation committee, and do I need to be focused on SAP implementations for the next 10 years when maybe I can wipe out my own SAP and so on. So I think there are some big decisions ahead of enterprise IT leaders that no one knows how to solve.

But I would say that for the enterprise, the solution is in applications. Like AI application, the application layer is the most critical thing, as you said. How do we actually build things that innovate faster?

Who's going to build it? How do we make it secure and so on? The GPUs will be there, but ideas for application layer and being able to actually develop it and change the business, that's still innovation that very few people know how to do.

And to your point, I think the tenor of the conversations are starting to change a little bit where this time last year, it was all FOMO. It was all, "I'm going to miss out," and, "Oh, my God, we have to do something," and there was a crisis mode. But there's two ends of this.

I think there's still a focus on what can we do that's unique and different with AI, but the other side of this conversation that people don't seem to be having is that AI is becoming the new table stakes to remain competitive, and I may need to invest in it just to stay relevant, and, well, I might not even be able to charge extra for it because, well, it's just the cost of doing business. Yeah. Two comments on that.

The first one, this is why the small models are so important, because the cost of inference of OpenAI and Anthropic may be too high to have a bad business model. If you pay so much for tokens, how do you actually make money on that if it's a table stake? So that's one of the things that drives local inferences.

And the second thing is, again, it's the application layer. How do we actually-- Like two years ago, most of our large enterprise customers believed that all they need to do is to set up a chatbot. You just kind of get a chatbot on your data, that's going to be the AI.

A, no one uses it, no one needs it, and it's became so difficult to actually build AI if you kind of, once you shift from chatbots to real kind of business applications, agents, and so on. So yeah, the kind of the early days of, "Oh, it's easy, it's just a chatbot on top of my existing infrastructure," those days are gone. All right, folks.

Well, you heard it here. That sand beneath your feet, AKA that AI foundation that you're standing on, is shifting. There's no two ways about it.

We're just not quite sure what direction. Hey, Roman, thanks for being on the show. Thank you.

It's good to see you. Thanks. AI Leadership Insight Series.

You can find this episode and others on our website. We invite you to check all those out. Until then, we'll see you next time.