Backstory Tackles Enterprise AI Token Economics

ai Leadership Insight Series. I'm your host, Mike Vizard. Today, we're with Jason Ambrose, who's the CEO of Backstory, and we're having a little chat about, well, tokenocalypse.

It's coming, apparently, and the root causes of this has to go back to, well, everything from the way AI model consumption has been subsidized to the fact that we're now using these things at much higher levels than ever before. Jason, welcome to the show. Thanks, Mike.

Great to be here. So walk us through what's going on here and what should we be concerned about, because it does seem like people are starting to choke on the number of tokens they're consuming, and I don't think this was in the plan. A lot of folks I talk to, they're running out of tokens halfway through the month, and they don't know what to do next.

Yeah. I think there's a few factors to this, right? I think there was just so much enthusiasm for what we were going to do with AI that, at least what I could see in a few accounts is, people sign these commercial agreements and the impact to follow.

Right? So it was, let me go sign this big agreement, let me get access to all of these tokens or this LLM capacity, and it'll work itself out. And I think like a lot of change management, especially in enterprise, to your point about what the plan is, I'm not sure that there was a plan in some cases, right?

" I think for some of this, and probably some of us who've been around a long time with technology, knew this day was coming, right? It's maybe coming a lot sooner than we thought because the prices have come through, but now you have to be very thoughtful about what you're trying to accomplish, how you're going to make sure that you got that, and what your appetite is to experiment to see if you can do this with AI, to go through the natural cycle of trying some things out that maybe don't work to find the right answer. And then once you've landed on that, being ready to then follow through and scale versus scale upfront.

It's almost like starting a company in that way. You hear a lot more people talking about tokenomics these days as well. And so can we optimize our way through this, or are we also going to have to make some tough choices about, well, the number of projects that we're actually willing to fund?

Yeah. If you think of tokenomics as, I'll just super simplify it, right? This is essentially setting up a new currency and the value of that currency.

So any time you're applying the currency, it's a store of value, and you need to make sure that you're exchanging it for something with equivalent value. Right? If I thought like an economist, which my dad was and how I grew up.

But to your point is, we should have the end in mind when we take on these projects, right? What makes it worth it? What is it that we're trying to achieve?

What do we want to see on the other side of this to make sure that it makes some sense? " I think the second piece is the vendors in this space have to be pretty consistent with the costs that they're charging, right? So whether it's an explicit raising of token prices or essentially hidden inflation in the sense of degrading service for the same amount that you spent for the tokens, that's going to make it hard for enterprises if they feel like the costs are escalating either in a degradation of service or true cost.

Enterprises do best when they can manage under a predictable spend. Right? Is the whole token model in terms of pricing broken?

And do we need to think about a different model here for how we provide this capability to folks? And is there just a smarter way to think about it for something that is kind of based on inputs and outputs? It's designed to get expensive in a hurry.

It is. I think like anything else, it's probably less about whether or not... So I will say, I think tokens as a unit of work performed makes sense, right?

Passing that through as a straight, unbridled cost, maybe that's what needs to change. So, for example, in our model, we have our equivalent, which we call credits. We do what we call a true forward model.

So we say, for whatever you use in that first year, we don't know when we go in together how much you're going to use the product or the value you're going to find, but once we do find that usage pattern, we're going to treat you forward in the next year. So that takes away the risk for customers to think about what they're doing in year, and they don't have to go back to the CFO for more budget, which is the hardest thing for enterprises to do. That's our answer.

I think there will be others, but it's more structural around how do you de-risk this for customers before, during, and after use of the product. Do you think that as this kind of currently evolves in the state, that we're going to see more organizations struggling with and coming up with something that feels like an ROI that justified the investment in the first place? Or are they still going to get maybe, I don't know, a carte blanche card from the CFO who said, "Well, AI was critically important, and we just spent money on it, and whatever happens happens"?

So I think that day is over. I already see it in accounts. And let's be clear, I'm not sure that it's the CFO.

In a lot of cases, I think the CEO is anxious for the transformation agenda and said, "Look, I'm going to short-circuit this process. " And they cut that check, $10, $50, $100 million, whatever it was. The CFO said, "Okay," and probably patiently drummed his or her fingers on the desk and waited for the ROI to come in, and that's not coming in.

So I think those days of blank checks are going to start tapering off, and somebody has got to be on the hook for there to be a return against big spend initiatives around this, or programmatic spend. And so I think that that's the piece that's going to change. And again, I don't know that you necessarily have to go straight to a hard ROI every time before you fire up ChatGPT or something else, but you've got to have that end in mind.

Do you think we will maybe start playing various AI models off each other to try to get a better price per token? Because I feel like, as I talk to people, you start to hear phrases like auto routing, and they're trying to- Yeah ... connect APIs to different models that may be better suited to perform in different tasks on a cost basis.

Is that going to happen? I think you've seen both, right? I've seen that in customers where they have a, let's call it an abstraction layer, where there's an agent that's sitting across, that that's what their users within the organization use, and they don't necessarily see which LLM is underneath that because they wanted to abstract that away, and they wanted to have the leverage to say whether it's token cost or model performance or whatever else.

I think for the state of play right now, one school of thought is, "I can't commit to anybody because three months later-- Six months ago, it was OpenAI was taking over the world. Today, it's Anthropic. " That's hard for an enterprise to commit a multi-year technology choice, especially when it comes to the front end for users.

Now, the other school of thought is, "Hey, we want that partnership. If they can help us lock in these costs, we don't want to have to be managing all of this. " This is sort of the Microsoft Copilot model.

But increasingly, I think you'll see that from OpenAI and Anthropic is what are the features that we unlock when you are able to commit to this, and we become the standard across the organization. So I think in terms of features and value, that's to me where if I were them, that's how I would be focusing on locking in the user community and abstracting away the economic risk. But if they continue to push forward that economic risk, then I think customers have to go into that first camp.

It's just not viable for an enterprise to try to manage this and have it be truly open-ended. Are we too obsessed with the latest and greatest models and the latest GPUs, and we need to get smarter about how we're using these things? Because it seems to me, in a lot of instances, a version of a model that is two years old running on a GPU may be just fine and dandy for the task at hand.

I think you have a good point there to say what does it all matter if it doesn't truly change the behavior and the processes? 8 or whatever the numbers or whatever the models that's underneath it, if we're not changing the way that we operate, that has material impact for the organization, it's not going to matter whether the model gets any better. So I think for us, and what we see is that change management piece, is this actually getting to that last mile of individuals in the organization acting differently, not just in their own work, but in a process.

That has to change before you get to a point where you're optimizing for the latest and greatest models. Now, the AI shops, the innovation teams, maybe some use cases around development, yeah, maybe they benefit from being hyper-focused on the latest and greatest. But I think you're right in the sense of most enterprises have a longer horizon to think about this.

Are we part of the problem, and I ask the question because you brought up change management, and a lot of folks to drive change created incentives based on the number of tokens you were consuming, AKA token. And as that occurred, people responded accordingly and started generating all kinds of use cases for AI, not all of which maybe are the best- Yeah ... use of AI.

Yeah, I think in the most generous interpretation of token maxing is, hey, look, the point was to push you to think about all of the ways to use AI and really reset all of your preconceptions about what you should do versus AI. And I think that that's a noble intellectual enterprise. But I think to your point, that quickly became a scorekeeping mechanism.

And I don't think anybody would say it makes sense for one person to spend $10 million of AI tokens at this stage. Maybe any stage. But again, I think it comes back to this point of what's the means to the end?

How does this connect to generating real value? If you're just burning tokens to burn tokens, that's just hard to justify for that CFO that didn't want to write the check in the first place. So what's your best advice to folks to avoid the apocalypse that we started with at the beginning of this?

Because it's still early days, and it seems to me that we could get in front of this, but will we? I think it's everything that we've been talking about is, I think either internally or working with partners who can help you have a clear sense of, what's the A to B and what does the journey look like to do that? I think there are complementary technologies like ours that help de-risk the idea of just using brute force inference for every piece of work that you want to do.

I think that there's going to be more players like us that solve a specific problem that's useful in this world as you think about standardizing on the generalist LLM layers. And that combination needs to be thoughtfully applied to what are your business objectives, what are you really trying to change, and how do you make sure you have a closed loop process to experiment, explore, and then exploit the results of that. I also think maybe we need a lot of what I would refer to as Hamburger Helper technologies.

Yeah. And all that stuff that comes to mind is whether it's a graph, an index, a database, some sort of storage system. But it seems like we are just slamming too much into the context window that's driving up the processing cost and the reasoning cost.

So maybe we need to kind of just look at the entire IT environment rather than just throw in a couple of tools and hoping for the best. Yeah, Mike, it's like you sit in my sessions with our engineering team. Look, you described what our business is, right?

We look at the unstructured information of activity, and we understand that that needs a lot of pre-processing to be presented as context, either through MCP or into the context windows of the LLMs, and we saw firsthand what happened when you didn't do that, right? Is you need to really flatten the curves on that of what has to show up in the context window. And what you described is who we are, right?

We're a player that does that in the sales organization, and I think of course, I think that there's a future in that. But I also think out of respect for the folks who've been pushing really hard on this, you're coming with a very nuanced, informed opinion that, hey, if I'm a head of RevOps or something else, what's the learning journey for me to reach that point to understand that, and should I even go on that? The answer is no, right?

But you're doing your job as somebody who's following the market and seeing that this is something that's coming, right? To your point about that, we saw the rise of FinOps when cloud spending got out of control, so- Yeah ... wouldn't we apply many of the same principles now to AI agents and agent technology, and at the end of the day, they're fundamentals and they just seem to be missing.

There's so many places of this space that are hard to predict, and I feel like that's not where I'm going to place my bets personally. I don't know how it's going to shape up or what the answer is to solve it. What I do know is it needs to be solved, right?

I think everybody learned their lesson from the days of runaway AWS spend. They're not going to do that again in the AI world. How they manage it, and how the vendors help them manage it, that's going to have to be figured out, and so I'll bet on that versus which horse wins that race.

Are you at all worried that the fundamental economics of this whole thing are a little shaky? Because we wake up in the morning and we see people like Sam Altman petitioning the government for some form of financial assistance, and we all start to wonder, is this thing- Yeah ... profitable on his end?

And then if his end isn't profitable, then to our earlier point, does the whole house of cards come down on top of it? Well, I think there's two parts to that question. There's what has become almost a macroeconomics question, right?

That one I'm not equipped to answer. Who knows what's going to happen with all this. And like I said, my dad was a microeconomist, and he brought me up to not look kindly on the macroeconomists.

No offense to those that are listening. And then I think there's the enterprise question, right? In the enterprise question, I do think there is going to be impact, and it is going to change at the customer level.

In the short term, is there some inefficiency in spend as they learn? Yes. Is that going to change pretty quickly?

Also yes. And I think, at least for me, it's changing a lot faster than I expected. " So we'll see how that ladders up to what it means for the global economy and bailouts or whatever the questions are.

All right. So what's your crystal ball telling you for what's it all going to be like a year from now? A year from now, I think we are going to see a much more well-defined ecosystem of the things that we talked about.

Certain specialist players like us that are providing a very specific purpose. We're not trying to own the agentic layer. We're not trying to own the interface layer.

That's going to settle into the right tools that are all working together, and I think the landscape is going to shake out very differently in who survives this transition. I think the ones that are trying to hold or stand up wall gardens, if they don't recognize that that's not what customers want, their future's going to look a lot different. And I think you're going to see surprising enterprise customers making choices with different players that they use to anchor and run their businesses.

All right. Well, you heard it here, folks. Hey, a lot of good advice was given once upon a time to somebody named Buttercup, so maybe we should all buckle up.

Hey, Jason, thanks for being on the show. Thanks, Mike. Appreciate it.

All right. AI Leadership Insight Series. You can watch this episode and others on our website.

We invite you to check all those out. Until then, we'll see you next time.