BasedAI Challenges Enterprise AI Theater

ai Leadership Insight Series. I'm your host, Mike Vizard. Today, we're with Tina Baker Taylor, who's the CEO of Based AI, and we're having a little chat about, well, theater.

AI theater, to be specific. Tina, welcome to the show. Hi, Mike.

Thanks so much for having me. It's a pleasure. I think it's no secret that while we're all heavily invested in AI, a lot of us have projects that don't seem to be going anywhere, or they seem to be stuck.

And some of that is just pure theater because somebody at the board level said, "We must do AI," so a lot of people launch projects. And others just seem to be in some sort of experimental phase because, well, they're missing access to something that actually makes this thing work at scale. But what is your assessment of what's going on here, and where are we on this journey?

Well, I think you're right, Mike. Some of the trouble with adopting AI, I think for businesses, they hear all of this hype. They're worried about being left behind.

They want to remain competitive across their competitive set. They don't really know what agents are or what some of this technology means. They don't want to fire their staff, right?

So there's a lot of concerns that go along with this hype about wanting to keep up, and wanting to be tech forward, and protecting their business, but not necessarily knowing what some of this technology is, how to deploy it ethically without firing a whole bunch of people. That's a big concern. And then physically actually doing it, right?

And we're seeing a lot of businesses now that have tried to be first movers and adopting a lot of the frontier model tools, getting extraordinary bills as the subsidies start to subside, as some of these companies are thinking about going public and have shareholders and board advisors telling them that they can't give all of this stuff away for free. And we've seen lots of that in the news where the people that they fired, they actually need those people back, right? Robots can't right now run everything for us, and nor should they.

A human should always be involved. And equally, having all of their staff use AI for everything they do actually is probably not a great idea from a cost perspective, and a lot of these companies are rolling back a lot of the stuff that they have deployed. So I think they're realizing that there's real operational challenges, and these can be really complex when integrating these systems into their work stack.

At Based AI, we're trying to help businesses solve where does AI make sense, how do you augment your team. But really, we have this theory that if you just start plugging this stuff in without really thinking about the architecture of how your business runs and how your processes work today, and ensuring that this is additive to that, that you're not creating more processes, then that's how it's going to be effective. And I think that businesses haven't taken that holistic view, and are starting to see some of the problems and barriers.

It also seems that a lot of folks are experiencing, shall we say, call it token shock. Sure. And what that comes down to is that the cost of consuming the tokens is a lot higher than they anticipated, and maybe we're not using the right models for the right job, and that's part of that issue.

But do people need to kind of think through the economics of AI in a way that they hadn't imagined before? It was just kind of like use it no matter what, and now it's kind of like, hold on a little minute. We got bills here that look like they're going to spiral out of control.

Absolutely. And I think that is not necessarily the business's fault. The way that we have been taught to frame the question around that token expenditure is maybe the cost per token, right?

And that doesn't really mean anything to businesses. What businesses want to understand is, how much is the output going to cost me? So we shouldn't be buying tokens, right?

We're buying a go-to-market strategy. We're buying a process that reduces the complexity of our onboarding of new staff. We're buying a website, right?

We're trying to get a finished output from these systems. And so understanding what the outputs cost is a better measure of what the ROI is going to be for that investment that you're making. And looking at it on a per token rate, which by the way, I'll say again, is being heavily subsidized today by the main frontier models.

What it costs them to produce those tokens, give those tokens to you is not what they're charging you. And so either your business gets to a size big enough where you have to move to an enterprise plan, and now you're paying per token, you're not paying per seat anymore, and you're seeing this Cambrian explosion of fees, and your CFO is wondering, "What is going on? " Or you're having a situation, most importantly, in my opinion, as you said, we are renting Ferraris to go to the grocery store.

You do not need a frontier model for, I would argue, 85%, 90% of the tasks that most businesses are using AI for today. If you're a nuclear physicist or you're doing something that is heavy in code or heavy in math, you're running huge amounts of data, there are absolutely use cases for these frontier models. But you don't need a Ferrari to go to the grocery store.

And there are plenty of smaller models or open source models that today are equally competitive. So yesterday, two days ago, an open source model came called GL... Sorry.

GLM2 came out, so it's the second version of GLM, and its ELO score, which is basically a measure of how human and useful a model is, was higher than the Anthropic model, Fable 5, which Anthropic just had to pull, and everybody loved it. Now, it's not on the market, so we can't actively compare them side by side now that the score is out. But that gives you an example of the gap between the capability of open source models and closed source models, has basically closed.

And the cost to run those open source models is a fraction. It is 25 times cheaper. So yes, we need to be pairing the right model with the right use case, and I think we need to be looking a little further afield, from the latest and greatest that comes out from the big frontier labs that we know about, and understand that they're trading all of the-- There's a lot going in to building out what ends up being a price per token to you.

And you might not need that level of model for everything that you're doing. Do we need to maybe narrow the number of projects that we're actually experimenting with and start picking some winners here that are actually going to be deployed at scale? Well, that's an interesting question.

I think within any hype cycle, there's always that period of consolidation. I don't think we're at the consolidation phase yet, but I do think that there's a lot going on in this space from regulatory concerns. What are the risks?

The regulatory risk is that some government somewhere is going to decide that this model is too powerful, and you've got to turn it off. The geopolitical risk, we just saw that happen with Anthropic's Fable 5, and previously with Mythos before that. Is this model too powerful for your average consumer to use?

They said yes. It wasn't rolled out. So I think within that, there is a need to have an understanding about what you're trying to achieve.

And from my perspective, I'm very focused on building with open source models for a number of reasons. I think for businesses today, they want to be careful about a couple of things. One thing to consider, unless you are a huge enterprise with an agreement with these big labs, your data is being used to train models.

That's why they're giving it to you at a cheaper price. The old adage, if you don't pay a lot for this thing, and you're giving it all your data, you're probably the product. So they're mining this data.

They need this data. And that's not necessarily bad per se, but if you're in a highly competitive industry, you've got to appreciate that whatever you're putting in there, in some form may be accessible, whether it's your actual IP or just the information around it by your competitors. There could be regulatory or legal risks by sharing customer data, for example.

So if you're using open source models, those models are closed. They stop on a particular date, and using them is not going to continue to train them. So that's a great privacy and security feature of using open source models.

As I said, they're cheaper. And you basically, once you download the model or you're using it from a service, like based AI services, you're potentially in your own segregated instance. So it's almost like having your own access to these models, and you're not in this soup or sea of data sets with everybody else.

And what that allows you to do is really have more control. And we've seen what happens when you're essentially, businesses are, or they're not able to own what they don't control. And so you could build out a load of processes and workflows, and then have the model turned off because of some geopolitical something that happens that has nothing to do with you.

So we feel strongly that if you're using these open source models and the stacks that go along with them, then that's where you could start to feel a little bit more confident about the sovereignty of your business' AI. And I think that's really important and something we don't talk about enough. To your point about that also, theoretically, I would imagine the open source model is going to be less expensive in terms of how it gets consumed because, well, I'm not having to pay fees back to whether it's OpenAI, Anthropic, or whoever.

That's it. So, a very quick comparison. 6 is somewhere around $8.

55. If you're using something like DeepSeek, it's in the cents. Mm-hmm.

So the cost is, like I said, 25% difference, 25 times the difference. That is palpable. And I think that if you're looking at something like you're running a particular project, and we've costed some things out, and I ran a task the other day in testing one of our functions within one of our products, and the cost for using a Claude model was 15 bucks, and the cost for using GLM 5 was 60 cents.

And that was basically to produce a report. So yeah, it's a material difference in cost. Are you at all concerned there'll be something of a backlash because so many folks were sold on the promise of AI, and if it's not realized, and they may not particularly appreciate some of the nuances that we're discussing, and they'll just try to throw the baby out with the bathwater, as it were, and just say that the whole tech sector, once again, is overpromising and under-delivering?

Yeah, I think you're right. So, the current cycle has been really focused on experimentation, right? And these businesses have struggled with the operational integration, so we've talked about that.

And I think that the key has been that most businesses are adopting AI with an interface-led approach, right? You're buying Claude Code, you're buying Cowork, and not necessarily looking at it from the infrastructure perspective. " And what that's led to is that businesses are deploying chatbots essentially really quickly, but then face challenges when they're trying to scale that automation into other functions across their businesses.

And I think that the advent of the very capable agentic AI, these agents that we hear about, these coworkers, there has been a lot of challenges around the workflow coordination between the human and the agent, and the agents and the agents. And where do these agents persist across different systems, I think is a big challenge. How reliable are the tasks that they execute?

And how can they maintain vigorous oversight? How do they partition information? How do they make sure that that agent has the right context to be helpful?

Because without context, the agent can't understand what you need it to do for you. And there is this level of permissioning, like how much information do you want to give these agents? And I think that's really been a struggle for businesses, and there hasn't been a lot of options other than the big labs that you just get what you get off the shelf.

Right? And so again, coming from an open source perspective, there is a huge suite of tools, and that reduces or eliminates a lot of these problems, but you need to know how to put them together. And so that's what we're focusing on, putting those tools together so that businesses can pick and choose what tools they need for the outputs that they actually want.

And that is a little bit more bespoke than just plugging in an interface that is available off the shelf. So I think you're right. Businesses have really struggled to realize the ROI that they expected.

But equally how we're using AI, I don't think we've thought about all of the... There was an article I read the other day basically about electricity, and how when electricity was-- You read it, okay. So when electricity came onto the market, people were just plugging in electricity, and there was an interesting study about a manufacturing plant that plugged in the electricity, and they thought they were going to have exponential output, and they didn't.

And they realized that they didn't motorize each individual section of basically this, what is it called when you-- Like assembly line, right? And so each part of the assembly line there needed to be an element of that motorized to speed up the whole thing. So very similar to this situation, that analogy kind of illustrates that you could just plug in AI, but that doesn't necessarily mean that you have re-looked at the factory to figure out exactly where those motors need to go to get to the end result that you're after.

To your point, and you kind of touched on this, but it also seems to me that we need to think through the art of context engineering a little bit more because at the very least, we're asking the AI agent to put too much stuff into the context window. It consumes too much memory when a lot of that stuff, it can just be made readily accessible either through, I don't know, pick a knowledge graph, index, database, or some sort of persistent storage. So do we need to think through the whole architecture of how we use these things?

Absolutely. And I think that's where these agentic frameworks have really come into their own. So the way that we think about it with Based AI, for every company that is working within Higher Brace, there's a company brain, and that brain is completely segregated by each company.

The company chooses what information they want the agents to be able to access, and different agents can be permissioned for different things, just like you would in your workforce. Right? And so somebody from HR may have access to this information, but somebody from engineering doesn't.

Right? And some of that information is shared across the entire company. And so what that does is it allows those agents or those AI workers that are brought in to support your team to have the same information that your human workforce has so that they can work together more symbiotically and in a more informed way.

What you're talking about with these context windows is something that's very endemic in chat. Right? So going back to what I said, companies are just plugging in these chatbots.

And they're quite limited, a chatbot. So you will have a context window that's only so big. And it does fill up over time as you're using it and using it.

And then Previously, we had these issues where the memory wasn't capable of being persistent, and so then you were having to re-explain yourself, and go through this information again. And that became very frustrating, and again, eating up the context window. So these agents almost work in this kind of symbiotic loop where they can have larger memory files that you create, and you give them access to different elements in your company.

It could be your Notion, it could be your Slack, it could be your emails. Wherever you want them to be able to continually extract information from to update their memory files, that significantly reduces the active context window that you're using. So it is about picking the right tools for the right things.

A chatbot or a chat application is far more limited than a more complex agent workflow that can use information and access it in a very different way. Also, are we in danger of maybe getting locked into a particular LLM, and we need to think through how we construct our harnesses to make sure that we can swap out an LLM relatively easily? And since we're on that subject, how dynamic is that going to be?

Am I going to quite literally maybe have a situation where LLMs are bidding to fulfill a request because I can send them to anything? Yeah, absolutely. We are seeing that now in the form of autonomous agents.

So you can have agents that you instruct to go and do a thing. And there are a lot of cryptographic primitives that allow machines to talk to each other and transact with each other. One of the things that has been really interesting to see is the advent of some protocol standards for essentially cryptocurrency payments.

So something like a stablecoin like USDC, which is basically a digital dollar. So you think about, it's not like other cryptocurrencies where the price can go up and down, and it's not like Bitcoin, for example. It's just always a dollar.

And so these agents, using a cryptographic protocol called X402, can pay for things using this USDC, this digital dollar. So if you connect that to an agent, and you tell an agent, "Hey, go and create this website for me. " An agent wouldn't have been able to do that before because it has to go and purchase the domain name, and now it has the tools to be able to do that.

But you've not told it how you want to do any of this stuff, right? So the other thing that these agents could do is go out and find their own imprints. So if you have an API that is machine discoverable, that is agent ready, these agents go out, and they find the path of least resistance, and probably the cheapest mechanism to fulfill the request that you've asked for.

So that's already happening today. And a machine does not have a brand bias. They're not going to necessarily go and search for an Opus model because they heard that's the best, or somebody on X said they should be using Kimi.

It is going to go and find the model that is going to best suit its needs, being offered on an API for the best price at that moment. So that's already happening today. I think from a company perspective, one, I think we'll start to employ more of these agents within our companies to do especially the busy work that none of us should be spending our time doing.

There's much more value-added things that we could do for our businesses than a lot of the busy work that we have to do today. And I think agents will take a lot of that off of our plate. But equally, the harnesses that we're building within those environments, if a new model comes out, especially in the open source space, you should be able to switch those out relatively easily by just pointing to a new API key.

It should be as simple as that. We have built a product that allows businesses to do that. If you want to shut off your pod and plug into this API key, then you go have access to all of the open source models that we host, and you could pick them, or we can route for you.

There are other companies that do something similar. So again, there's a lot more choice within the open source space than plugging in and basically having Anthropic route you wherever they want to route you. And I don't know this to be true, but if I was Anthropic, and I had complete control of what was being served up to you, maybe I choose the most expensive model every time.

There you go. Well, folks, you heard it here. First thing you got to do is establish something that looks like a control plane that keeps you in charge, and then focus on a couple of things that are actually going to matter.

Hey, Tina, thanks for being on the show. Mike, it's a pleasure. Thanks again.

AI Leadership Insight series. You can find this episode and others on our website. We invite you to check them all out.

Until then, we'll see you next time.