Why Data Readiness is Killing Enterprise AI

AI Leadership Insight Series. I'm your host, Mike Bazare. Today, we're with Sushanka Das, who's the CTO for Datahub, and we're having a little chat about, well, data management and context engineering, and all these things are starting to converge.

And in some ways, maybe we're just heading back to the future, but I've noticed one thing. If we don't get the context right, the AI isn't right. Sushanka, welcome to the show.

Thank you, Mike. Indeed, we are having so many "Back to the Future" moments. When I started my career in data, I was a young engineer at LinkedIn, and I had been hired to replace their aging Oracle database.

That was really my kind of Mission Impossible, replace a source of true database for LinkedIn, and while LinkedIn was about to go IPO. So it was a pretty hard problem. com right now.

So that made me kind of fearless, I would say. It gave me my first taste of what it takes to build a source of truth database, and I've never really looked back since then. But I did get stuck on a pretty hard problem, even while I was at LinkedIn, and that was really making data easy to understand and easy to manage and easy to use for all of LinkedIn's internal employees, and then turning it around and making it be something that was in the product and usable by the whole world.

And that really led me to create Datahub. It seems like, to me, a lot of these issues have been around forever, and we just kind of didn't really address them correctly the first time around. But now in the age of AI, seems like there's more importance than ever of making sure that the right data shows up at the right place at the right time, and that's the context.

And yet we don't really seem to have the mechanisms in place to make that happen. So I feel like there's, at least within the data management hierarchy of things, a missing component in the age of AI that people are still wrestling with to figure out how to implement. But is that a fair assessment of where we are?

Absolutely fair, and you shouldn't take my word for it. We actually did an independent survey of 250 IT leaders and asked them, "Hey, are you AI ready? " And a bunch of questions.

It's a fascinating survey. And the most interesting thing that I noticed when I got the copy of the survey results, it was done by an independent firm, so we had no input into it. But we got the output, and it said 90% of data leaders and IT leaders say that they are AI ready, but 87% say that data readiness is their biggest blocker.

" And so we looked at it and went, huh, we've got a real perception reality gap playing out in this survey. You've got leaders saying, "I've got it all under control. I have been investing in all the right things.

" And then you ask them about everything that's getting in the way of AI going to production, and they say, "Well, my data's not ready. My context is not ready. " And I think as an industry, we have realized that maybe two years back, we were all obsessing about the specific model.

Are you really on the latest one? Which specific version of subversion are you on? And now we're not really talking so much about that.

I mean, the model is the model, and it's evolving. What we're talking about is the context. What are you feeding the model?

And that's what's going to determine your outcomes. And also to your point, will I not wind up selecting different models based on how well they handle a particular use case in, quote-unquote, the context that goes with that? Yeah.

I think horses for courses, right? Different models do better at different tasks. But the thing that is universally true is that the performance of a model is a big determinant of the input that it gets.

Of course, the age-old saying of garbage in, garbage out holds true. A16Z wrote a big piece about it a couple of weeks back, I think got a million plus impressions. They talked about this context layer that needs to get built, otherwise AI applications are going to essentially just hallucinate all the time and people are going to lose trust.

Bessemer actually invested in us literally on that thesis and on that belief that Datahub is positioned to become that context layer for AI. And I think you brought up a good point. It's still an old problem, but I think the problem that has happened is that old problem which used to run at kind of human scale or at human timelines.

Dashboard, someone created it, it's wrong, or it was built off of bad data. Okay, someone's going to smell check it. Someone's going to go back, fix it up.

Next day, it's going to be fixed up, or maybe next week it's going to get fixed up. That was the pace at which the data world used to operate. But now with AI and AI optimism and AI agents within the enterprise, you end up having CEOs and CTOs, and I'm as guilty as the next one, right?

Just chatting with an agent and asking, "Hey, so how is this customer growth looking like? "And if we get wrong answers and directionally incorrect answers, and we are generating so much at such a high speed, the mistakes compound, and they compound super fast. " And to your point about that, one of the things that I find people are struggling with as they wrap their minds around this is a lot of the processes, and especially in business, are deterministic in the sense that they got to be done the same way every time, and I need an accurate result.

And the models themselves are probabilistic, and so we're trying to kind of meld the two to kind of create some kind of value add on top of that. But it seems to me the only way to narrow the output of the probabilistic is the more context you get, the more deterministic it will be, and that's kind of the art of this thing. That's right.

In fact, the word context is simultaneously AI's biggest gift to engineering and its worst gift, or what's the opposite of gift? Curse, to engineering because we're getting on this, and we're talking about context like we actually know what it means. But the reality is that when an AI engineer sits down and reads the context engineering manifesto, I hope someone's written one, but the equivalent of that, they're basically thinking about how am I going to tune my prompts?

How am I going to actually execute retrieval? I've got this token window, and I need to arrange and organize the context that I feed the model within the context window in this way. And that's what an AI engineer calls context engineering.

It includes, obviously, getting context from somewhere. It also includes storing results and storing traces somewhere and then learning and all of those good things. But what it misses is the fact that that context that you sourced, that you thought was good, could be complete garbage.

It could be completely missing the nuance of the business. And when you and I talk, and I'm presuming here, but when we talk about business context, when we talk about, "Hey, you know what? " The context that we are talking about when we talk about as humans are organizational context, business context, understanding about a project.

And the two contexts are kind of the same, but not exactly the same. Right? And when we talk about context engineering, we say, okay, we're engineering the window, we're making sure that it all fits, and it's beautifully organized for the model to make sense of what it's seeing.

And I think we're missing out on the bigger problem, which is context management itself. And you brought up data management. I think context management is the new problem to solve for organizations because there is so much context, and I mean here business context as well as technical context, right?

Where are all the tables? Who created them? Was it refreshed recently?

Is there an incident or a data quality report that's published about it? That's kind of technical context. But then on top of that lives your organizational context.

Are we even doing this project anymore? Who even funded this project? Was it like last year's project and no one's even paying attention to this, and there's a new project that actually has all the attention, and it's where the company's betting all its initiatives?

All of that is organizational context, and most of it is locked up in people's heads, and some of it occasionally gets written down somewhere. And if you don't have that, then it doesn't matter how good your model is, it doesn't matter how good your context engineering engineer is, your AI agents are going to make mistakes. So context management, which is kind of this discipline of first sourcing context from humans, I call it brain to text, and then sourcing context from machines, somewhat easier, still a hard problem, and then managing its life cycle to make sure that when you finally source context for context engineering into an AI agent's context window, you've got the highest quality, grounded, trustworthy context so that the model can actually show its powers.

So long rant or ramble, but I think context engineering is kind of a narrow way of looking at things, but context management is really where organizations will either win or stumble on AI. One of the things that I've seen people not quite appreciate is that context requires memory, and people are interacting with their AI agents, and things are going fine, and then suddenly the AI agent starts to go off the rails a little bit because it's not remembering that some core element of a conversation or whatever, and what the people are not appreciating is the fact is that the AI agent just ran out of memory and doesn't have access to maybe the initial part of the conversation. And so it's part of the context engineering just to remember to refresh the mission as we go along, because, well, AI agents are like humans, they forget.

Absolutely. An AI agent, a naive AI agent, is like an employee that just joined your company and got a badge and got access to every single ID system that your company owns and now can roll out kubectl commands and ship stuff to production. If that's not scary, I don't know what is, right?

And so our job, actually, as engineers and leaders is becoming, how do we make sure that we can onboard digital employees at the speed of light? And literally, agent pops up, they actually can bootstrap organizational memory, they can bootstrap their own local session memory that they've had with this particular individual, and then they can also bootstrap um-I would say contextual memory, which is, hey, on these topics, these are the conversations that I've had, or these are the activities that I've done, and these are the outcomes that happened. It's really a combination of all of that that is needed for an agent to do a great job.

And people are calling it reinforcement learning. There's many technical terms around it. But the core principles, and this is so magical about this whole AI wave, is that if you start thinking about how humans operate and kind of map it to how the world works, it actually is pretty simple when it comes down to the core principle.

Write things down, remember things, remember to organize your thoughts, remember to organize your files and folders and your journals, and if you don't do that, you're not going to be an efficient person. " Well, a lot of people think that all I need is to point my agent to my warehouse and MCP my way through an answer. And that's where we see that they stumble, because an agent that is connected up to just a pure data store can be very efficient at retrieving data, but I don't think it has any idea what it is doing or why it's doing it.

And so the semantic layer, which really defines what data lives and how data is operated on, lives across systems. It does not live within a system. I was talking to a semiconductor company.

They have 15,000 engineers, and they're using AI daily, like AI coding agents, autonomous agents, things like that. And their number one problem that they're trying to solve is context routing. So they've moved from this artisanal, I'm going to engineer context for a specific app or a specific agent to, I need to think about context engineering and context management as a central discipline, like a platform discipline.

We ourselves ran through this. We are a small company, and guess what? We built a bunch of agents.

I have Lisa. Of course, The Simpsons are kind of the most attractive, I think, franchise to build agents around. I don't know why.

We had the Ralph Wiggum loop, and that led to OpenClaw. But we, in parallel, were building agents named after The Simpsons. So I built Lisa and Bart.

Lisa, of course, is our customer intelligence agent. Very thorough, very meticulous. And Bart is our super-fast coding agent, scouts around and fixes bugs and ships them for a human to review.

And the first few rounds were terrible. Tons of hallucinations, lots of problems. Lisa was giving great insights about something, but the data quality was terrible.

Her ingest pipeline was completely broken, and she didn't know about it. So of course, now we're running Lisa and Bart on top of Datahub, and they're reading all of this context from a single context layer. And they are writing memories back.

And I still remember that moment when I thought to myself, "Eh, I think we're too small. We don't need Datahub. " Because I only have two agents to ship.

Turns out I was wrong. And so if companies this small can be so wrong about how to organize context for their AI agents, and we are pretty tech-forward, I'm very worried about where the rest of the industry is going to go, and I think that we will see a few stumbles and a few faltering steps before we kind of recover and kind of understand how to organize that reference stack for agentic AI development. To your point about that, will context engineering kind of be a layer unto itself within the IT environment, or does eventually it just be subsumed into the data warehouse or becomes part of that functionality as kind of an orchestration layer embedded in the hub?

Or, I don't know, I guess two things can be true at once, but how do you see this evolving? Well, I see that there is context engineering, which essentially becomes the new software engineering almost. Essentially, the software engineers of the future will be directing and building agents.

And so when they do that, the kind of tools of the trade, the tips and tricks that they will be learning and optimizing. If you go back a decade, we were learning SQL optimizations, we were learning how to write the partition query just the right way. Java developers were figuring out how to allocate off-heap memory so that they were not having garbage collection pauses in the middle of an important event.

Those were the kind of software engineering experts doing their work back in 2010, 2011. And now, where I think engineers are going to spend their time is in making sure that they're sourcing context and organizing context in the right way. And so that context engineering discipline, I don't think goes away.

I'm sure libraries and frameworks evolve, and you kind of pick and choose the best one. Where we see that the new world moves to is the role of the business user, the pointy-haired person who just comes in and asks questions and walks away, becomes actually pretty important. Because in the future, when they engage with AI, they're not going to be engaging through just one interface.

And so Datahub, actually, the way we look at the future enterprise is that Datahub as such disappears and becomes kind of invisible infrastructure connected up to every single AI surface area that you have in the organization. So maybe you're in Claude, or maybe you're in ChatGPT, or maybe you're in Cursor, or maybe you're in CoWork. Wherever you are, you are essentially sourcing context and connected up to the same kind of context layer within your organization.

And in the act of asking questions, you will be asked to explain yourself. " The agent might be like, "Well, I found a couple of conflicting definitions. This one says blah, blah, blah.

This one says something else. " And so I think that crowdsourcing and showing up in different AI surface areas and kind of a curation of context as an activity that happens organically is really where the future is going. And so context infrastructure just becomes shared and important and essential, and then the AI surface area just kind of connects up to it, almost like JDBC.

There's a CDBC context protocol, and all AI agents speak it, is kind of what I see the future looking like. All right, folks. You heard it here.

This whole AI thing, maybe it begins and ends with context, because without it, well, it's just a bunch of bits that we don't know what to do with. Hey, Srishank, thanks for being on the show. Thank you, Mike.

This was great. All right. ai Leadership Insight Series.

You can find this episode and others on our website. We invite you to check those all out. Until then, we'll see you next time.