Enterprise AI Starts With a Context Layer

AI Leadership Insight Series. I'm your host, Mike Bizard. Today, we're with Gilad Salinger, who's the CEO of Uvi, and we're having a little chat about how to reduce our AI consumption costs.

Why? Because, well, tokens are expensive these days. 5 million to figure out how to reduce those costs.

Gilad, welcome to the show. Thanks so much, Mike. Pleasure to be here.

So walk us through what you're doing, because I think everybody's having the same problem. Everybody's using AI as much as they possibly can, and the more they use it, the more the bills go up, and everybody starts to maybe lose their patience a little bit. So what's to be done here?

Yeah. So basically, the problem that we're tackling is that every agent starts the job knowing nothing about the company, right? So, it rebuilds the knowledge on every question, and the company pays a frontier model to do it.

So think about that, kind of the AI grinding the knowledge through MCPs, the different datasets. You have many datasets and different models that are working, and basically, it costs a lot of money. And we fix this.

So basically, we own the context once outside the model, then every agent can use it, and you can swap models without relearning the business. So that's basically it. So how does that work exactly?

Because I have so many different sources of data throughout the company, and so how do I create something that is static enough for AI agents to use? What does that look like? Yeah.

So that's exactly the challenge. How do you deal with large amounts of data? Our clients have terabytes of data.

For instance, one client has more than half terabyte of code, just code. So think about the additional data through tickets and more datasets. And the challenge is to basically represent all that live with the permissions and the security that is needed for such enterprises and being able to then narrow it down and traverse it at a scale, and then basically get the agents whatever they need to operate.

And that's what we do. It's building the relationships, understanding the relationships of the data, being able to traverse that graph. Then down the road, obviously, understanding the unique way the enterprises communicate internally, and how does each company basically identify things in its unique way, and then making this live at all times.

We hear all the time that organizations are struggling to get past the initial experiment, which is usually a finite set of data that somebody put together, and it worked in the lab. But once you get into production, you're dealing with massive amounts of data that is maybe somewhat related to each other but hasn't been classified all that well. So, are you not only helping me deal with the cost, but you're helping me deal with the relationships between the datasets so that I get something that is more cogent for AI?

Yeah, I think that context is heavy lifting, and it's here to stay. I think that technology of the future is going to be a mix of context, identities, and the LLMs. And the context part is something that the enterprises are basically trying to defend and have their own unique way of working and keeping this kind of remote at the moment.

And they don't want anyone else to see this, right? So you have to do this precisely to each one of them, right? So it's basically doing this heavy lifting, and being able to do this at scale is a challenge.

And we've been doing research in that space for about three years, so it took us a while. We sat on the fence for quite a while, focusing mostly on research and figuring out the best way to do this, to build these graphs, and using different technologies in order to make this work, and that's our job. That's basically, we love this because it's so dirty and heavy lifting and requires really basically reconstructing this each and every time for each company.

Obviously, we do most of the work automatically, but still, you have to maintain it and to execute it per use case, and that's something that we've been doing. I think the other issue that people are starting to come to terms with is they don't want to necessarily be locked into a particular AI model. And if I use that model to do most of the reasoning, the more I get locked in.

So, does the context platform kind of create that layer where I can treat these things almost like back-end commodity services? Yeah, I think that we see more and more. So more and more companies use more than just one LLM provider.

And as in the past, companies used to optimize cloud costs with different cloud providers, companies do the same with LLMs and navigate the LLM, basically the usage through different LLMs for different use cases. Also, you see the different models and the different capabilities they have. And the same with tools.

We believe that as of today and in the future, companies are going to keep on using many tools to do their jobs. Each company is going to have their own way of using different tools, and it's going to remain the same. So that's another challenge.

How do you adapt to the different workflows and the way each company works, and treat basically the different datasets as it is? For example, we have codes. As datasets, we are very experienced with code, which is codes, for instance.

If a single agent needs to execute a task that requires a context from code and from Snowflake, let's say, for raw data, as well as the decision is in Slack, right? So basically, being able to call these three different tools and to narrow it down to understand, to basically get to the understanding that these three specific pieces of context are required for the agent at specific time and place to execute the task. That's something that's basically, that's the level of understanding that we need to get of an organization and how it works internally in order to do this at scale at all time.

There's a lot of talk these days about open-weight models. Do you think that they're going to play a larger role as we go forward? If we have the context engine as kind of separate from the AI models per se, will people start to use a wider range of AI models beyond the obvious frontier ones?

Yeah, I totally think that we are going to see more and more ways to use such models and techniques. We've seen the decision that Jeff just released, and it's super interesting. I think that it adds more complexity to what enterprises are facing these days because you have to adopt these in order to stay relevant.

And being able to adopt so many different models and new technologies is a challenge, and it's something that we've been seeing already within the clients that we are serving, and we are helping them to execute this and basically to connect this to their context. I think that in general, you've been interviewing and learning about CIOs and data in companies for more than 25 years, and you know that this problem has been around forever. Everyone knows that managing knowledge is, excuse me, but it's hard, and it's not forgotten, and it will keep on being like that.

But I think that the bar is just rising If you want to adopt all these different models, all these different tools, and to execute this and deploy this on your environments, you will have to have very good understanding and, basically, ability to manage and kind of use your knowledge wisely. So that's the challenge that we are facing, and I think that over time, it's just basically being bigger and more difficult to do, because also the expectations are getting higher. Okay, I hear you.

" You're talking from a- From- From what I see inside enterprises? Yep, absolutely. Okay, that's a great question.

So I was on a trip to New York City two weeks ago. I met with a few very large companies, and I've seen what they've been building internally. I think that one of the biggest questions lately is buy versus build, and many companies have been choosing build.

So I see companies that are trying to build company brains internally and their context layers, and they face all the challenges that we've seen, that we have tackled through the years. One of the most interesting problem that I've seen is company that is trying to separate the agents and the company brain for their business and the company brain that they are building for their tech, for the R&D. Then you have basically two different entities that serve agents that necessarily need sometimes context from cross-departments.

So basically, the company is a software company. They obviously have code-grounded answers in their code. They have the business logic, which is in the client's hands, and the client's in a conversation with sales and support, and they are struggling with agents that are trying to execute tasks that will require context from both sides.

Okay? So the context layer that you develop as a company, and I'm saying this out loud, and it's not a secret, you have to have understanding across the board in the company. You have to have the ability to connect the business logic on one side, on the sales and the technical support side, with code-grounded answers, because you will see that many use cases require these connections, and it will basically be a big impact on your agent's ability to execute precisely at scale in your organization.

There are a lot of ways to go about context. There's knowledge graphs, and other people have index engines, and all kinds of stuff is being thrown at this, including various databases. What's different about your approach, and how should people be thinking about context engines?

So I think that lately, companies have started realizing that just one type of context graph or data, a vector database, it's not sufficient for all the use cases. And you have to have basically different pieces, right? So our company, the contextual graph that we are building is not just one graph, it's not just one technology.

It's a mix of things. It's a mix of things that we've tackled and challenges that we've tackled through the years. And I would say just in a high level, okay, because we don't have time, and also it's our secret sauce, but I would divide it into four pieces.

One is the ability to connect the pieces of information, so basically doing the relationships, the relevancy, and understanding, and being able also to traverse the graphs. And then it relates to what I just said, being able to state that one piece of context that is further on one side is connected to one that is on the other side of the organization, for example. So that's the kind of the necessity of, and the functionality that you have to get within the context graph.

So that's one thing. In addition to the context graph, it's the enterprise graph, being able to understand the way a company identifies things internally. What's a feature?

What's feature flag? If an agent needs to know what blockers are there in a specific feature, how do they identify features at all? What does a feature have?

Because it's not connected. The feature flag, where do they get to the decision, et cetera. So basically understanding the company and the story.

We love doing this on the timeline, so understanding the stories behind the companies and within the different teams and departments over the timeline. And then you can basically state which piece of information is more relevant than the other. It's kind of a deep understand.

The enterprise graph, it's a deep understanding of the organization. Then the entities, under the understanding of the different teams, the interaction of the people with the information. That's the third thing.

And the fourth thing is being able to mine and find information that is not documented, right? And that relies on two things, which is, one is a prediction, right? So being able to predict.

You see the two pieces of information are related to one, something is missing in between. How do you predict, and how do you get a conclusion that something is necessary here in order for an agent to do something? And if you're not able to do this, then you have to do it in other ways of speaking to the folks.

We can do this with agents as well. We are trying to avoid APIs. And lastly, obviously, permissions.

So permissions, it's a huge subject, and it's something that is getting more complex over time. I just had today, a deep dive into this with our CTO, Dor, and he showed me a kind of an infographic of how do you define this at all? How do you understand?

Because what we did in the beginning is inheriting the permissions from each application. So if you have three different application with three different permissions, people are building a weak LLMs, I'm sure that you have heard about this lately. So building weak LLM that constructs out of three different tools that have three different set of permissions, and an agent interact with this weak LLM, how do you keep the permissions moving forward?

What the agent can do and what agent cannot do. So basically, these are the kind of in high level, how do we treat this, and we have a lot more to say in each subject. All right.

5 million, it's not OpenAI billion-dollar investments, but it's still a significant amount of money. So what's the plan from here? 5 million?

Yeah, that's a great question. We believe that with smaller teams and people with experience, you can get very far these days. We obviously are very agentic from the inside.

So we are going to build a link team that is going to expand into the US. We have started building the team in the US and the R&D team in Israel, and we are going to spend the money on marketing, on team, and basically supporting the clients that we already have. All right.

Folks, you heard it here. I know everybody starts with the model, but maybe, just maybe, we should start with the context and work our way backwards. Hey, Gilad, thanks for being on the show.

Thanks so much for your time. Lovely being here. ai Leadership Insight" series.

You can find this and other episodes on our website. We invite you to check all those out. Until then, we'll see you next time.