Lovelace Cuts AI Costs With Context Engines

ai Leadership Insight Series. I'm your host, Mike Vizard. Today, we're with Andrew Moore, who's the founder of Lovelace, and we're talking about AI, knowledge graphs, token economics, and well, the whole kit and caboodle.

Andrew, welcome to the show. Thank you very much. Very excited to be here.

We've seen suddenly that there's been a greater appreciation for graphs in general in the age of AI. People are starting to figure out that these things may be the missing link in a lot of these applications. You guys have been in this space for a little while now, and you led development of YottaGraph.

And now you've built a context engine on top of that. So explain the relationship between all these things, if you would. Absolutely.

So it has been so successful in so many places to take your AI, your large language model, and stick it in front of a piece of data, and you've automatically got a great application. This has been useful for creating chatbots. It is the simple uses of AI.

But when you stick a simple large language model in front of a massive wall of data, so for instance, if a bank tries to use this to help with anti-money laundering or credit portfolio analysis or something like that, this poor little agent is facing terabytes, petabytes of huge amounts of data all over the place, and it's very hard for it, even using this technology called RAG, which it helps it search for relevant objects. It's very hard for that poor little agent to figure out how to answer a big question. And so a lot of us are fighting a battle to put something here between the little agent and the massive data, something here in the middle, which provides very useful signposts for the agent's questions to get a plan for how they're going to be answered before going to this terrifying wall of data.

That kind of thing is called a context engine. It is the thing which means that rather than asking your agent a question and having to stuff its prompt full of background information, the agent can just figure out its question, and then the context engine figures out what to do with this massive wall of data in order to answer the question. Now, as I understand it, you went beyond just creating a graph that I would throw in in a particular use case for in an enterprise.

You've created this kind of massive index, as it were. It's kind of almost akin to something that feels like a search engine. But explain that part as well.

Yeah, it's very interesting. So there has been this community, including a lot of... I used to be at Google, and we used this concept a lot in Google, but a community of folks who really view the world, everything going on in the world, everything on planet Earth that's happening right now, it is a set, a massive set of entities, nodes in a graph, where those entities are things like my car, me, Lovelace as an organization, Mike Vizard as a person.

And the world consists of billions of entities, and then between them, there are links. For instance, Mike and I have a link that on this particular date and time, we were talking over a Zoom meeting on the topic of context engines. That's a link.

So what we decided to do was build the biggest graph which link every piece of public data together with all the information about how they're linked, where they're linked, and so forth. " The agent goes and finds Dell, the entity, and it's already sitting there in a very, very fast, compressed access form for it to quickly look, get information about the competitors, and start to rank order which ones it should talk about and move on from there. " It can answer that question.

It can look for the location of the Straits of Hormuz in this context engine. It can then find out all the ships which are moving around there at the moment, and it can then look up who owns both the ships and the cargo and come up with an answer to do that. And you'll see what's happening there.

This LLM is using graph traversal algorithms, very, very fast jump between lots of links in this very fast data structure, instead of having to use its own approximation of a human-like brain to try to do all of this reasoning within its own limited context. Now, this just isn't an exercise in interesting computer science because there's a real-world cost tied to this little agent that could, and if I keep stuffing stuff into its memory, A, it slows down, B, it'll probably hallucinate more, but more importantly, it consumes more tokens than we need to. So, is this kind of the way that we're going to maybe, the word of the year seems to be affordability, but are we going to make AI affordable?

I really hope so. So it's so funny you mention this because it wasn't the reason I founded Lovelace, but we did an experiment a couple of weeks ago, we finally published it, where we used this graph reasoning-based approach to answer deep research questions of the type which would go to Google Gemini Deep Research. And our answers came in at approximately the same quality to within limits of measurement, but at between 1,000 and 100 times cheaper.

So these deep research things, which Gemini was chomping on for 15 minutes, we would answer in a small number of minutes, and we would use about six cents of tokens to answer the question ourselves, where it was costing $7 of Gemini tokens to do so. So do we need something that feels like a set of best practices for building agentic workflows? Because I feel like everybody's going down the same path, making the same mistakes, skinning their knees, incurring the same misadventures, and yet others have been down this road before.

What it does need is people who are very stingy about compute. Folks who love to squeeze every little bit of compute out of any kind of server or GPU of any size. At the moment, foundation labs, they've got a bigger battle.

They are racing to see how far they can push this large language model type reasoning, and it actually wouldn't make sense for them to stop, spend two years making everything cheaper, and then keep going. So they're willing to pay the cost. Rest of industry, it hasn't taken long before we've hit this point where any cost savings you get from reducing human toil are being matched by the massive cost you're incurring on actually running these pieces of AI.

So for those of us in this part of the space, it's the perfect time to try to reduce the costs of all of this, so that people can actually progress with AI. There's also a subtle point in all this in the sense that the more dependent upon I am for the LLM to do the reasoning and the processing, the more tied in I wind up being to that platform. So I think as we go along, people are starting to realize they want to be able to bounce from one LLM to the other one in terms of best one suited for a job, which one costs less to do that job, and there are just some fundamental economics tied to independence, right?

Yes. If you don't mind, can I add something to that? Because I completely agree with you, but there's something else I really care about here, which is, it's not just about cost for me.

I started on this path because I really care about safety-critical AI. What happens when an artificial intelligence is giving advice for a problem where massive amounts of money or human safety might be at stake? And in those cases, there has been decades of really, really careful work on how to combine uncertain information using probability and something called Bayesian reasoning.

There's been real care in how you deal with five different sources of information where you think that at least one of them might have been poisoned to make sure that you can reason in a way which gets you a good answer. Now, LLMs are brilliant at dealing with underspecified problems and things where you have to think out of the box and reason by analogy. They are terrible at doing this kind of rational combination of multiple, possibly contradictory pieces of information to help form a opinion which they can show to have corroborating evidence for and so forth.

So sometimes those of us here working in 2026 in artificial intelligence need to build a bridge back to 2019, to reuse some of the sort of mathematical tools of cognitive reasoning that we wanted to use there. The right tool for the right job. LLMs for doing some of the higher level analogical reasoning, but for God's sake, use probability theory when you're dealing with uncertainty.

Use game theory when you're dealing with deceit or competition. There are tools out there, and let's make sure we use the right tool for the job. To your point about that, though, we live in a world where we can't seem to agree on what a fact is.

So how do we come to some sort of consensus that we can agree to that the AI agent can rely on in the first place? It kind of feels like at some point, somebody's making a decision about something. Yes.

So I think the... Really, really good point. So I think there have been times when people have proposed knowledge graphs as a sort of representation of all the known true facts in the world, which is an interesting idea.

It's very much not what we do. What we use the graph for is to represent what we would call all the observations of the world. So an observation might include the fact that a journalist said in some article that such and such a person was the CEO of such and such a company.

They might have made a mistake. We record the fact that this journalist said this about this in this article. " We very rapidly, and actually in using really, really fast algorithms, mostly in RAM and using GPUs, get all the evidence about who that CEO is so that we can rapidly put that together.

But inevitably, most questions like that, you'll find there's inconsistencies there. If we see 12 different journalists on highly reputable news sources, like Techstrom, have reported something, and then one crazy blog, which has often made incorrect statements, disagrees, we take that into account when we're combining the evidence. So think of context engines, at least the way I think of them when we're dealing with safety-critical systems, is it's a place to store all the observations about the world dispassionately.

But then when someone asks a question about them, that's when you have to start thinking about which of those observations you trust, which ones you can't trust. You also hit on this, but let's make it a finer point. It seems to me a lot of organizations and businesses especially, are struggling to operationalize AI.

And at the risk of a gross simplization, they're trying to figure out how to take a probabilistic technology and wrap it into a bunch of deterministic workflows that need to be done the same way every time. And yet, the AI doesn't do anything the same way every time, unless maybe it's a machine learning algorithm. But how do we kind of balance these things in a way that lets two plus two equal six or better?

So yeah, no, and this is a problem. In fact, when I talk about the need for managing uncertainty, I'm not saying I would tolerate an algorithm which gave radically different answers each time. What I'm saying is I want an algorithm to admit, and actually put quantities on it when it's uncertain about something.

So if I ask it, where is the British Airways flight between London and Pittsburgh right now? It won't tell me the exact lat long coordinates, but it will be able to tell me roughly where it is because it will know a certain amount based on information it has available. So I want these things to admit their uncertainty and to use rational tools for doing that, just like we humans have done for at least the best part of a century.

Rather than pick one plausible answer and confidently state that this is the answer. So everybody's kind of having that same question, but how did we get to this state of affairs? Because everybody has noticed that the AI will confidently give an answer regardless of what the answer is.

So is that just built into the LLM, or where does that logic exist? Yeah, I don't fault the LLM or the people who built the LLM, but I do fault folks who stood up on stages and told us all that it was going to be really easy to just use these LLM. So the LLM will find a plausible sequence of words or a very clear chain of reasoning that all adds up and cannot be improved in its own local optimization of its response.

So it searches for internal consistency with every little individual thing in the response. But its reasoning as it's doing that is it's specifically using a beam search to find one good answer where everything fits together. I'm going to say that it is, the LLM is truer to us humans.

If you ask me, a human, about something uncertain, I will not actually usually give you very good probabilities. Humans are very bad at doing that. So I don't blame that the LLMs comes up with these sort of overly confident, I'm an expert, you must trust me, types of answer.

What I do complain about is when there's genuine uncertainty in what's really going on. I want us to use the tools for dealing with uncertainty. My company is called Lovelace because it's named after Ada Lovelace, who's one of the pioneers in probability, and so that's my nod to her legacy.

Out of curiosity, are we holding the AI to a higher standard than we do humans? Because we all have known people, some of whom are our friends, who confidently give us answers that we know are wrong, but it doesn't seem to bother them from the fact that they are confidently giving an answer. And the AI agent I don't know.

Is it any better or any worse? That's a great way of putting it, actually. I think if you ask a, suppose we're dealing with a court case where people take evidence very carefully.

You might ask a regular human in the judge's chambers, are they guilty? And the answer is, yeah, obviously. Look, how else could they have got to this place at this time?

Whereas the process that we humans have created to actually take much more care than a gut human reaction is to look at every piece of evidence and be very skeptical that if something doesn't add up, we're not willing to proclaim that we've got the correct answer until we've explained the discrepancies. It's the same with the processes used for targeting strikes in the military. The human processes are very careful that a subordinate has to explain to their commander all the pieces of evidence, which means that this is a good decision.

We've built processes to account for the fact that an individual human's gut reaction is not appropriate for this kind of decision. So it's crazy if we then invent these LLMs, which are like rough simulations of human cognition, and give them the responsibility that we don't even give to ourselves. Where are we on the journey?

And I'm asking the question because I feel like there's been a massive amount of experimentation that's gone on, and some notable successes, but probably more miscues than successes. But at the same time, I can't help but wonder, are we on the cusp of something greater? You see flashes of brilliance involving AI, but it's not quite everywhere, and it's not evenly distributed, as that saying goes.

So where are we, and what's next? Artificial intelligence in the mid-2020s is no longer an engineering discipline. It is a natural science discovery type of discipline.

No one, none of my peers, certainly not me, can explain how they actually work or how they get done what they're doing. So we have to explore how we can make the best use out of these really weird and wonderful things which seem like intelligences. But that exploration involves some toil.

" It is instead a tool to be used as we figure out, by experimentation, how to insert it in various different places to disrupt something that we want to have disrupted. But it actually involves a lot of work and creativity from experts in, in this case, sex trafficking, as well as in data management, real-time data security, anonymization technology. It requires all of that work to still happen, and the one thing that I'm really frustrated about is the-- By the way, your organization does not do this, but I'm really frustrated when I see people saying, "Oh, yeah.

" A, that means that all the people who you need to solve it suddenly become disenfranchised, and B, it's just not true. You need to do so much work to figure out how this strange phenomenon of large language models can be tamed for each of these different use cases. So to your point about that being a natural science, as it were, we've seen the rise of quantum computers, which are fundamentally based on natural science principles, at least how I understand it.

So might we not use the quantum computer to one day explain to us how the AI works, and then conversely, we're using the AI to talk to the quantum? I'm very interested in the history of technology development all the way through the Industrial Revolution and so forth, and you absolutely see this phenomenon where people think that one thing by itself is going to be really big, but then it kind of goes dormant for a few years, and then when it's combined with something else, you have these breakthroughs. So I think it's absolutely not a good bet that it'll turn out it's LLMs plus quantum, which mutually reinforce.

But I do think that we will still see these staggered sudden move forwards when pairs of technologies match up and help each other out. So what's your best advice to you folks? Or conversely, what's that one thing you see folks doing today that just makes you shake your head a little bit and go, "Folks, we need to be a little bit smarter than that"?

" That's what has to stop. It has to be the answer to the CEO is, "I'm really excited about the use of artificial intelligence. " But it is not automatic.

It is not a done deal that we'll be able to deploy it. All right. Well, folks, you heard it here.

Even in the age of AI, there's no such thing as magic. Act accordingly. Andrew.

Absolutely. Thanks for being on the show. I really enjoyed it.

Thank you so much. All right. And thank you all for watching the latest episode of the Techstrong AI series.

Please check out the rest of our website, and there's plenty of other episodes similar to this. Until then, we'll see you next time.