The Technical Foundations of Articul8’s Agentic Al Platform

Hello everyone. I'm Rena nato, head of Technology, articulate, and today we'll pick under the hood of articulate platform architecture and understand what it takes to build, um, and run, uh, hundreds and thousands of, uh, domain specific, task specific models and agents. So there are many solutions out there and, um, it's a known phenomenon that's not hard to build a cool demo or a POC that you can run on your laptop and show some cool demos.

Um, but the, uh, also very common, which is they all fall short when you need to bring that in to run hundreds of thousands of concurrent flows bring to production. Usually that's when they, um, fall flat. It takes a whole platform that was designed in built for that.

So it can actually that in production. So we built, uh, that platform that can run, uh, on the customer premises, either that be on a cloud provider, um, on all the major ones or on their own, um, on premises that being even our ga right? No need to access, uh, external call to the internet.

That architecture is centralized on a Kubernetes, right, that allow us to build resiliency on the system where every component of the architecture is isolated and can elastic scale. For instance, we have our, um, uh, API service layer where those are stately pos that can scale, uh, horizontally and, uh, are high available, and then can route requests as you're receiving that. And this, it means few requests or hundreds of thousands that allow the platform to scale.

On the same token, we also have our model mesh orchestrator. That's where the intelligence of the platform, uh, is where it allows you to smartly run dynamic, uh, autonomous flows. And that was the challenge when we were building this platform.

There is no available out of the shelf solution out there that can do low la real time dynamic, uh, execution. There are many, many solutions that check some of those boxes, but very, uh, few are, we couldn't find one that run in production that achieve all of that. Um, and that's why we built our own.

Um, and then that allows, uh, the, that intelligence layer allows you also to dynamically and smartly route between many models that you have, uh, deployed on the platform as well. As you guys seen so far. Uh, we deploy internally on this, uh, self content platform, um, dozens of models and at the run time you need to be able to, uh, execute, uh, and run flows between those models in a smart way, right?

Um, and on the bottom of the platform here, we have the intelligence layer where, um, we persist and keep all the information important for enterprises like you have observability and, um, very important for in auditability, very important for enterprises, uh, compliance and also so to, uh, resilience as well. Uh, all of that we pro try to provide that to our customers in a very easy way with our, uh, interface with our CLI, that their IT team can just come and run on their cloud, provide that will provision everything. You don't need experts on your side to do that.

It helps if you have them, but, uh, that's not required. Uh, and that can also, because it's based on Kubernetes, can also be run, uh, in on-premises as well, what it takes to, uh, build domain specific, um, and, uh, task specific models and run them at scale, right? Building the models itself is really just 10% of the effort.

We know that it can be expensive depending on the size of the model, but the, uh, intellectual effort and the actual creating the model, it relies on, do you have the right data for it? So much aspect of it, it goes into our autonomous data creation where we have the knowledge graph, which we we're gonna, uh, show on the live video in a second, which is builds, uh, augmentation of your data. So you have your data, and then we create information on top of it.

That information it creates on top of the technology, which is unforeseen connections between your data that's very important, helps you to build those, uh, domain specific, task specific models. And then you need to understand what is the, uh, right architecture for that, uh, specific use case as well. And then be able to autonomously, you know, deploy those models on our model mesh, uh, and make sure that they actually achieving, uh, good results, uh, for that specific task.

So all of that needs to be orchestrated on a platform, and it, it takes, it takes a lot of effort to make that brand in, in production. Um, Atos the data, the knowledge graph, uh, generation updating, is that part of this model mesh, or is that, some of It is. So when you are ingesting data, you don't need to say anything about your data.

That data goes to our intelligent model match layer. It'll break that apart and understand, um, hey, this is a image. Um, it is a chart, not a picture.

So I do need to go to the chart, uh, task specific model to understand that, or this image is actually a table and I'll show in the demo while it happens. No. So that now it's a table.

I, I know, I know, uh, only need to do OCR, but I also need to do table understanding on this. I need to do summary statistics. I need to understand what this tables is about.

All of that happened during ingestions without the user provide more, uh, data about that. And that's key component of building a model. 'cause if you just drop the data, we talked about domain specific data with public available data on domain specific, you go to a public, like our partner, uh, every right, they'll have the papers, they'll have those PDFs.

But how you understand specifics about those papers, it's really important how good your model gets, right? That's 90% of the effort that goes in building those models. And questions about model size, right?

Uh, there is a also common misconception that domain specific models are all tiny or small models, uh, not the case, right? Uh, model size depend on model capabilities. That depends on the task complexity, right?

As you can see here on the, uh, on the bottom left on the plot, um, vertical axis is model size. And on the horizontal plane, we have data size and test complexity on the bottom left of it. Uh, or a use case, you have very little data.

Um, and that text, uh, is, uh, very complex. You might actually require a, a model with significant size. For instance, you're doing a manufacturing analysis of a sensor data that expire data.

Um, you might have very little data, uh, but you still need a model that can understand and comprehend that, uh, what that information's about. So you might need a very, uh, a large model to understand that. On the other, uh, side, uh, if you look where you have a lot of data, but your task complexity is not that high, for instance, I want just to categorize a bunch of, uh, support tickets that I have.

Um, then you, even if you have thousands and thousands of documents, you can still do with a smaller model that can achieve that ization. So that's key to understand. And we play a role in all the spectrum, have, um, dozens of models.

They go all the way, uh, to be on the large side and have, uh, dozens of models that go to the more, uh, smaller, uh, uh, specialized sizes as well. Um, now that we talk about what is under the hood, what is important, I actually wanna, uh, to focus, uh, and see it, uh, running live as well. So I'm gonna switch my screen here to our, uh, platform.

Give one second. There we go. And the first thing I'm going to show, um, uh, I hope everyone can see this, uh, clearly is, uh, a visualization of our knowledge graph.

And, um, it's really important to understand that's the visualization, right? Um, this data set is about, uh, aerospace engineering in general. 3 million entities on this graph.

So this is not, uh, a simple example. There's a lot of data here that came from 200,000 pages. So tons of data into, uh, generate this, uh, knowledge graph.

Our system automatically detect 160,000 topics. This is not given by the user. We understood that, understood what was the topic of that, uh, documents.

And we also create hierarchical clusters connecting. Then we're gonna, uh, that's what you're seeing on the visualization and 800,000 images and 130,000 tables. Why this is important.

And then this visualization, we call internally the shape of data. Um, it is a dimensionally reduction. So the access actually mean how the data is closed or far apart of each other.

So the clusters here on that corner here, for instance, rock proportion and com combustion analysis are further apart from the other corner of the, uh, um, um, of the, uh, uh, the cluster that talks about something else. Uh, like for instance, uh, altitude in a, uh, uh, orbit control techniques, all of that automatically done by the platform. You don't have, uh, chop in it anything.

You just drop the files here. How, how is that important that allows you and the platform at any time tapping into this, uh, uh, knowledge, either if you're building a domain specific art task specific model like we do, but also if you're running this live and you wanna understand some, uh, connection between data, and I'll show an example of what we talk about, uh, here. So the, uh, I'll do a live search here.

Okay? Um, all right. So you see how fast that, uh, search was.

This is running live. We are not, uh, um, uh, mocking anything here that shows where that search is correlated on the graph. So where in the, uh, semantics position that is, um, and bring that together, uh, for you all across those 6 million entities.

Um, the reason that I like that query is it identify a table image, right? So this was a image that was inside the paper. Uh, we not only identified that image as a table, but understood the table.

So this was the generate content that our platform with the table specific model could do. So it is explaining what the table is about. So it is not only doing OCR, obviously we do the OCR, we have the actual content as well, but also what the content is about that augments the data, and that gives opportunity to connect this with other, uh, similar information you have on the knowledge graph as well.

Very, very, very important. So explaining all of the data besides just, um, uh, doing LCR with a data, with a table expert model. And to be clear, this is really just the knowledge graph segment of your solution.

This is not the, the domain specific models behind aerospace or space or anything like this. No, they Can be used to generate that information. Hello everyone.

I'm Arun sub. I'm the department CE of Articulates. I wanted to augment the discussion here.

So one thing to first notice he searched for escape velocity. The table doesn't have escape velocity anywhere on it. It doesn't even have the term velocity anywhere on it, number one.

So if you go search for these, These v you know, V and yeah, are some indicators of velocity on table, yes, That's a v on the image though. On an image. I understand It's an image, but it's ocr.

And in addition, on top of it, to your question of this has to be understood by the domain specific model that understands aerospace. The description came from that. Okay?

And this is just the beginning of the process that goes in, right? And it's just starting off with that. There's more to come.

I'll stay around for his assistant. All right. I'm just gonna show another example here that, uh, uh, it is, uh, interest to understand as well.

Um, another thing that although is explaining a graph, a chart might be very hard to explain if you get a general purpose model as well. So here you see, this is this chart and our, uh, generate summary explaining exactly what the graph is about. Also, um, not straightforward to do.

It's very important if you are, uh, building domain expert knowledge, all of that. It is interconnected with the knowledge graph. This is important.

Um, how can this be used on a traditional matter? What enterprise are you using? If you're, I'm gonna shut, gonna start with a table stakes example, right?

Everybody has that. But I, I just wanna show how articulate the, uh, do things differently on here. So this would be, um, a question that we put on the platform.

I ran that, uh, um, um, yesterday to, to do the test. And we see that, uh, a normal question. We get an answer in about 30 seconds that goes through all the knowledge graph.

Obviously it tells you each section of that answer belongs to what, uh, paper, where they came from. It tells you the citation normal, uh, uh, answer, uh, uh, chat chat capabilities. That's not all on what we do too.

So we do what we call a detail answer, uh, on our model mesh technology. And we did this before, uh, deep research was invented, but that now, now that's common, uh, called other deep research, right? 3 minutes.

That means we are transversing your graph, seeing all the unforeseen connections that you have about that topic, and allowing the model match around that dynamically and make those decisions. And we actually show this here. This is execution graph.

here is an action that was taken on this dynamic execution graph for either from, for instance, guard res, uh, where, um, is it safe to run, uh, to, I need to do search, uh, and I need to break that down because this is too complex. I need to find documents. I need to do context relevancy.

I'm gonna show that in a second. So how much relevance is actually I found this connection, but is it really relevant for what I'm trying to do? That's not only pertinent on the question and the answer type of interface, but, uh, any mission that you wanna run on an agent system, that's really important and all of that.

So, question that typically people have is, okay, you have knowledge graphs from data that is static. You have new data that is coming in. This is not even new data.

This is actions you're taking on the system that's creating its own graphs. That's execution graphs that have to be combined together with the, the data knowledge graph to answer any questions. And now this is being added to the knowledge Graph.

Everything is getting added to the knowledge craft. Every action that a user takes in the platform is part of the knowledge graph, not just the user, user agent system, anything, whatever. Yeah.

And finally, I just wanna point that, uh, we have a task in domain specific models that look into your answer and tell me how good this answer is. And, uh, like I said, how relevant that, um, piece of content citation from that specific paper versus this one, how relevant they are. And the helps the user to understand how relevant it is and the system to, uh, surface those to the top as well.

Uh, very important as well. These are more common scenarios. So now that we have this platform that can run, um, autonomously, uh, agents, um, what else can we do, right?

How can we achieve hyper personalized outcomes? So one thing that we would like to do today is we would like to show you guys how you can build your own digital twin. Something that can, uh, be your digital persona in the platform to do things as execute, uh, tasks, uh, based on your persona.

The experience that we're gonna do here live today is we're gonna base on public information, but if you're an enterprise, you can bring all the data that you have privately and help build a digital twin for, uh, a employee, for a department, for something that you want, like a Factory digital twin, those sorts of things. I mean, is that what we're talking about? That's right.

And also like this can be as shallow as, okay, I'll just get your information from the public and then I'll start sounding like you to actually having something that works like you to expanding that, depending on how much knowledge you give. And of course, obviously it won't be accurate or perfect the first time you build it, but the more and more you give feedback, it gets better, right? Can You share digital twins across a team or Group?

Yes. Yes. And that's even more is coming.

So let's just, uh, go ahead, right? I, in interest of time, I'm not gonna, uh, And we didn't get, we didn't get permission from Ryan. Yes, we, time for me to Leave and go back to bed.

Got this going. So this is all your information here, Ryan. Like, uh, I found your linking, but I also found your Twitter.

I also found your private websites, uh, that you have, uh, Information, nice, private, keep it clean, Right? Everything is on the internet talking about you. This is internet information.

Yeah, yeah, yeah. So, uh, it gets, it can be used to build your digital twin. It takes about two minutes to build.

In the interest of time, I'm gonna jump, just Go, just go ahead and build and Then, uh, I'm gonna jump, I'm gonna show you what, what can you do with, with a digital twin. So just a quick, uh, example here, like, uh, with your digital twin, who are you? Um, what are you most About?

This will be interesting. It'll be interesting to you. Let's See.

Like having a live therapy. So I know, See for, oh, I should clarify that they did get permission from Ryan before doing this. That's right.

That's right. Yes. Right?

Uh, And like, we wanna make sure that this is, uh, only the beginning, right? Meaning once you build the twin, you can actually go in and teach it to be more like you exactly what you're going to do, right? That's the intent.

Then how Do you refresh the data set as his corpus roles? How does The, so we can have a schedule, for example, you can say every 30 days, go and refresh it. We can have, uh, a trigger that comes from the knowledge graph saying if new information becomes available on a particular topic, go in and update all of the downstream effects.

You could do all sorts of things. Now, this is pretty cool we could keep here, but I wanna interest of time again, I wanna jump forward and say we have a platform that can run thousands of concurrent calls and we can simulate digital twins of, uh, multiple persona. So the next, uh, logic step for us is like, let's put a bunch of you guys here and ask something and see what you guys are gonna talk about it.

So we asked permissions from, uh, both, uh, uh, Ryan, Nick, and uh, uh, Dave. They all gave us permissions to build their digital persona here. So we are gonna select them here.

There we go. We built them ahead of time. And now, uh, lemme see if I have that, uh, question.

I don't, lemme, I, I have a, a question here that, uh, it is interest to us. So I'll read, read out loud as well. So what do you think about, uh, domain specific models?

Are they going to be needed for complex use cases or general purposes? Uh, LLMs are going to catch up. That's, um, inter stock for a discussion, right?

I select you guys. Um, I put in that squad mode. I will explain as we are running what that means, right?

So now we are, uh, autonomous run the same questions, uh, to all the agents as you can hear on the main diagonal that's happening live. Um, as those users are getting complete, I can, uh, focusing one. So you guys see that this is running live.

This is using your digital twin answer that, uh, Nick is, is creating that based on whatever, uh, persona was created, but also after that is done, every other agent are going to opinionated about the other agent. Answer Does it, does a concern here, here, a security concern, stuff like that. You know, how do you, how do you lock down this capability so that somebody couldn't take and and do?

Absolutely. So that's why this is built by with consent. It's inside the lock, the enterprise environments.

And this is not open to anybody going and building a digital turn or somebody else. However, if your information is out there, It's public, it's public, I'll have public information, Public information, people can do that Information exhaust, so to Speak. But the, the next generation of it is if somebody wants to officially use your digital twin, you have to attest it.

And to the monetization question, if you can attest a digital twin and somebody is using it, you could potentially monetize it. That's really where it could go, right? Imagine, Which is exactly what like virtual CTO, like Keith is Already doing.

But here, the reason we did this is you can see the diagonal is the opinion of each of the experts cross diagonal is what they think about the other person's opinion. And then in that one round, here is the summary, and then no brainstorming session ends with one round. So the next round goes on and at each round some experts might change their opinion, some experts might dig in and the overall discussion can continue on.

And the whole point is when you want to get an an actual nuanced answer from a group of experts, it's not just a question and an answer. You need to go through analysis. And the analysis also is fully auditable.

And every expert who allowed their digital twin to be used can go in and say, you know what? That particular answer is wrong. I don't agree with it.

Change it. And you can read on it, you can go back to it, right? And, uh, we have, uh, systems where once, uh, everything is complete, it'll give you the summaries, it'll tell you whether the discussion is converging or diverging.

And if it's diverging, you can inject some other questions or topics to get it back to conversions. So all those things are possible. Our intent here is to get into enterprises to get outcomes.

It's way more than just a question and answer chat bot, right? And in order to do this, I mean imagine it's, it's nine calls going simultaneously, multiple things going in, and one has to wait for the other. All of these things have to work.

In this case it says it's actually converging. Uh, if we had time, we would inject, uh, a maverick who disagrees with everything that everybody says, and you'll see that, uh, the actual discussion diverges very quickly. His name is Ray.

Thanks Dave. I love you Ray. So this is just an example, right, with public information.

But, uh, if you're running this on your own enterprise, this is a multi-tenant system running production only your own premises. You can control with fine grain control. Who has access to what you control?

Who has access to your agent? You control who can talk with your agent on the a i level as well, this always controlled by you, uh, as you, uh, own that, uh, agent. So this is, um, no one can talk with your agent if you don't allow to.

Um, And where this would go is also where AI LOP has gone today, right? Which is anybody can build a digital twin of view of course in a general purpose system, but it'll only be as good as what you can potentially guess, right? And many times it'll actually give you misleading information.

You walk into a situation like this where if I ask are, uh, the eight people here going to agree with me or disagree with me, I'm more or less going to get an average answer, no offense to any particular field. But if you go talk to a, um, uh, an astrologer, you more or less get an answer that is on average, that's because on average more or less sounds, okay? That's really what you might end up getting.

But if you want specifics, you need to go get an expert who had tested their own digital twin. They should have trained it. So could you plug this into like a zoom, uh, call and stuff like that?

Absolutely. We've done, we've done that Relatively and, and provide a summary of Provide summaries. Say, okay, these are going to be red flags, these are going to be okay, meetings, Briefings.

In fact, it all started with me doing an experiment where I had a BS meter running and it basically went up and down and I was initially watching it, other people started watching it, but then it, uh, it started out fun. But then you'd be amazed the the BS score for meetings in some, uh, some enterprises. I think that's a must for y'all's next presentation.

Some equals mo And I just wanted to bring this up because you asked about observability and lability insecurity. Yes, this is a very important concern for our enterprise customers. Uh, although the actions are autonomous and dynamic, every single thing is observable.

You can tap into and see, understand what, uh, was action that was taken. You can change the behavior and you can have that set for yourself that will update the knowledge graph. All of that is observable.

You can see everything that was done on your behalf and you can see the details of it. We are doing a visualization here on that, uh, the flow. But you have on that, on the, It's not hiding the whole agenda side of this thing behind.

This is the outcome I'm looking for. Yes, it's, That's why this is, is actually hidden, but, but if you see it's, you gonna see it there, this out available, the API. So you know, if you wanna add it, that's important, but, uh, lemme see if that completed.

Yes. So after it is done, it, and then it actually gives you a summary about all the discussion that happened between, uh, the digital twins, what they achieved while they, you know, got in agreement, and also what are the action plans for you to do after that. Very important as well.

And obviously if you have that report, you can come export that as a, uh, as a, uh, a, a PowerPoint presentation, a PDF presentation that will come nicely from art for you, you can use as a report. So at least the eyes up here is a little bit better because, you know, you have digital twins actually debating about the content. It's not just a one shot.

Uh, Israel, there.