OpenWALDO Brings Open Source Discipline to AI Training Data

ai Leadership Insight Series. Today we're with Gregory Kurtzer, who's the CEO for CIQ, and they just helped launch a project called OpenWaldo, which is creating a repository of data for training AI models. Gregory, welcome to the show.

Hi. Thank you so much. It's great to see you again, Mike.

Good to see you. Why do we need this? Explain to folks what is the mission here?

What are we trying to standardize here? Because, well, it seems like we've been training models, and most people are complaining more about what they're doing than the actual training. So what is it that is at issue here, and what needs to be fixed?

Well, I think it comes down to even open weight models. We do not know what's inside these models. We do not know what they were trained on.

We don't know what they are set up to do. And a lot of people now are complaining about some of the risks associated with these models. Everything from the commercial models that are watermarking some of their messaging, and copy that's coming out of them, which, depending on if you've seen, I don't know if you've seen any of the watermarking coming out of it.

You can't tell that it's been watermarked, but the verbosity and the message is not quite as clear, because it basically has hidden code in it. If we can have watermarking, well, certainly we can have other things that the AI models have been trained to do. So for example, on code, the AI models could very possibly be trained on code that is known to have back doors in it.

We don't know any of this, because these models are basically binary. Even open weight models, you can add things to them, you can continue training them, fine-tune them, et cetera. But at the end of the day, what went into that model and training of that model is completely opaque.

We don't know what went into it. And this is what made open source so fantastic to begin with, was the fact that you can see the source. You know what that program is written to do.

You know what it is about. And in AI, we don't have that right now. It's still very opaque.

So the idea of OpenWaldo is to create an environment of open source data. Data that is distributable, it is licensed, there is accountability in terms of who is bringing that data in and who is maintaining that data, and then people can go and build models off of that data. With OpenWaldo, you can also do an AI bill of materials.

So again, you know everything that went into that model. Organizations that are building AI models and releasing their weights could be leveraging this in terms of augmenting their models, finding additional training data, as well as people who, and companies who wish to build very small, kind of targeted and focused models to do very specific things. OpenWaldo's a very good solution for all of that.

Can we have full transparency on everything, or is this more like the most common things are going to have some standardization, but there'll always be the 20% to 30% that are somewhat unique for whatever purpose that model was trained for? That's a great question. It depends on who's building the model.

You can build a model that has a full BOM, a full bill of materials, so you can have absolute transparency on every bit of data that went into it. Most people are probably not going to do that, and it'll probably be much more like how open source in general works with software, where you can have an open source project, let's use Kubernetes as an example, that a lot of organizations contribute to, and then you have commercial entities that go and build their proprietary software, or professional versions of that software based on that open source repository. This can be very similar.

You can have organizations that are building off of this data, training off of this data, and maybe they are adding some of their own commercial capabilities on top of that, to either fine tune it or to make that model perform in a particular way that is necessary for a particular need. Or you can have organizations that are building their complete model from scratch using this, and then when you're building the entire thing from scratch, you can actually manage and see the entire transparency of everything that went into that model. Are we, in a lot of ways, reinventing the wheel over and over again every time somebody trains an AI model because, well, we're training it with a lot of the same data?

So is this also maybe an effort to kind of reduce the total cost of training these AI models? Because what's the point in all of us going and coming up with the same description for, to your point, Kubernetes over and over again? Exactly.

Yeah, 100% agree. Why wasn't this started before? Because, as you and I sit here and discuss it, it kind of starts to come under the heading of common sense.

But what was the hurdle here? Or what's the thing that we all overlooked? That's a great question.

I think along with a lot of things that we've seen, in terms of software and industry, a lot of times you'll see kind of the forerunners building everything because they have to. And many times, open source kind of comes after the fact, when we start recognizing that we're doing a whole bunch of redundant work between these different AI labs, between various organizations, and there could be a community that can come together to help and build that foundation. And again, I kind of talk about OpenWaldo as being an open source project, focusing around the input data for training.

But it also has tooling around it, so it's very easy to go build models. And as an open source project, there's a community that's now building around this as well. So people, they want to understand, they want to learn more about building models, understanding what training means, understanding how to do this.

Maybe they've already purchased some GPUs, maybe they want to rent some GPUs in the cloud, and they actually want to try building a model for themselves. OpenWaldo gives you this ability to not only have the source input going into the model, the tooling around the model, but also the community that you can now go and ask questions with and talk and collaborate with. As you kind of think this through for a second, what's the process going to be?

Is there going to be some sort of governing board, and I submit something to them, and somebody reviews it? Or what's the mechanics of this going to look like? So it all depends in terms of, I think, what level of scale we end up achieving through the community.

If there's a small number of people that are contributing and maintaining data within the corpus, it'll probably be fairly easy and reasonable to maintain. If this ends up being a gigantic open source project where you can have, let's just use the Linux kernel now as an example, where you have sub-maintainers of different trees and different subsystems within the kernel. This can be very similar, where you have maintainers that are maintaining various portions of the corpus.

And the technical perspective of how it is done is actually fairly interesting as well, because it uses Git for the index, but the actual data blobs are gigantic, so it doesn't make sense to host those in Git. You can't do that. So it uses a lookaside cache, which is then hosted in S3-type buckets.

And you put the index for what is the data, what is the corpus, how is it organized within the corpus, and then it actually points and redirects to a federated object store that the tooling will automatically pull from and then help you build your models from. You can filter based on licensing, based on if there are email addresses in the corpus. So for example, if we train on a mailing list, for example, or we have some data that's available in a mailing list, you may find that there's things like people's names in there, email addresses.

GDPR, among other regulations, would consider that controlled. And so you may or may not choose to train on that data. Likewise, you may choose not to train on copy-left data or other forms of other licensed data that you may not want in your model.

And where is this going to be hosted? Who's going to provide the resources for this? Because I assume it might be sizable, maybe I'm wrong.

So the way that it works and the way that it operates today is the index is in Git. So GitHub hosts the main index, and this main index can get to ridiculously large amounts of size before it even puts any bit of a stress on Git. The object store is really where the main bits of the data is going to reside, and the maintainers, the people that are contributing the data, are going to be the same people that are hosting that data in their own buckets.

We are cryptographically validating that data to make sure that it hasn't been modified and it hasn't been changed from what the index points to. But at the same token, people can now host their own data. We will have CI processes that will go through and make sure all that data is reachable and available.

And if it's not, it can easily flag the maintainer. And all of the maintainers are using Developer Certificate of Origins, DCOs, which is the same signing system that most open source projects use for accountability. Does this have uses beyond training?

And I'm asking the question because a lot of times you hear people talking about agentic workflows, and they don't want to have to stuff everything in memory. So if you have an index out there, can I maybe point the agent at the index a little bit and reduce my load on the memory consumption of the AI agent? Does that make any sense?

It does, but the shards of data are giant binary blobs, Parquet files, and those Parquet files are fairly large. So you'd have to then go and you can look through the index, find things that match via a agent, and then pull that data. But the data it's going to be pulling is generally going to be fairly large and probably too much, probably way too much, that you would actually fit into a context window for a model to be able to process.

" Well, I'm really worried about the data that is in some of these freely available open models. We don't know what it's being trained on. And just like I opened with, we're seeing now that a lot of the data coming out of these could be watermarked in such a way that the words and the letters are in particular orders, that there's almost an embedded code in it, some sort of embedded fingerprint on where did it come from, that people wouldn't even recognize when reading it.

My mind naturally goes to, well, one of the most common use cases for AI today is to build software, to build code, and if we are able to watermark text to the point where we don't even recognize that there's hidden messages or there's fingerprints inside the text, well, it's completely reasonable to assume that, well, there could also be backdoors that this model has been trained on. So most people that are doing, whether that's vibe coding, whether that's expert-assisted co-programming with the AI, there's so much code coming out of the AI, it's almost impossible for an engineer to go through and validate line by line. It's very possible that we're going to start seeing backdoors, and we're going to start to see other exploits going through these models, where if we had true transparency of these models, this would be a non-issue.

Will the large companies that have made a fortune on this stuff, the frontier model people, get behind this effort, or are they going to view it as something that maybe helps too much to level the playing field? I think that this is something that doesn't necessarily level the playing field, but it increases, it raises everybody up simultaneously. If the big frontier models wish to leverage any of this data as part of their training, I think that's fantastic.

I think some of the newer AI labs that we're starting to see come out and starting to make a bigger name for themselves building open models, I think they can leverage this as well. And I think companies and organizations that want to, and have the resources to build their own models from scratch and then augment those models with some of their own custom training data, I think will also find this project extremely valuable. You mentioned smaller models earlier, and I guess my question would be, a lot of organizations kind of like the idea of smaller models.

They're going to be easier to manage, less costly to produce, but I think they don't have the resources and the skills to get started. So is this whole effort some way to jumpstart all that so that we can build smaller models and we're not making all these API token calls for everything? So this is where I think is a really important point.

It's super critical that these models are built from a point of knowledge, and it does take a certain level of expertise to build these models properly. We're seeing this quite a bit with the AI labs, and not just the major ones. We're seeing these with many of the up-and-coming AI labs.

They're doing a ton of research right now and development on a lot of these, being able to create something like a sparse model with mixture of experts. And you're starting to see the ability to route to these different experts, and kind of activate and keep activated a certain portion of your model and keep that in main memory, for example. And there are some projects out there that are now trying to allow you to run very large models while maintaining just the active portions in memory and the rest of it on disk.

I think we're going to see more and more capabilities around this, but I think we're also seeing, especially with mixture of experts, we're seeing that smaller, more focused models very well may be much more within reach than larger models. And an organization may say, well, instead of having one model that's general purpose and is going to require this huge amount of infrastructure in order to run, I can get away, just like you said, with smaller models that are much more tuned towards specific things. But the one thing I do want to still push back on is this is still very new, and it requires quite a lot of knowledge to successfully build a model, as well as a lot of resources to successfully build a model.

One of the points of OpenWaldo is to build a community around this, such that we can leverage off of each other's experiences and learn from each other. And then together, start pulling together some of the best practices in such a way that we're going to lower that barrier, lower the bar, so people can start building models from this community. Do you think given all that, that maybe we're a little over-provisioned on the infrastructure side because we're building these massive general-purpose AI models when maybe we just need more specialized AI models?

So, I think from my perspective, the major frontier labs have done a wonderful job kind of demonstrating, here's the easy button for getting involved in AI. Go get an account with some of the big AI labs, and here's an API that you can just go hit and run agents against, and they're doing all of the back-end infrastructure. I think over time, we're going to end up seeing much more of that infrastructure needing to move on premise.

And we're going to start needing to see much more organizations controlling their own AI destiny and their own AI infrastructure. And we're already starting to see that. I talk quite frequently with a number of large corporations that can't just go and use that easy button to start leveraging AI.

And it may seem to an outsider that they're just behind the times, they're not picking up new technologies, but actually, many of these large organizations run in highly, strictly controlled environments where they have regulations and policies that they have to manage with regards to their data, who has access to the data, and is that data leaking. And as a result of it, they can't just go leverage these APIs. They can't just go use an easy button.

They actually have to get to the point where they are controlling it and managing it themselves like they do for the rest of their infrastructure. How did anybody get involved with this project? Where are all the cool kids hanging out?

org, and in there you will see a link at the top for community, and there's a couple different ways to get involved. The easiest and lowest hanging fruit way to get involved is just to join into the Slack, and just partake in some of the conversations. You'll see people talking about building models, some questions and some other people's experiences about building models, and some people starting to talk about, okay, let's start integrating the OpenWaldo corpus of data into these models, and how do we start providing data back into the corpus?

There's one area that I think will end up getting utilized with this, which is actually a fairly interesting aspect, which is, in marketing, people have for years been thinking about SEO, search engine optimization, and how do you properly rank your website and your content such that a search engine properly gives you ranking when somebody does a search on something. Right now, marketing teams are also doing this for AI. How do they ensure that their data is getting picked up by AI bots by AI crawlers that are going through the web and then pulling data into the models that they can choose.

Now, there's a whole slew of reasons why crawling is probably not the best way of ingesting data, because you don't really have authority to use that data necessarily. It can get into a very sticky area like we've already seen with copyright and other aspects. But what this actually now gives the companies the ability to push their own content into the corpus of knowledge for AI training.

So for example, at CIQ, I want to ensure that our company and our products and our services are always being ranked properly with as most current data that we can possibly produce into this corpus of data. So I want to make sure that all of our public documentation and everything that we could be providing to an AI is included. OpenWaldo gives me a very easy way of doing that by allowing me as a company to push data into it about my company and about my products.

And I think if I were to look into a crystal ball, I think it's just a matter of time before companies start recognizing that a corpus of data like OpenWaldo is a really great way to ensure that their company's products and capabilities are being indexed by various AIs. And as OpenWaldo grows, I do believe that everybody who's making AI models will start leveraging this, so it will naturally become incorporated into these larger models. All right, folks.

Well, you heard it here. Hey, there's a new way to think about training AI models, and the good news is it's a lot more accessible to the mere mortals out there than somebody who has trillions of dollars of financing. Hey, Gregory, thanks for being on the show.

Thank you, Mike. It's a pleasure. All right.

AI Leadership Insight series. You can find this episode and others on our website. We invite you to check all those out.

Until then, we'll see you next time.