BigData Boutique Launches WarpSearch for the Agentic Era
Why WarpSearch rebuilds search now
Syn-Hershko believes the next wave after the AI boom will be efficiency. AI agents now hit databases and search engines harder than ever. Yet teams wrote many of today’s search technologies in Java for the needs of 20 years ago. So his team rebuilt search natively in C++, close to the CPU and cloud native from the start.A drop-in replacement for Elasticsearch
WarpSearch does not ask teams to start over. Instead, it supports the Elasticsearch APIs as a drop-in replacement for aging Elasticsearch and Solr clusters. It covers use cases from internal knowledge bases and e-commerce to log analytics and SIEM. According to Syn-Hershko, it can run 10 times faster, sometimes 50 times faster, at a fraction of the cost.WarpSearch is agent-first by design
Getting started is agent-first. Developers can point an AI agent at the WarpSearch MCP server, and the agent registers an account and indexes the data. WarpSearch supports keyword, vector, geospatial and graph search. It also offers hybrid search, which blends keywords with semantic meaning so agents find relevant results whatever terms they use. WarpSearch stores and searches data on S3 with caching layers, so it scales without dedicated clusters. As a result, a small data set can run for about $10 a month. Elasticsearch and Solr will stay around for a long time, Syn-Hershko says, but search needs to change for the next decade. Learn more at warpsearch.io.Transcript
Hello, everyone. Welcome back here to Techstrong TV. Continuing our programming for today, my next guest, he's been on before, you may know him, it's Itamar Sinhersky.
Itamar is the founder of Bigdata Boutique. And Itamar, first of all, welcome back to Techstrong TV. It's great to have you.
I hope all is good. Yeah. Thank you for having me.
So Itamar, you've been on a number of times, but I don't know if we've emphasized the Bigdata Boutique name. Yeah. Tell people a little bit about that.
Yeah. So I've been a consultant. I'm a tech guy, right?
Been a consultant for search systems, data technologies, text search, and so on for so many years. I think almost 20 years at this point. And we've been doing this for a long while, and we've been consulting for a lot of companies.
The team has grown over time, and then we started taking on some projects in adjacent areas. So if you do text search, you still need to do some big data, and some analytics, and so on, and clouds. And Bigdata Boutique was founded from that premise, effectively starting as a single-person Elasticsearch consultancy, so search engine, internal systems consultancy, and then from there to analytics systems, big data technologies, clouds, and so on.
And that's Bigdata Boutique. Absolutely. Very quick.
A website for people to go check out Bigdata Boutique? com, and that is- That's good. We call that a descriptive domain name in the business, right?
Yeah. No guessing. That's the name of the business.
Go check it out. So Itamar, you're that classic tech inventor, tinkerer, always trying to look at things and find better ways of doing things. Your latest creation is something called WarpSearch.
Yeah. First of all, what led you to do this? What is it about?
Yeah. So we've been doing consulting on, I mentioned, search engines. Effectively, you have an internal knowledge base, you have a company brain, now they call it, you have an e-commerce website, a CMS, and what have you, and you build this kind of search on top of your data.
And we've been doing this with customers. We have these other things that we do to optimize Elasticsearch, optimize OpenSearch, Solr, all of those search engine technologies. And then, AI and everything that's going on, basically, I think once the AI revolution will be done, and we pretty much understand what's going on, I think the next revolution is going to be efficiency.
Effectively, agents are hitting your database, hitting your search engine. Something needs to change in that respect. And we basically realized a lot of those technologies that we use now for search engines, for databases, are built in higher-level languages and are built in a way that was great for, I don't know, 20 years ago.
And it's probably not well-suited for today. And also, you don't need to write Java anymore, those technologies written in Java, because effectively, software is meaningless. It's very easy to write software at this point.
So why don't we just go and do everything that we know how to do, how to build a search engine correctly, how to do cloud native, and so on, just do it in C++? So we are basically quickly building something that is entirely native, entirely closer to the CPU, doing everything correctly, is cloud native, and we have a couple of additional innovations up on our sleeve, and we'll probably talk about that later down the year. And effectively reimagining how search needs to be done for the agentic era.
I love it. Now, David and Goliath stories. Elasticsearch is well-anchored in a lot of people's infrastructure.
But I've seen this play out before. Just because people use it doesn't mean they love it. And they may love the functionality without loving the application, without loving the people behind it.
But what made you say, "Okay, I think we could build a better mousetrap here. People want something better"? Yeah.
So our approach is not to build something that is completely different. And also, we're not going to replace Elasticsearch entirely. So the whole idea is to be able to give you a drop-in replacement for aging Elasticsearch clusters or Solr clusters, which is yet another aging technology.
So we basically support those APIs. Effectively, any search use case. So Elasticsearch is being used for many other use cases as well.
But any search use case, anything from company brand, all of those that I mentioned, e-commerce, log analytics, SIEM even, we are able to support those use cases and just support them with the Elasticsearch APIs and do it so much more effectively and efficiently. So we're talking like 10 times faster, even 50 times faster in some cases, 10 times cheaper. So it just works.
Excellent. Let's talk about user experience. How do people get this?
How do they get it up and running? What kind of support? All of that.
Yeah. io, and as the name implies, warp speed, and so on. And it's easy as that.
Basically, we have an MCP server that you can point your agent at it without having an account. It will know how to register an account for you. You'll get an email.
You approve the registration. So it's basically agent first. We are building it in such a way that agents will basically know what to do, without knowing WarpSearch at all.
So WarpSearch kind of guides them on how to do it. So effectively, all you need to do is to go to code, write, "Index this knowledge base for me in WarpSearch, create a searchable collection for me in WarpSearch," and it's going to get it done for you. As easy as that.
No strings. That's really it. Agent first, right?
Yeah. This is the new world we're living in, huh? It is.
Crazy. What comes next? Let's keep that a secret.
We have a couple of takes. So we've been doing search for, I've been doing this for 20 years, the team with me as well for many, many years. We know all of those things that hurt people when they do search.
So when you build a search engine for your e-commerce store, when you build a search engine for your catalog, for your company brand, and so on. Let me have some surprises for you later down the year. Let me just say, for now, we support keyword search, vector search, geospatial search, and graph search.
All of that is supported out of the box in Warp. And we'll have a couple more things later down the road. What about scalability?
Infinite. So it's cloud first, built for the cloud. S3 is pretty well-known, probably for everybody chiming in here, and effectively, everything is being written to S3.
It is searched on S3, obviously, with warm layers and caching and so on. That's how we get to those speeds, and also how we can allow for those very low costs. And effectively, it's infinitely scalable.
We played with Elastic here at Techstrong. com, for instance. com probably has more than 50,000 articles over the years.
Some are older, so we don't have them, they're archived or whatever. But more than 50,000 articles. We want a better search experience for people.
You want to come here, you want to look up, I don't know, CI/CD pipeline, some bill of materials, SBOM or something. Mm-hmm. Is this set up to handle something like that?
Absolutely. So 50,000 articles is cheap, right? Even for Elasticsearch.
You are probably paying, I don't know, $100, $300 per month for that small cluster, but just because you need a cluster. So one problem that we solve is the cost element. You don't need to pay that much just because you need this kind of small server to contain your data.
So with WarpSearch, we just do $10 per month for this kind of- Really ... small data set. So it's like- And what about, though, I want to be able to search it by speaker, by author, by date, stuff like this.
Yeah. So again, that's the easy stuff. That's just predicates and maybe- Yeah ...
they call it facets. But to your question, the whole point of the new modern search engines is that we need to be able to support anything that the agent is going to throw at us. So you are going to search for with one keyword, a couple of phrase words.
Agents are going to search for you don't even know what. They just come up with the words to search. And that's where semantic search is interesting, right?
So you can search for CI/CD, you can search for just CI, maybe some product name, maybe something that CI/CD does, right? So compile, test, and so on. That's what the semantic part of it catches.
So if you search for keywords, that's going to work well, but you're going to get just the documents with the keywords you searched for. Semantic, or using vectors, is going to get you the semantic meaning of that. So it doesn't matter what you search, you're going to get something that is relevant.
And hybrid search is kind of meshing the two together to get you the best result that there is based on the keywords that you actually use. So that's the new age search. Everybody's doing it now, or everybody is going to do that.
And the idea is to make it very, very easy. Like I said, tell your agent, "This is my data. " And it just does it, and for $10 a month.
I love it. We're going to have to check it out here at Techstrong, and then we'll come back and we'll talk more about it. Please do.
Poke holes as much as you want. Well, no, hopefully, it works because we've been looking for a solution for a very long time. There are, I don't have to tell you, there's some very expensive kind of search, boutique search kind of solutions that are out there, and a lot of them use Elastic under the covers anyway.
Yep. And, I'll talk to the engineers. We'll be in touch.
Yeah. I'll have my people call your people. Yeah, definitely.
So, some of our customers at the gate of boutique are the biggest, I won't say names, but are the biggest companies that you know as providing search under the hood. And under the hood, they use those technologies, right? Elasticsearch, and Solr, and others, and we help them support those clusters.
And that's where the realization came in from, right? So why not just do something that is more efficient and doesn't require those clusters to maintain? Now, obviously, Elasticsearch, Solr, are going to stay with us for a long while.
We still provide support for them from those many customers that we have, but something needs to change, right? If we look 5 years, 10 years ahead, I mean, even Musk is sending power plants to the moon, right? I mean, something- Right ...
is changing. So that's one of those changes that we think we need to start doing. I love it.
Nir Bar, I wish you lots of luck with WarpSearch. We'll be in touch. Keep doing what you're doing.
I love the innovations, always coming out with new stuff. Thank you so much, John. Best of luck.
Thank you. Thank you. Okay.
We're going to be right back here with more Techstrong TV in just a moment.