AI Observability Moves From Insight to Action
Software and infrastructure are becoming autonomous. That shift makes AI observability more important than ever. Agents need rich production context to write code, tune systems and recover from incidents.
groundcover CEO and Co-Founder Shahar Azulay examines that shift with Mike Vizard at KubeCon + CloudNativeCon in Salt Lake City.
Closing the Loop With Wand
Azulay sees observability as the grid agents rely on to make decisions. In software development, coding agents already pull production context to write and test better code.
groundcover acquired Wand to bring that idea to infrastructure. Wand dynamically tunes large Kubernetes environments for cost and reliability. It runs on the same eBPF data groundcover already collects. Azulay says customers can start optimizing with a click.
Agents Are Different Animals
Traditional traces covered a handful of microservice hops in a minute or two. An agent can run for hours and call thousands of tools. Its behavior is also non-deterministic. A single changed letter in a prompt can send it down a different path.
That calls for a different kind of APM. Teams need to see prompts and tool calls. They also need evals that help other agents judge whether a run was healthy.
Bring Your Own Cloud
Heavy sampling no longer works for agent workloads. Most of this telemetry will be read by other agents, not people. Renting storage back from a SaaS vendor at a markup stops making sense.
Azulay argues bring your own cloud offers a better balance. Customers keep data in their own cloud accounts while groundcover manages the platform. That approach is cheaper and more private.
eBPF and OpenTelemetry Together
Azulay calls eBPF a perfect complement to OpenTelemetry. eBPF delivers a strong baseline without code changes. Teams can then add OpenTelemetry where their business needs custom detail.
Looking to 2027, he expects more autonomous operations and fewer humans in the loop. He also sees observability merging with security and the software development life cycle. AI observability will sit at the center of that change.
Transcript
Hey everybody, welcome back to KubeCon. We're here talking about observability, Kubernetes, AI, everything else that everybody else is talking about with my good friend, Shahar. How are you doing, buddy?
Very good. All right. You guys were at the forefront of all this push around observability, eBPF, and we're seeing the adoption of workloads starting to scale to the point where everybody's getting stressed.
So at the same time, you recently bought a company that you guys are using to do some optimization of cloud consumption resources around Kubernetes. And then I look at AI workloads and all of this stuff that's happening on that side, and now AI agents. I feel like there's a primordial soup of stuff happening here, but what's your take on how all this comes together?
Well, the main thesis is that software and infrastructure is becoming autonomous. Every company in the world is trying to build some form of autonomous operations right now based on agents. In software development, it's already pretty clear.
Companies are writing code with agents. They want to start testing it and shipping it with agents. In infrastructure, it's becoming there.
You want to tune and optimize your infrastructure more dynamically, but also if there's an incident, you want to recover also from an infrastructure incident before an engineer wakes up. From our perspective, the thesis is the same. Observability is that grid on top of which telemetry these agents can make decisions and understand what's going on.
In software development, again, super clear already, everybody's writing code with context from Groundcover, Datadog, or like. You have to have some context for production on how your application is behaving so the agent can write and test code better. We've acquired Wand to take this thesis to the next level when it comes to infrastructure, being able to dynamically tune, optimize the infrastructure in a large Kubernetes environment for cost savings, but also for reliability, which is otherwise almost impossible manually- Yeah ...
without interruption. Almost sounds like you're closing the loop a little bit- Yeah ... because we observe is one thing, but to take action is another.
Right. And so now are we getting to the point where, I don't know, are there AI agents that are going to look at the observability data and then push through an action to optimize a Kubernetes cluster? Is that how it's going to play?
Yeah. Again, in the software side, I think it's easier to imagine. You will soon have in months from now an observability agent delegating tasks to a coding agent saying, "You know what?
I lack visibility," or, "The ability to observe a specific part of the stack. " Or if there's an incident and I reach a root cause analysis with my observability agent, why not delegate to a coding agent and fix it right now, as the humans collaborate with it or as the humans wake up. I think it's already happening in software.
In infrastructure, it's the same thing. When it comes to optimization of a large stack of infrastructure like Wanda's, in some cases, we see 80% waste in a big environment. It's almost impossible to do without an algorithm or something that helps you out.
But we will also want to remediate the infrastructure and make decisions and change configurations. So definitely, we see agents taking these decisions for us and participating in that cycle because I think the market is already pretty clear about it. People are trying to ship faster, ship better with the help of AI.
There's a lot of folks here walking around just trying to wrap their heads around the fundamentals. How do you observe an AI agent when the thing is ephemeral? I don't know when it's going to spin up, and I'm not even sure what it's doing at any given moment.
So that's why there's a new field called AI observability. It's literally new product. Even though the fundamentals of observability are still the same, infrastructure monitoring, log management, application performance monitoring, the core hasn't changed.
You want to see how applications interact and consume resources. But the reality is that they're different animals, these agents. They're non-deterministic.
They behave differently. If a trace before behaved like it had 15, 20 hops of microservices for a minute or two because it was a user-facing API, right now, these agents can go into a two, three-hour spring of calling thousands of tools because they're coding something. They're coding a feature.
So you have to treat it differently. It's a different type of APM. It's a different type of monitoring solution.
Groundcover is going deep into that part, leading the market, we think, with AI observability, but there's a lot of other products out there that are trying to support it. It's just different. You can't sample it the same way because of the non-deterministic nature of these agents, and you need more telemetry.
You need to see actual prompts. You need to see actual tool calls. You need a layer on top of it called evals that allow other AI agents to estimate whether this was a standard and healthy procedure or something that hallucinated along the way and reached a different conclusion.
So different tools, different layers on top of the same core data, but a completely different product at the end. I think a lot of people are also under the assumption that since the AI agent is essentially a piece of software, that it behaves in some rational way. But in reality, these things do things- Yeah ...
on their own volition in ways that you don't expect. So how challenging will it become to observe all this stuff, and do they require some level of constant supervision? I think that the main challenge is that observability in the past was built to reduce data to the amount that you actually need.
We sampled the traces heavily. We sampled logs. We reduced metric cardinality to focus on what human cognition made sense.
Specific alerts, specific dashboards that the engineering team would look into. When we look at AI observability and monitoring agents, since it's so non-deterministic, the reality is that sampling it through a very short lens is really hard. So you need to store more telemetry, and you need other agents to go through this telemetry.
So the fundamentals of how we store it, the fundamental physics of how we store that in SAS vendors like Datadog and New Relic don't make sense anymore because most of this telemetry will never be consumed by human. It's huge amounts of telemetry needed to be consumed by another agent that has to go through them. So the fundamentals of storing data somewhere else and renting it for a markup wouldn't make sense anymore.
That's why Groundcover went so deep into bring your own cloud. That's why we don't host our custom telemetry. That's why we're trying to not mark up data, so we can allow these new experiences to emerge in a more cost-effective way.
But that's the fundamental part of it, how you can allow more telemetry to describe what these agents are doing, basically. To your point about that, we were choking on the telemetry data we were collecting before agents and AI. Right.
They generate even more telemetry data- Yeah ... more so than just about anything else I've ever seen. So how are we going to collect that data in a way that makes sense and then analyze it?
Because if we try to push it all up in the cloud, we're going to run out of storage quickly. Right. I think eventually, the main idea of a third party storing it for me, marking it up and renting it back to me when I query it, it fails.
Because it was hard before in cloud environment where telemetry exploded, and I couldn't save all that. So there's value was stagnating when telemetry was exploding, but right now it doesn't make sense anymore at all even. Specifically when you want to build an AI layer on top of it that makes decisions with this telemetry, if some of it is missing, you're not going to get the same experience.
So the bring your own cloud is somewhere where the market is going to. It's the best of both worlds of SAS and on-prem. You host your own data and you pay to AWS, GCP, and Azure, but Groundcover manages it for you, so it's a fully managed experience.
But you buy the infrastructure directly. You buy it at the lowest price you can get. As an enterprise that is already buying infrastructure, it's your top bill.
So it's cheaper, it's more cost-effective, it's more private, which is also critical in AI, and you now can consume much more volume of cost-effective telemetry and build experiences on top of it. I think that's critical. We see it in a lot of different domains besides observability.
In data streaming and database arena, a lot of products are going there because customers want more private and more cost-effective telemetry. And Murphy's AI law would stipulate that if there is an incident with an AI agent, which is highly probable, I won't have the data stored to analyze for the telemetry because that will be the data I decided not to collect. That's true.
Is this how this works? That's true, yeah. Eventually, it's really hard because right now this data has already also been triggered by user input.
That is also completely non-deterministic and very wide. So a change of a letter can take the agent to a different path, which might be completely wrong compared to what you've tested before. And part of the power there is to make sure you can cover more of these cases, see more of these behaviors, sample it differently.
The classical terms of APM, assuming you have a customer-facing API that you can sample one out of every 10,000 and you'll be okay- It doesn't apply anymore. I think that's where the world is going. Not necessarily less telemetry, just the ability to store it more so agents can process it more.
And again, it breaks the fundamental understanding of how SaaS observability currently operates. Right. As you look forward on all this stuff a little bit, and I know you're here, you've got WAN.
Yeah. How does this all get integrated? What is the relationship between the observability platform and WAN, and how does that all come together?
We believe it's all part of the same story, because the same data is already there. GroundCover is basically building WAN on top of the exact same foundations. We have an eBPF sensor that basically, without code instrumentation, collects infrastructure monitoring and APM metrics.
So it allows the WAN decision algorithm to operate on the exact same metrics and data we've been collecting before for observability purposes. So now the observability platform is turned proactive, and using the same data exactly, can make decisions on how to optimize your Kubernetes environment. The next step will be to remediate an incident that happened.
But it's basically we see it as part of the same story, and there's other platforms here from great companies here in this conference that go only on the optimization and kind of management part without the observability part. To be honest, it's operating on the exact same telemetry. Right?
And I want to make different types of decisions from optimization to remediation on telemetry that I'm collecting also for different reasons. So it ties back really nicely, very natively to the stack that customers are already running. In fact, in our platform, it's literally a click of a button to start optimizing the platform because the data is already there.
Wouldn't that also play out for security? Because when I look over there, they're collecting massive- Yeah ... amounts of telemetry data too.
Creating a separate stack of repository for it usually, but can we bring all that together? We actually see that in AI. AI governance and for observability, and as AI security are basically starting to merge.
We see that with acquisitions like Palo Alto acquiring Chronosphere and other different acquisitions. So this field is definitely kind of getting closer and closer together. I think that in GroundCover's case, we're definitely thinking about that.
Some aspects like SIM, for example, are really close to log management. So these high telemetry domains are getting closer and closer with AI for sure. At least at this conference, when you say observability, everybody's mind immediately goes to OpenTelemetry.
Yep. And that's how they instrument things. Then we have eBPF.
What is the relationship between these things? Because I think a lot of folks are still unclear. Is one a replacement for the other?
A complement? How do I think about this? It's a perfect complement.
First of all, GroundCover is fully OpenTelemetry native. We store our data scheme as an OpenTelemetry. We ourselves digest OpenTelemetry from most of our customers.
eBPF allows you basically to kind of get the relief of doing OpenTelemetry instrumentation where you want it and when you want it, because it guarantees that layer of APM infrastructure monitoring without you doing anything. If you do want to instrument specific parts, and most of the GroundCover customer base does that and send OpenTelemetry into our platform, but you can pick and choose where to do it, where you care. Because eBPF provides that generic layer, right?
It doesn't fit everything in your book. You have a unique business. There's specific things you want to customize and measure.
Use OpenTelemetry for that. But right now, for a customer that doesn't use eBPF, it's kind of a zero or one situation. You either instrument everything or you don't.
Mm-hmm. And so eBPF, we see it kind of as a complement for OpenTelemetry to get into a very strong baseline, and then on top of which decide what you want to instrument. And it's easier for a lot of customers to go through a journey like that.
And I can be a little smarter about what data I'm collecting- Exactly ... through what mechanism. Well, let me ask you this.
How smart does smart get eventually? Can I expect to come to work someday and because of observability and a bunch of AI agents, there'll be a message saying, "Here are the three things that are likely to get you fired today. " Well, I think a lot of specific places, specifically in software development, we see engineers moving into more of a planner role, right?
I don't think software engineering is dead. I just think it's changing, and it's governing the work of a lot of agents. It's just not in the code anymore, right?
It's on a different layer. But it's still important. It's still governing what you want to build, how you want to build it, what's your quality measures, how you want to ship to your customers.
Maybe closer to what product is today. But I think some roles are just changing dramatically. Hopefully not firing all of us.
You talk to a lot of customers. " I think the value of AI is so powerful. The customers are being pulled into that in speed we haven't seen before.
Enterprises are usually very cautious and very late to market. I don't think it's happening with AI at the same rate. So a lot of them are taking fast decisions maybe that would be considered before non-conservative, right?
In financial instances and other places. I think it's fascinating. It just shows how powerful the pull from AI is.
I think it's the proof of where this is going. Before, we used to see these late adopters coming in two, three, four years after the head of the curves. I think they're coming in three or four months late.
So that's the most interesting thing we're noticing right now. Sounds like necessity is the mother of invention. Let me ask you this.
We see everybody has been collecting metrics for a while, and they're all familiar with logs. But traces was always something a little bit unique for the more advanced. But as I look at AI, do we need to get better at traces?
Because this is the thing we're going to use to investigate these incidents. Right. Of which it looks like there's going to be many.
I think APM or traces was kind of that cream on the top of observability. Most of the customers out there don't have it. Yeah.
And I think we have to face the reality this is a very mature product. It's been around from about 2017, give or take, in full GA with all the competitors in the market, and yet most of the market don't use it. Most of the market still relies on infrastructure monitoring and log management.
And the reason is, it's hard. It's hard to instrument. It's hard to instrument well, and it's really expensive.
And I think that eBPF allows us to get there really quickly and kind of solve the hard problem, and Bring Your Own Cloud allows to solve the cost issue. And as you say, AI observability is basically built on top of traces. It's a different type of APM.
So if you have to instrument everything to get there, that's going to be really painful, and I think customers are going to get into the point where they don't get there. It's too hard. AI is pushing code too fast that you will just not be able to see that.
I think eBPF is going to play a crucial role in how that type of APM is being perceived because it kind of runs at the speed of software being shipped out. So it maybe puts eBPF in an even stronger position than it was before. How smart do I have to be to use observability?
" I think we just talked about it today, actually. Continuous profiling is coming up as well for GroundCover. It's an example of a feature which is really interesting, but 1% of engineers even know how to use.
But agents are going to make amazing decisions based on that, this new telemetry data. So I think we're definitely moving into a point where better telemetry can not necessarily improve the average engineer more. And as you can say, a lot of them are not sure exactly how to act.
But as we become more and more autonomous, these agents are going to get exponentially better than us in making decisions based on that. And as we trust them more, we will be able to let them be more autonomous, and they will act better. 2x, right?
You don't know how to use them properly, but your agent is going to get 100x out of these features are going to be really good at reaching root cause analysis better, and that's the value you're going to feel. We are coming up on the end of the year. As you get your little crystal ball out there and you look into 2027, what do you expect to see?
I expect to see much more autonomy, a lot more autonomous operations happening, people adopting agents with less guardrails to do more things in production and shipping more code without humans in the loop. And I think the SDLC is going to change rapidly. We're going to see a merge between observability and the SDLC.
We're going to see merges between observability and security. It's going to be an interesting year. All right.
You heard it here, folks. Observability for all. It's just a question of how soon we're going to get there.
Hey, thanks for coming by. Thank you so much for having me. All right.
And we'll be back later. Starting tomorrow, I think, is a whole other series of shows, and we look forward to talking to you then.