AI Agents Ease Open Source Maintainer Burdens

Hey everybody, we're back at the Open Source Summit in Minneapolis, and we're having a little chat about what AI agents can do for open source maintainers, because, well, there's a lot going on, and it goes well beyond just writing a bunch of code. Madeline, welcome to the show. Thank you so much for having me.

So I think everybody's playing around with AI at this point. They've probably all used copilots, and they know how to generate some code. But there are more interesting things that can be done and assigned to AI agents, especially if you're maintaining a project.

So what are you seeing folks doing with these things that maybe they're just assigning a bunch of tasks nobody wanted to do in the first place? No, that's a great point. So definitely what we've seen is a lot of folks have been seeing, in open source projects, a dramatic increase in the number of contributions, the number of PRs being opened, and the number of issues.

And, the phrase that's been thrown a lot is there's this AI wave, right? This swarm of newthing from new contributors. And the bottleneck has really always been on the ability for maintainers to actually review and incorporate that code into projects.

And so the big thing that both myself, a maintainer of the Valkey project, have been working on is we're trying to figure out how to offload the work that maintainers don't really want to do, so they can spend more time focusing on the important parts of maintaining the project. So some specific examples. We've been using AI recently to better maintain some of the older versions of the Valkey project.

So Valkey does two releases every year. So we have usually a minor version in the spring and then a major version in the fall. And we maintain about five to six of those kind of going back in time, because we maintain versions for about three to five years.

So when a bug fix comes in on the latest version, we need someone to go and basically take that commit and apply it to the old versions to make sure that they're still being supported. So this counts for both security fixes as well as minor bugs. So sometimes those commits can be cleanly taken back, because the code's pretty similar.

But other times, the code's maybe changed significantly. Maybe we built a big new feature, it doesn't apply cleanly. So one of the things that we found AI is pretty good at is trying to kind of figure out how to take the code and apply it to those old versions.

Mm. So we used to have some tooling that would try to cherry-pick it, and if it was applied cleanly, we could then just use it and run the tests. But then we kind of had to engage a maintainer to actually deal with the times it didn't merge cleanly.

Mm. So now we have some AI agents that work on that. And that's one of the things we've been using recently.

I actually used it last night. We actually released a new version of Valkey today. Unintentionally.

It wasn't intended to be part of this conference, but that's kind of how it worked out. And we were using this to take the final set of commits from the development branch into the release branch so we could do the release this morning. Mm.

And so that agent's great at making sure that they all apply cleanly. It lets you know what stuff applies, what doesn't apply. And yeah, it provides that sort of helpful information.

How does that change the cognitive approach of a maintainer of a project? Because as I understand it, in time everybody will have a small army of AI agents that are being assigned various tasks. And I'm going to have to manage these AI agents, and a lot of these things will be happening in parallel.

How does it change the way you think about this stuff? So the way I've been thinking about it is the role of maintainer is really to help make sure that open source projects have a vision and that they're moving towards that vision. And increasingly over time, we kind of expect them to use more and more agents to sort of help kind of course-correct the ship so that the project is moving in that direction.

One of the other things that we've been doing a lot is we've been using agents to look for security and bugs. " And so if all of that mental burden is offloaded both to the automated agents and my own agents that I run on PRs, my focus really just becomes, hey, is this the right commit for the project? Is this the right feature we want to be building?

Is this the right architecture we want to be having? And maybe in five years, agents will be great at that too, but at least right now, humans are sort of the best individuals situated to really be the guiding principles for projects. When it comes to things like documentation, everybody's a critic.

But will, in the age of AI, the documentation get better? And will I be able to kind of understand what's happening in that software because some AI agent will organize the documentation in a way that makes it more readable? I would like to hope so.

I know the big trend in software development is moving away from a bunch of disparate projects to monorepos, where basically when an AI agent goes and updates code, it will also go update the documentation at the same time. Because you want all of those to be together. The agent can use the documentation to sort of help better infer what it should be doing inside the code as well.

And so I'm optimistic that agents do also collect context. They're reading documentation. So, if you're doing a good job maintaining the documentation, it will make it easier for you to actually write better code and vice versa.

Mm-hmm. So at some point, is there a symbiotic relationship here that starts to emerge because the AI agent that the maintainer is using is creating the documentation that's going to be read more frequently, at least, by another AI agent? That's an interesting discussion that we've been having.

Right now, my personal opinion is that the documentation that's good for AI agents tends to also be good for humans. So we're trying to build one cohesive set of documentation that's good for both. But there could be a world in which you might want to start writing documentation that is purpose-built for agents.

I know, I think Netlify, if you go to their main page, that you can see here's the view for humans and here's the view for AI. And the AI view is in markdown. It removes some of the HTML tags.

Just kind of strips up some maybe the unnecessary context. And we might be seeing some of that moving forward, but I think the jury's still out on what's the right approach for how to optimize that. There's more stress in being a maintainer of an open source project than many people care to admit, and it's a labor of love.

But if we have AI agents who are offloading more tasks, can we find our joy again? Will we be able to take all the things we don't like doing and hand that off to an AI agent so we can focus on the fun stuff? I like to think so.

I was mentioning earlier that I was helping doing a release this morning, and this is probably the most mechanized release we've ever done. I was talking about we were using the backcountry agents to pull the code in. We were using more AI to basically help make sure the code quality is there, the reviews are there.

And I've been enjoying having more time to write more code because agents have enabled me to do that stuff more quickly. So I am optimistic that at least for the near-term future, that I think we'll be able to build a lot more stuff as maintainers, and that's really what got me into the projects to start with. Mm-hmm.

I started contributing originally to Redis open source because I really liked helping people. " And I enjoyed solving their problem. And when you transition from a contributor who gets to primarily write code to a maintainer, you have the same type of constraints.

" And you have to be like, "Well, this isn't quite the right fix," and you still have to help them get there. But I've been enjoying maintaining the project more with the help of AI agents than I was not too long ago. I think earlier on in the AI wave, it was a little bit more painful because there was a lot of AI slop, there was a lot of pain.

But everyone adapts, and we've found new ways to find more joy in the process. How is the relationship between the maintainers and the contributors evolving? Because you do hear a lot more about the contributors are using AI to create code, and some of the maintainers are being overwhelmed by the volume of that code.

So will the AI agent kind of level that playing field a little bit? So over the last about four months, we've seen about a 30% increase in PRs to the Valkey project, but we've seen about a 500% increase in number of lines changed in those PRs. So yes, there are more people contributing, and the volume is not-- You can't really handle that volume, right?

Because the problem is nobody can review that code anymore, and you can't really offload that process of review to an agent unless you really can trust that it's going to do the right thing. And what I've seen is we've, in the Valkey project, gotten a lot more accustomed to just kind of saying no to things. " But if all you're doing is basically taking a vanilla agent like Claude Code, pointing at an issue, and going and solving it, we could do that too, and we could probably do it a little bit better and a little bit faster.

So I think the dynamics between contributors and maintainers is changing a little bit. Mm-hmm. One of the other things we used to do is we used to maintain a good first issue set for the project, and if someone was like, "Hey, I want to come contribute," we could point them to this list, and they would go pick it up.

And a lot of that is understand the mechanics of contributing, learn about the projects. And what we've noticed is a lot of people, we saw OpenClaw, random agents go and find them, go pick them up, and go open the PRs and submit them. And that's not really the point of those issues, right?

We don't really care that much that they're fixed. We care that someone's learning about the process. Mm-hmm.

So we've also become a little bit more protective of these issues. They're still issues, and they've changed a little bit. Recently within the project, we had Apple start contributing, who's a big tech company.

And they were able to ramp up much more quickly with their own agents. " And it was more hands-on with us to sort of help them learn about what needs to be done. And then the agents helped them actually understand the code base well.

So I think for people who know what's going on, so established senior engineering folks, they're able to get involved in open source projects much more easily with the help of AI than before. Do you think we'll also be leaning more on the AI to help with vulnerability remediation? And the question comes up because we've seen all these new AI tools are discovering more vulnerabilities than ever, and it's kind of overwhelming the maintainers, many of whom are not on a payroll to go fix these things.

" But will the AI agent enable the maintainer to kind of get to it faster because it'll analyze the vulnerability, create the patch, and maybe even automatically update the code base? Is that too much to ask? It's a lot to ask, but I think AI is going to help a little bit with that.

I mentioned before, we actually built some tooling to sort of help automate the process of looking for bugs inside the code base, and we actually found one CVE that's currently under embargo inside the Valkey project. And it's much better when we find the bug than when some security researcher finds it. And-We've been trying to use this tooling more readily across more individuals across the project.

This is one of the things that I appreciate, at least my employer, AWS, has been helping with. They've been like, "Hey, you guys can use burn as many tokens as you want," to basically help make sure the Valkey project is secure and safe. So I think there is some responsibility from some of these big tech companies that are around to sort of help provide some of this infrastructure.

And then, ideally, some of the tooling as well and some of the expertise to efficiently apply the agents to actually be able to find these bugs. And as I said, if the maintainers are able to find them first, it's a much better situation than some malicious actor finding them, or some random security researcher who might have different expectations of when it should be made available than you. Yeah.

There's also a lot more regulations in the world as it pertains to software development, and most of those were probably not written by people who have experience maintaining software, but the rules that they come up with are still the rules. In some odd way, is AI agents now going to allow us to actually comply with rules written by people that never really understood what the issues were in the first place, but maybe it will all come together in an interesting way? It's possible.

When ChatGPT first came out, my biggest concern is people were going to write a couple sentences for ChatGPT, say, write an email based on it, but you're going to send that email to someone. " And you add this whole level of abstraction that was unneeded in the first place. And I'm a little worried that's also going to happen with a lot of these regulations as well.

On one side, there's going to be people writing for humans without thinking about the fact on the other side is going to be AI that's actually going to be trying to reason through it. But in actuality, a lot of that stuff is also going to be AI-generated in the first place because we see AI being kind of used pretty much everywhere. So I like to believe that we will get better at coming up with more fixed standards, like more declarative, more deterministic ways of evaluating whether or not something is done or not, so then we don't need to rely on AI.

This is the paradox of people are using AI to say, "Take my four data points to create a presentation," and then I send it to somebody who then says, "Reduce this presentation to the four core data points," right? Yeah. So I think there's a place for AI in all of this, but I don't think we've quite figured it out yet.

It is. So what do you know now about working with AI agents that you kind of wish you knew a few months ago? In hindsight, looking at it going, "Geez, I wish I had thought about this a little bit more before I got started," because other folks are probably not as deeply into it as you are just yet.

So I'll give my sort of broad idea of how I've been thinking about AI. So I first became convinced AI was not just a hype basically when Kiro came out. So Kiro's big idea was, okay, if you basically tell the AI to take a moment and think through what you're about to do, it does a much better job about what's actually completing the task.

So Kiro came out, I guess, in about last June, and ever since then, I've been really trying to think through how do I get AI to very specific contextualized things. I've seen a lot of people give very open-ended prompts to AI and kind of hoping for the best, and it usually works the best when you really think of it as someone like one of the world's smartest college grads. They know all of the knowledge, but they don't really know how to apply it very effectively.

When I was talking before, we were looking how we found that CV inside Valkey. We gave it very specific prompts, which is basically like, "Hey, first, I'm going to work with you and we're going to write a test harness," and the whole purpose of that test harness is to look for crashes, look for memory corruptions, use address sanitizer, use existing tooling. " Same thing you do.

You want to give them very constrained tasks where they can still sort of use their non-deterministic thinking to figure out the best things. For the benefit of the audience that don't know what Kiro is, AWS tool? Yeah.

And it's spec driven, right? Yeah. So Kiro is basically an alternative to Claude Code.

I think it's what we call a harness for AI agents to basically make it so you can interactively talk with it. All right. Well, folks, you heard it here.

When it comes to AI agents, all kinds of things are possible. You need to just kind of jump in and experiment, and hopefully, this will lead to better experience for all the people who consume open source software. At least we hope that's how it's going to turn out.

Hey, thanks for coming by. Thank you so much for having me. All right.

We'll be back in a minute.