AI Coding’s Real Bottleneck Isn’t Code Review

Hey guys, thanks for the throw. ai, and we're having a little chat about what's going on in our software engineering workflows these days, because we have a new bottleneck. It's called code review, but maybe there's a different way of thinking about this thing.

Adir, welcome to the show. Thank you, Mike. It's great to be here.

All right. So everybody's kind of banging the drum about code review and what we need to do about it, and this is going to be the thing we need to fix because we have AI coding agents that are generating too much code and we can't review it all. And maybe theoretically we're supposed to throw AI agents at this process, but I hear that you maybe don't necessarily agree with that conclusion.

So what's your thought on how should we approach this whole thing? Yeah. So I think the way we look at this, and maybe some context about us, what we do is as an organization, we have this ability to connect to existing code bases, actually model them, and then enable non-technical individual, like your PMs, like designers, actually ship code within the capacity of the organization itself.

And recently we just ran a research report we call The Bottleneck Moved. And with one month, we talked with over 131 software organizations at the end of the day. And we kind of tried to analyze what exactly are their pain points.

And I think, yes, the main pain point, and that's a common denominator for everyone, is the fact that coding agents are generating a lot of code, but it doesn't see the light of day or production for that matter. Meaning companies are not moving faster. And the bottleneck that we've seen is not necessarily just in the coding review itself, it's the actual process.

Who gets to build what? How do you enable different team members within your organization to actually deliver on the code itself? And how can you build this relationship between the PMs, the designers, to the engineers, and actually getting the right code base itself.

So that's the real bottleneck. The bottleneck that moved is okay, the bottleneck actually sits down with the product teams, with the designer team, and how they can actually ship code as well inside the company. So you're kind of saying that the review process is only one layer of this iceberg that we need to kind of resolve.

But are we on the cusp, therefore, of having to re-engineer our entire software engineering workflows and what might that look like? Well, first of all, I think it's a must, but let's give a quick review of what happened in the last 18 months, right? It's actually quite interesting.

I thought about it the other day. So we start with coding agents, and companies say, "Well, that's great. " And all of a sudden you say, "Well, maybe it's not enough.

" And then they said, "No, harness is not enough. " What we're seeing and what I'm saying is like we're trying to throw tools at the problem, but not actually changing the old processes that we have. You cannot throw a tool over tool on tool and not change the process.

So yes, 100%, Mike, I think we're not on the cusp. We're leading the revolution of how companies are actually building a new process to actually create an impact for the organization, not just for a single engineer that can move faster. One of the things that I see people trying to do is they're creating AI skills, and then they're trying to create loops, and these things augment or provide context to AI agents that are invoking LLMs, and people sometimes refer to that as the new generation of software factories.

" And in fact, some of the things that you're putting in the AI skills are conflicting with what's in the LLM. What are we supposed to do about all this? So I think, well, I think the LLM people, let's say the labs, right, they want you to spend tokens.

" The basic question I could ask those people is, if this is so great and I can build a website with plot code today, why is Lovable still gaining users, right? Why people are using Factory or buying Dev and why people are buying Autonomy through their organizations to actually execute on those kind of things. And what I would say is like, yes, LLM foundation models are getting better.

They can move us faster. They're not enough. You have to have an application layer on top of this, an application layer that can actually build bespoke context for your organization and understand the skills and the tools your organization need, right?

So those are the two layers. Yes, LLM can maybe execute the job better, but they still need a lot of context from the organization itself, and that's where the rubber meets the road. The one question I would ask every person who listens to this is, your process right now can actually generate pull requests that are being accepted in high approval rates.

If they're not, you need to be building more skills. You need to be building more tools in your flow. As I kind of think this through for a minute, to your point, we're generating more code than ever, but we're not deploying more applications faster.

Well, we are probably on a certain level, but it may be of uncertain quality because we maybe skipped a couple of steps here and there to make things happen and that happens. But I guess my question is, when do you think we'll get to the point where we are actually deploying applications exponentially faster and there's more software being created? " I always answer to my kids, "We'll get there when we get there," right?

It's very annoying. You should try it. But I think that the key part, to answer your question, companies who, we talk with many companies, right?

Autonomy has over 200 organizations today. We do a lot of those conversations. And engineers are fast, 20%, 30% faster.

Companies are not, as to the point you mentioned. The place when we can change this paradigm is when we change the process, when we start splitting the organization into three main parts, right? Who's in charge on the architecture, who's in charge on the system, the scalability, the stability of the product, and who's in charge on the application layer.

And if the companies will start looking at this with those three parts and then split the job between people, right? So PMs all of a sudden can take control of the application layer. You free up a lot of engineering power to actually work faster, right?

But they can take intent and put it into production without the need of engineer. We always say in Autonomy, a PM comes in, they have a hypothesis on Monday, it's in production by Friday. There is no reason it won't be like that, right?

And then engineers can actually focus on the architecture side of things and utilize them and make sure that they can build the next frontier of the company. So that's how we split the world, and I think we're almost there. At least with some of the platforms that are out there today.

Not just withdraw LLM usage. I was having this chat with some people, and they were telling me that we're actually getting close to the point where changes are being made through applications faster than the organization can absorb it because the organization has to adjust to whatever new capability. So are we close to the point now where maybe we can't go much faster simply because, well, the organization can't keep up?

I think it's a great question. First of all, it's relative, right? It's faster than what they had before, but it's still not fast enough in terms of measuring the ROI against this.

That's the first part. The second part, and that's what we talk about with many of our companies and many of the organizations, and specifically in this report, yes, they have a true enablement problem. They're releasing more features, but the sales team can't sell it.

They can't grasp what changed within the platform itself, right? So the problems that you need to resolve all of a sudden is an enablement problem of like, how do I tell Mike, the AE, what was changed in the last two days? It's quite a lot.

So a lot of those companies, what we suggest and what we do also as a product out of the gate, but also what we suggest to those, a lot of those organization, is every PR that you have, every merge that you have, build a skill to build a video and enable the video. Send this to the AE directly and say, "Hey, this is what changed today. " And all of a sudden you get this end-to-end loop that can enable your ecosystem.

Speaking of squabbling, one of the things that I think I've seen is that we are starting to deploy more AI agents, and each of these AI agents has been assigned a task, but they have personalities. And Frankly, they sometimes conflict with one another. They get jealous of one another.

So how do I manage these things that have quasi-human attributes? Am I going to wind up managing small armies of mini people that we're calling agents? I think Barack Obama said that every big problem you have, make sure that you can institutionalize it, right?

So I think that's the gap. You build all those agents, but it's not going to be free ions within the universe. You must institutionalize it.

You must build a process that you will work alongside, and each agent will have their guardrails between the agent itself, the task, and the guardrails that they can actually execute on. So a lot of those organizations, what we suggest to all those companies is, yes, you build a context layer and you think you're great, that's not enough. You need to actually build R and Rs, rules and responsibilities for each one of those agents and explain to them what they need to do, and then you will see a work stream that actually works better.

" Well, first of all, I think the first point is they just build agents, and they think it's going to work. " You can't hire yourself through a problem. It's not going to be helpful.

You just hire more people on the same problem without changing processes. So I think people, they want to be part of the hype. " And then they look at the bill, first of all, and they get shocked, but that's a different problem.

" Because you need to restructure the way you work and plan ahead. So that's the main problem, non-restructuring and thinking, I'm going to throw an agent, I'm going to throw a cloud code, Codex, what have you, and it's going to resolve my R&D problem. It's not going to happen.

We see it very clearly. Where are we going to be a year from now, do you think, as you look at all this? I feel like we're on some sort of journey here.

And a year from now, though, are we going to recognize the way we build software anymore? Is it going to be fundamentally different? Just how big a change are we looking at?

I think the problem is in your question, Mike. Are we building the software? So I think that's the fundamental change that's going to happen.

Everyone probably know or heard of Instinct, the assistant, the application assistant. And the one thing that Instinct or Muse by Meta are very interesting about introducing to the world is always-on agents. The ability for agents not to work based on triggers, but work behind the scenes and know exactly where they need to be called without the trigger.

And I think that's where we're going and where the world is going, this ability to actually have always-on agents on your software being deployed across the board, be understanding the different metrics of your company, and then being awakened to build part of your software in order to resolve problems or to achieve key metrics that you have as a company. So the autonomous part of it, that's where I see the world going in the next year, year and a half, for sure. What does the future of the IT team look like?

And I ask this question because, if you look, we've had administrators that are mere mortals, and they have graphical tools for managing infrastructure, virtual machines, whatever it may be. And we have DevOps engineers, and they are using scripts, or have been up until recently, to manage IT infrastructure at scale and create these pipelines and workflows. And then we've saw the emergence of so-called platform engineers who are maybe standardizing DevOps workflows to manage things at scale.

What does that IT organization look like going forward? Because it sure feels like to me that there's a lot of hierarchy and a lot of sometimes what I call snobbery between various teams. So, what does this look like going forward?

By the way, I think specifically for DevOps teams, they're going to have a very exciting work because the companies will expect and would want their employees to build more applications, but those applications have to live within the organizational capacities. And when we talk with DevOps and with DevOps teams, their problem is how do I build this guardrail, this infra inside my organization that Joe from finance can build their own application instead of go ahead and buying SaaS? And it's a very exciting place to be in from a DevOps capacity.

But from the other roles itself, those roles are going to take the front stage. Platform engineers are going to take the front stage in terms of what's being autonomous within the organization, what's being automated. DevOps will be in a place that, okay, what's actually touching my infra?

What can actually hurt the organization itself? So those people, in my perspective, are actually going to take the front row in terms of how this entire orchestration works end to end, entire automation works end to end. The drivers are going to be the business people that will just define the KPIs, and everything will work streamlined on that.

So in that model, is everybody essentially now an application developer because they'll use AI to express some sort of intent that then the DevOps team will shape into something that runs at scale with a little help from AI agents? Or how does this play? 100%.

The small caveat I would say, the DevOps team will build an infra that people can run on whenever they want, and everyone in the organization is an application builder for that matter. It could be an application builder internal use or external use. We talk with many companies today that are decided to forgo their CRMs.

They're building a CRM inside the organization within the boundaries of the company itself and saving a lot of money on Salesforce or on HubSpot. But yes, that's the reality and that's how it's going to work. So what is your best advice to folks then, as you look at all of this and they're all on different journeys and different levels of expertise, but I think some of them look at all this and they get paralyzed like the deer in the headlights, and they don't know what to do next.

Or we talk about the people that are getting paralyzed. Let's start from the end. I think the expectation that AI is magic and will work out of the box for your organization, and if it doesn't, you fail the experiment, is wrong.

You need to deploy AI within the organization. It's a learning system. It's a learning platform across the board.

Once you give more feedback, it gets better, and that's how you essentially build the harness and the learning mechanism within the organization. So that's the first thing out of the gate. If you want to implement AI within your org, make sure that you will go in a journey of at least two quarters until it gets in a place that it's giving you good value.

Second thing, I think from a paralyzed perspective, define small task. When we talk with companies, sometimes they don't want to go all the way to pull requests from PMs. They want to define a smaller task.

" It's a small thing. It will go a long way. It will help you move faster.

It will reduce the problems that we have in the review of the code itself. So it's a huge impact, and it's not mission critical yet. So define small tasks that you can start with within your organization, start training the model, start working around those kind of things, measure the KPIs, and then go deeper once you build trust with it.

All right, folks. Well, you heard it here. No one knows exactly where we're going to wind up with all this, but we do know that whatever we're doing now won't be the same soon, and I think the only wrong answer at this point is to do nothing.

Hey, Nadir, thanks for being on the show. Thank you, Mike. It was great being here.

All right, and back to you guys in the studio.