GenAI in Software Engineering with AWS’ Neha Goswami

This is Textron tv. Hey guys, thanks for the throw. We're here with Nha Kasami, who's director of engineering for Amazon Q for developer, and we're talking about a, the latest edition of it, and b, some new code transformation capabilities that they've been adding as well.

Neha, welcome to the show. Great, thank you. Happy to be here, Mike, and thanks for the opportunity.

Um, I, as you said, I am the director of engineering for Amazon Queue Developer, and, um, I'll be talking about, uh, some of the new agent capabilities that we recently launched. So yeah, go ahead, Level, set us a little bit for those who haven't been following everything, but, um, the original agents just came out not too long ago and they've been out there. What's been the reception like so far before we jump into the latest version?

Yeah, we've, uh, seen a lot of excitement over all, uh, q develop or just around the overall q developer product, which includes the agent as well. Um, specifically for Q developer are coding, uh, acceptance rates are the highest, uh, recorded in the industry, and we've seen, uh, as high as 50% acceptance rate from some of our customers, which has been great. Uh, very delightful for us to see.

Uh, on the agents part, uh, specifically on the Q code transformation agent, uh, internally, we have seen some pretty dramatic success. Uh, Andy j he recently posted about it, um, that by using Amazon queue, uh, core transformation agent, we have been able to upgrade our internal pipelines from Java eight or 11 to Java 17, uh, which just saved us like 4,500 years in terms of just development here, that that's a pretty big statistic, and that, that's been a huge success story for us internally. And, um, on the queue agent for software development, which I will talk about in a lot more detail, uh, we were already at the top of, uh, the SWE bench, uh, leaderboard.

And with all the things that we've been, um, iterating on with that agent, there's a bunch of new improvements that have come to that one. So all in all, uh, just very positive feedback on overall queue developer and also just the agent bit. We've seen some real results, Right.

So what's the latest and greatest, what have you added in this release? Yeah, so we launched the first iteration of this agent, um, you know, with our GA product. And in May we, uh, had a blog post about it.

At that time, we were top of the leaderboard for SWE Bench. SWE Bench has dataset, which basically has a Python code for a lot of just, uh, common coding problems, uh, that, uh, people solve for. Uh, this particular version of the agent has a lot of advancements, uh, compared to the previous ones.

Uh, notable ones being, first of all, like a much more powerful model behind it. Uh, the world of models is moving really quickly, so just four months. Uh, a lot of advancement has happened there.

Uh, but it's not just about using, uh, the best model available. It's, there's a lot of fine intelligence and tooling that's built around the model that this agent has, and that's where we have invested a lot of our time. Uh, to be more specific, um, we have the, this particular agent is able to explore a lot more coding parts.

So it tries and explores multiple parts, sees what is working, what is not working. It is even able to backtrack and take a different path. And that sort of intelligence is something that we worked on through a series of experiments in the last four months.

Um, I think the approach we took with this one was just to see if you are a senior developer and you're trying to learn a new language and you are in a new code base, how would you approach the problem? So the, a lot of the agent is mimicking that particular workflow. So the agent takes a goal, you can talk to it in natural language and give it a prompt to say, Hey, I want to implement, then say a shopping card, uh, for an e-commerce website.

So it will take that prompt. The more specific the prompt is, the better it is, um, the better guidance it receives or the better results you will get. But what it does is, just like the engineer would do it scans the code base, it looks for, Hey, which files do I need to modify within those files?

What methods do I need to touch? And it will generate the updated code for that as a human, as the engineer who's using the agent, you get to see the, uh, diffs within, within your IDE, and you can, uh, accept those if they look fine, but at, at that point, you can also give the feedback to the agent. So that, that was the second piece of what we, um, developed in the agent, is the ability to take feedback.

Previous version did that too, but this one, I think we just made a lot more advancement in how that feedback is incorporated and then used for the next step. So it takes that feedback, it will generate a strategy, again, as I said, explore multiple parts and pick the one that it thinks is the best and shows the results back to the developer. Um, I think, uh, a few other things I would say, which power this agent, we actually came up with a text representation of the code, which I thought was a very interesting way in which where our team members were is like, if you're a human, uh, engineer who is looking at code, I think a lot of it is, uh, visually you look at the code, different repositories files in those repositories, and your brain is processing those lines.

What we did here for the agents to actually take the code we in, we came up with a format, which is, uh, text code. And, uh, as the name says, it basically takes the code and, uh, explains it as text to the agent. So it, it's an intermediate, uh, layer that we developed, but that's how we communicate the code to the agent, and it takes that and then determines which parts to take.

Uh, so that's been, uh, one pretty big breakthrough that we achieved in this version and some of the other things we developed. Um, we make sure the agent doesn't get stuck. That's, that's one thing, um, that we were experiencing is you can get stuck in analysis paralysis, even as a developer.

So again, taking the same analogy, how do we make sure the agent doesn't get stuck? So there's some very specific checks and guardrails put in place to make sure it's not getting stuck. And, um, there is a concept, uh, of, uh, agents using tools that is becoming, uh, a lot more popular and mainstream.

So this agent, our software development agent, has these tools at its disposal, uh, which make the code that is generating a lot more accurate. And these tools are, uh, simple tools, like it has the ability to open files, select files, uh, close files, even like, uh, run some basic checks, uh, unit tests. So I think in our future version, you'll say expansion of these tools, but in this version from, based on just where we were previous iteration, there's just not more tools that the agent has now in its toolkit that it is using to just get better results.

So overall, some of the behind the scenes things going on, but the net result is, uh, just 51% more tasks. It is completing on the, again, the SWE bench dataset, um, the verified one, and on the full one it's, uh, 43% more. Do you think as we go along that more organizations are gonna stay current on their code base?

Because let's take Java for example. There's people running on Java eight, still 12 and all kinds of versions, and they don't upgrade it because, well, it's just heavy lifting and they got other things to do, but then we discover that, you know, that older code has more vulnerabilities in it, and then we're surprised when bad things happen. So can we, um, get everybody to stay on the latest and greatest version of something?

Yes, I think that's, uh, where a lot of the advancement in AI will really help the developer workflow. Like any, any team of developers, I feel like it'll be much easier for them to stay current, uh, because a lot of the work can now be done with these agents. Like I was talking about the GO transformation agent that is part of QI mean, these are some real results that we achieved that basically you, we have a pretty big repository here internally, and we, uh, had the agent run on like 30,000 pipelines internally and take the old Java e Java 11 code and upgraded it, uh, to the new version.

And I think that's just, um, going to become part of, uh, you know, the maintenance or the upgrades, the part of, uh, the team that performs the, these functions. I feel like, uh, a lot of these agents is going to be very helpful for that team. Mm-Hmm.

Are we also gonna convert applications in one language to another language? We have a lot of application modernization projects out there. People are running everything from COBAL to whatever it may be.

Um, is it worth actually converting to a different language or are we just gonna stay within our language lanes as it were? Uh, no, I, I do believe the agents are going to expand and, uh, the language translation that are very natural next step to just the, uh, version upgrades. The other thing that comes to mind is we hear a lot about agentic AI these days, and there are these agents.

I think we're gonna have multiple agents that are trained on specific tasks and domains, but, um, how do we orchestrate all those different agents that are doing those tasks? Is there some sort of central command and control for all the agents? How does that work?

Yeah, I think at very interesting times, uh, in the top, in just the Asia architecture. Uh, I don't think the Asia architecture itself is brand new. I mean, we've had agents in the AI world for a while, but I think what has happened is with the, uh, what the L LMS have done and the tooling around that, it's just become a little, uh, easier for us to have way more advanced.

So yes, I think individual agents will become experts in a certain domain, and you will need like a system, uh, an orchestration system, which is really all the job of that orchestration system is to make these agents run in the most efficient way and the most effective way. So the orchestrator is essentially deciding when to pick which agent to run, uh, how do the agents talk to each other? So I think those kind of workflows are becoming easier to build, and the power of each of these agents is just only going to increase.

Um, I was talking about the tools. Uh, a big part of agent workflows are the tools that these agents are able to use. So how to use these tools to make their output better.

I, I think we'll see a lot more advancement there. I feel like we're still in the first innings of all this, and one of the things that people are talking about is we're using these tools to generate more lines of code, but it's not necessarily resulting in better and more applications being developed because, well, we don't quite have the workflows as fully automated as we might. Um, what is your sense of where are we on this journey?

I think the developers are thrilled, but the people who hire the developers are scratching their heads going, well, how come we're not getting more software out faster? Uh, I think that that's an interesting question. Uh, we do have some real, uh, data from our customers that, that who have said their developers are like up to 50% more productive.

That's how they have measured the developer productivity as a result of these, um, AI and advancements, uh, that we've seen. Um, I think it is only going to get better. We are just starting in this journey.

Uh, today I think it's available with these coding assistance and agents that are available in certain, um, IDs today. The next step I see is integration with workloads that teams have built over time. And as soon as that integration starts, I think the power of these agents just becomes more real.

So I think in the next two years, we are going to see a pretty dramatic, uh, change in the landscape as far as SDLC is concerned. Mm-Hmm. And part of that, it seems that the LLMs themselves, the reasoning engines are getting smarter and that may manifest itself in time and, um, better, um, a series of tasks can be completed by an agent in a, in a, in a more natural fashion.

Is that a fair assessment? Uh, I would say yes and no. I, I think the reasoning MO model getting more powerful is going to allow the agent to do, um, tasks a broad set of tasks really well.

But I think more intelligent agents will likely be using, like you, you'll have to do more work on top of the model to make it specialized. And when you talk about the quality of work improving, it has, it will be a lot more than just the model itself. Hmm.

Um, as you kind of think through this whole thing, um, what does the future of an application development team look like? It's starting to feel like, to me it's a mix of humans and agents that are kinda working hand in glove or hand in machine glove or whatever you might wanna say. But, um, you know, how should we be thinking about what is the role of the developer?

What does the machine do? And, and how does all this come together? Yeah.

I think it's an exciting time in the life of a developer at this. Uh, I would say that. And, uh, I, I think a lot of the, uh, undifferentiated work that developers have to do today, uh, like think to upgrade cases, a classic example, or modernizing the code base.

I, I think agents and AI are going to be very helpful for the developers for that kind of, uh, undifferentiated work. So I think developers are going to really spend their time on higher value tasks, and I think they will become the guide for some of these agents, though. They, they still have a very important role to play, which is to teach these agents.

Uh, they become teachers and guides to say, this makes sense, this doesn't make sense. Uh, but I think it, it's pretty, uh, it's pretty exciting that they, this is where they will be spending a lot more of their time. Um, I think just the developers who are at entry level coming out of college are more used to these assistants, um, than the previous batch that came.

And, um, I think they, they wrap up, I expect would increase, uh, and be productive much faster. And I think they'll be able to just do much higher value tasks sooner. Mm-Hmm.

Um, they say that we might be producing more software in the next few years than we have in the last decade, and the whole rate of development is gonna dramatically accelerate. And I wonder, are we approaching a point where maybe we're creating software faster than the business can absorb? I think we'll be surprised.

I think businesses will catch up. And, uh, I, I think to me it's like the past. We produce the software, the past, we will enable the business to run, and it's just, uh, going to be applied.

We, that, that's my prediction. All right, folks. You heard it here.

It's gonna be a lot more fun to build software, that's for sure. And remember the whole purpose of building software just to solve problems. So let's go solve more problems.

Hey Mia, thanks for being on the show. Thank you for having me, Mike. All right.

And back to you guys in the studio.