The Impact of GenAI on App Development with Prismatic’s Tanner Burson

This is Textron tv. Hey, thanks for the throwaway here with Tanner Berson, who's vice president of Engineering for Prismatic. And we're talking about, well, what's real with AI and application development, because we certainly hear a lot of claims, but it's not quite clear to me that they all play out just the way we all might think when we initially hear that first promise.

Hey, Tanner, welcome to show. Thank you. Thank you.

Happy to be here. Folks seem to be all over the spectrum. On the one hand, some folks are saying that developer productivity is gonna increase dramatically, and we will build more software in the next two years than we had built in the last decade.

Other folks are helping me to type ahead. Uh, my coding is nice, but you know, it's hardly a game changer. So, uh, between those two extremes, where are we and what are we seeing and what's reasonable to expect?

Yeah, I think it's a great question. I don't think any of us know yet exactly where, uh, where where the, the real is. Uh, you have folks like, I think it's the, the CEO of the AWS division at Amazon saying, you know, within a decade they won't have any software engineers and it will all be AI and flip side.

You have, uh, it was the CTO of Azure recently, or not Azure, it's the Google Cloud side. Um, talking about, you know, they've invested heavily and maybe 25% of their code is generated by AI today. And so there's a pretty broad spectrum even at the, the very high end of the industry in terms of what's realistic today and, and where that's going.

Like, like most things, it probably lands in the middle, but I also think most day-to-day engineers aren't likely to see their, uh, their lives dramatically changed or altered by where this technology is and where it's likely to be in the next couple of years. And one of the things I'm still scratching my head about is like, well, just because we generated more code doesn't mean that that code actually made it into a production environment because, uh, theoretically that code is running to the average and we want the code that goes into production average environment to be better than average. So, um, what exactly are folks doing with all this stuff?

I mean, is it kind using, it made me well for, um, if I need to integrate something to a pipeline, fine, but am I really writing business logic? I'm not sure. Uh, similar.

I think a lot of the claims are overblown based on the, the kind of a zero of, of starting a new project. There's a ton of boilerplate. There's a ton of kind of speculative, uh, rough sketch kind of code that gets written.

And I think a lot of the AI tools are great for that. They can help you think through some of those initial, uh, rough sketches very quickly, uh, and maybe work through a couple of those rough sketches much faster than you would've before. They don't have the context on your actual business domain.

You know, the, the AI tools are looking at a handful of text files and trying to give you a, a likely next set of things that would be helpful given, given that limited context, you have the context of your business domain, you know what your customers are looking for, you know, the real problems to solve. And so ultimately it's still up to the engineer to figure out what the right software to build is, how that needs to to work and what's going to happen. I think some of the ai, uh, planes are a little overblown in in in light of that.

It does seem though that we have this obsession these days with developer productivity. So if, if it's not coding, what is hampering developers from being more productive? It's the same things.

It's always been problem solving is hard. Uh, debugging is hard. Um, there's a, a famous quote I really need to go get the source of, 'cause I've used it a couple of times recently.

Um, but everyone agrees that debugging is harder than writing code. And therefore if we write code as clever as we possibly can, by definition we're not smart enough to debunk it. And I think that has always been true.

And I think with AI tools, it makes it easier to write code that you don't understand and you don't understand the implications of, you don't understand the security, the performance, uh, all of the, the other characteristics that go into building software. So when we talk about developer productivity today and what's hampering it, it's an inclusively complex security environment. Compliance requirements are going up.

Everybody is socked to type two, at least for their, their SaaS businesses today. Performance is more important than it's ever been. The tools for managing that have gotten more complex, not simpler over time.

Building software is a complex environment, uh, and we haven't actually done much to simplify that. And I think some of the AI tools, uh, attempt to manage the complexity by doing it for you, not by removing the complexities that are there. And in a lot of cases, I think it adds more unknowns to a process that is already hard, not, uh, not removing them from you.

Are we kind of in danger of being overly reliant on these LLMs because they do endeavor to be extremely helpful and they kind of give you the sense that, uh, they're confident in their proposal to you and we don't review it then? And do we need to be really careful about what exactly we're putting in the system even though it might have been created by a machine? Absolutely.

I think companies have long relied on code review as a tool for keeping a high quality bar in their code, ensuring that more than one engineer on a team understands how, how the software is built and that it's following the standards and meets the requirements. I think with LLM generated code, that's even more important. And I think those code reviews have to happen even earlier in the development process, working with an AI coding assistant or a co-pilot or whatever.

The branding for those, uh, is for a particular tool turns into you having to do live code review as the code is being generated. And so the still required to understand that code, understand its implications, ask for iterations, ask for changes, and work through it are the same as if you were working with code written by somebody else, uh, in your code base. You just can't assume that they know the business domain, that they understand the underlying problems that they are working with.

The same intent that an engineer on your team is, you have to dig even deeper in those code reviews and make even fewer assumptions than you would for a standard code review. And I think teams that are really reliant on AI code generation are going to have to work incredibly hard to build that are code review, uh, processes and, and policies and practices compared to what they were doing previously. I hand help but feel it as great as it is to have an AI tool that writes code.

Maybe we need more focus on AI tools that do all the other jobs that we don't wanna do that are actually hampering productivity such as reviews and testing. I think there's a, there's a strong argument that code is not the most important place to, to work on developer productivity. If we think of the, the broader discussion we were just having around where, where is code, where is that productivity going?

And I think some of the, you know, second or third generation of these tools will be more focused on things like looking at the security implications of code, looking at your actual runtime environment and what are the real security and performance profiles of that, and being able to provide guidance and input into, uh, the process at an earlier stage. But those tools aren't there yet. Um, there's a lot of promises by some of the AI vendor or what some of the security vendors in particular, uh, that they will shift left, uh, with AI all the way into your dev environments.

And I, I haven't seen any of those work yet, but I do believe that's, that's where we're going to see a lot of long-term value is on the, the, the runtime side, the observability side of your platform, and the ability to reason about that more clearly and make decisions about that while you're building the software as opposed to only after it's been in production. The only thing that often leaves me scratch in my head is like, well, let's just say we can produce more code than ever. It's not clear to me that the DevOps pipelines and the delivery side of the house is able to absorb all that.

So do we need to kind of revisit how all those workflows are gonna work if we're gonna be essentially increasing the amount of code moving through those pipelines by several orders of magnitude? It, it's a good question. One of the problems I have heard discussed by teams that have adopted, uh, a lot more AI code generation is the size of the releases has gone up because they've been able to write more code.

The amount of code they put out in a given release has gone up. In some ways that creates more risk than before. A lot of the DevOps practice over the last decade has been moving towards smaller, more atomic releases, being able to push out small immediate changes, roll back, small immediate changes, and the AI generated, uh, code allows people the idea that they can get more done, uh, and put out larger changes, which now creates a debugging, uh, an observability challenge of larger changes going into production and needing to, to monitor those.

I haven't yet seen anybody try and tackle or to break those into smaller units and whether there's tooling that can help push that, uh, at a faster rate. But it's a really interesting thought of how do you continue to, to push that down even further And what is the mechanism that we're gonna use to generate the machine code? I mean, today we seem to be trying to insert these things into an IDE, but I scratch my head and I go, well, the machines are more efficient and we only created the Java language to make it easier for humans to interact with a machine.

But if a machine is talking to another machine, you know, do we need a different construct or what we see a more lower level kind of construct, or are we gonna go the other way and there'll be more of a higher level abstract and we will have something even more interesting than low code doc? There's a lot of interesting ideas, uh, in, in there. Um, some of those, going back to the 1970s and four dl, uh, languages and visual programming and things that have been kind of tinkered with for as long as we've been building software where we've landed is that code is a tool for humans to understand problems.

And if we were just trying to tell the computer what to do, we'd all still be writing assembly, but we've built these higher order languages and these, you know, complex tools and frameworks, not because they're more efficient for the computer, because they're easier for us to build more complex software and to understand that better. I think if we're going to move away from code as something that humans use to understand the problems, we absolutely should be revisiting whether LNS should be generating code at all. Is that even the right, uh, is that the right artifact for them to be creating if a human doesn't need to see it and doesn't need to interact with it, is that the right, uh, is that the right vehicle?

I personally don't think that's a great direction. I think ultimately, as noted, like there's not enough context in these tools to understand the problems. Ultimately, we build software to solve problems, not just to write code and to keep computers busy.

Um, we're, we're trying to solve real problems that have for people. Um, and to do that today, people need to understand those problems and be able to break them down, describe them, understand them, debug them, and code is still the best tool we have or doing that. Uh, so I think there's some interesting theoretical concepts in there, but I don't think we've yet landed in a place where those are, uh, super viable.

We also hear a lot of chatter now about AI agents and the subtle difference here being that in theory, at least an AI agent is able to reason and incorporate some data in real time and execute some tasks on your behalf. I can see how that might work within our workflow of an SDLC. But, um, one I have, I don't know, potentially hundreds of agents that I have to then figure out how to orchestrate and manage, and that SDLC is made up of a group of AI agents and people trying to figure out how to build that thing together.

It, it feels like it's gonna get complicated. One thing that has been true in my entire career of software is we've yet to make it actually simpler. Um, it, it feels like the process of building software today is as complex or more complex.

When I got into it, you know, 30 years ago, um, now the software we're building is far more complex and far more capable than what it was, uh, on a, on a small DOS terminal back then. But I don't see any indication that humans are likely to try and reduce the complexity here. Uh, we seem good at adding layers and adding more abstraction to this, but reducing it doesn't seem to be in our, uh, in, in our path right now.

I agree. I think currently the addition of AI seems to be increasing the complexity of the development process and the development environment saying more challenges and more things to overcome, not solving poor fundamental issues. I might think differently if I was on a greenfield team of two or three engineers trying to build, uh, from scratch new software, but working in a more complex established, uh, environment, it's much harder to reason about how those tools are going to fundamentally shift what we're doing.

Ultimately, once your best advice to folks, I don't think we wanna ignore ai, but I think we have to find some way to constructively experiment with it. I agree. I think the right way is the same way you would any other tool in your development environment.

Uh, engineers love fiddling with their IDE and finding new tools and new ways to improve their workflow. And I think there are places for AI in a variety of different pieces in the workflow. I think different people and different teams have unique problems in ways that where it fits is going to be different.

And so my recommendation would be the same as, uh, any other tool set, which is to figure out where you're having problems, where there's opportunity for you to improve it and look at what the landscape of tools is. Um, look at where LLMs could help. Look at what they're good at, look at what they're bad at.

Um, I also think actually understanding what they do, um, not everybody needs to be able to go build an LLM from scratch, but I think understanding at its core, the, the behavior system and the modeling and the way, uh, the systems interact is key to having some intuition as to where there are good, uh, fit and where they may or may not be a good fit. And so I think ignoring it entirely is the wrong answer. Uh, being dismissive of the capabilities is certainly not, uh, not right either, but I think having a, a healthy skepticism and understanding what, uh, what are good spaces and what are poor spaces for that kind of experimentation will be important.

So looking in your crystal ball and let's go all the way to the end of 2025, maybe into 2026, is the way we build software fundamentally any different or we we're gonna pretty much be doing the same thing maybe with a little less toil. I think we're doing the same Thing we are today, Uh, a year from now. I think next year is a year.

A lot of companies who leaned hard into AI this year are going to learn where the really sharp edges are. Um, like a lot of advancements, there are, uh, easy wins at the beginning, uh, and some of those you pay for later. And so I think there's gonna be a lot of companies trying to learn how to debug code that they didn't write, um, trying to understand how to, to deal with systems that they don't understand well enough, uh, to, to really make sense of, um, and figuring out what that means for what, what's next for them.

Um, I don't think most companies are going to look dramatically different, uh, at least on the software side. I I think we're gonna look at building software, uh, in ides with code much the same way we do today. Uh, maybe with a few better helpers, uh, and, and a little less boilerplate here and there.

All right folks, you earned it here on the warning stream. If you're terrified of ai, you probably shouldn't be on the other end. If you think AI's gonna do everything for you, you might wanna temper those expectations.

Hey Tanner, thanks for being on the show. Yeah, it was great. And back to you guys in the studio.