33. AI Doesn’t Make App Dev Any Better – Tech Field Day Podcast

AI isn't making applications and application development any better. This episode of the Tick Field Day podcast features Guy Courier, Jack Poller, Stephen Foskett, and myself, Alistair Cook, looking at some of the issues around how active Dev works with AI and how AI works with App Dev, and trying to find out whether app dev is actually being improved by ai. Welcome to the Tech Field Day podcast, where we bring together a group of IT technical experts to discuss a single idea about key concepts in the industry.

This podcast features a variety of perspectives from members of the Tech Field Day delegate community, and is often recorded in association with one of our events. Tech Field Day is part of the Futurum Group, and this public is also published on our sister company Site Techstrong TV on this episode before AppD Field Day. And CubeCon will be discussing the premise that AI doesn't make active any better before the discussion.

Let's meet who's on the panel today, starting with Jack. Hi, I'm Jack Poller. I'm principal analyst with PR Paradigm Technica.

Also joining us is Guy Hi Guy Courier, CTO and VP at Visible Impact, another sister company, and I cover app dev for Futurum Group. And on the other side of the table for a beautiful change is Mr. Fst.

Hey, Steven fst here. Thank you for, uh, doing this. I am very excited to be attending my fourth cube con as a, and, and, and being part of the delegate panel, uh, to some extent for this app dev.

Uh, field Day presentations as, uh, a bit of a, I'd say a dabbler practitioner, And of course, I'm Alice Cook. I'm an event lead for Cloud Field Day here at Tick Field Day, part of the Futureum Group. And I've been looking at AI tools and application development together, and we've certainly seen lots of these tools coming out.

All of the major cloud providers have their own tools and, um, their own ways of making AI shove itself into software development processes. And there's even other organizations, I just recently spoke with tab nine who, who have another AI approach to software development. And we start with really simple things of, you know, code hinting as the developers writing something that they know nothing about.

And that really resonates with a very simple audience who know nothing about programming and gets started with something completely new using ai. But that's not the reality for most professional software developers. Most professional software developers have high skill levels.

They understand these, uh, tools that they're working with, the APIs that they're working with. They have libraries of pre-written code that address these simple use cases. So what is AI going to do?

Well, I'll start. Um, so, uh, I think I'm starting from the premise that, uh, AI will not make app dev any better, at least not anytime soon, and is definitely not doing it now. Uh, what does AI do well?

Well, really we're talking about generative AI as a general rule, particularly, uh, all the AI integrations into, uh, various coding platforms, um, as well as the management platforms. Uh, and, uh, I don't wanna say that AI is not providing any value, but is it making the applications better? I think that it can make coding or deployments more reliable in a certain sense, uh, because AI is like your buddy that'll do some work for you.

He's just a really, or she, he's just a really dumb buddy and you gotta check everything that is produced, but that way you don't have to start from scratch or you don't have to scratch your head and look around, or you don't get lost on some, you know, trail of, uh, oh, this looks really interesting and I, oh, I never really read about that. And then before, you know, it's the end of the day or the end of the week. So it does make it more reliable.

And it also can help you with, uh, uh, a certain, um, uh, parts of application development that frankly get overlooked. Um, like documentation, proper documentation or annotation, like a good test plan, like a good regression plan. All of these things help.

But do they make the application better? I don't think considering the way generative AI works, which is to simulate additional, you know, uh, trails of words, text or tokens after an input naturally that is not necessarily gonna produce something that's good, it's gonna produce something that you can work off of. There's a couple things you said that I think I want to explore a little bit more.

And I think the first and most important to me when I think about AI is there's a huge difference between coding and software development and really what I consider software engineering, which is the algorithms, the design and the architecture of the application and what the application does and how it interacts with the data and with the users. And coding is just such a small part of that. AI can help with the coding and it can help you solve some of the most common errors, people, coding errors people make, but it's not, um, it's not making the application better.

It's solving that very small portion of translating what the developer wants into something the computer understands. And I think that's the simple part of software engineering and application development. Yeah.

I wanna kind of riff a little bit on what Alistair said. Uh, sort of characterizing these AI copilots especially, um, LLMs that are generating output for you. Um, it, it's not just a really dam assistant, it's that annoying kid back in school who always raised his hand every time the teacher asked a question and confidently answers the question sometimes right and sometimes wrong.

And the problem with this is that, you know, LLMs very confidently predict what you want them to predict. And often, especially now that the models are getting much better, and we've got much bigger models, we've got much finer trained models. I mean, you know, the, literally the, the co-pilot tm, uh, that was trained on GitHub, for example, has a real good understanding of coding style and what you probably are going to be typing soon.

It actually does answer the question correctly, but the problem is sometimes it doesn't answer the question correctly. And then, well, what are you gonna do? So, I mean, a as you know, when I'm using a, uh, a copilot feature, uh, myself, what I'm always doing is I'm always using it sort of the same way that I use sort of, um, a text suggestion on my phone or on the Mac, you know, type ahead.

Essentially, uh, what I find is that the copilot is going to fill out like, oh, I see you're in yaml. I see you probably wanna have four spaces before the next, you know, thing that you type. Yeah, absolutely.

Cool, thanks for putting that in there for me. I hate having to type all those four spaces, but at the same time, if it like actually puts the parameter and it like, God help us, the actual data in the aml, well, a lot of the time that's not gonna be the right thing. And, and it's the same with coding as well.

I mean, you know, you're, you're writing a common function that you, everybody writes all the time. Yeah, it's probably gonna fill out the thing correctly. Um, you're doing something novel and interesting, well make sure you're looking at it.

There's also a broader point to be made, like along the lines of what Jack was saying, which is what you're trying to do with your app is provide a user experience that, uh, you know, uh, helps a user do something, um, helps the organization do something, carries out a function, or even if it's internal, ultimately there is a user experience that you're driving. And that is a breadth of scope in terms of what's going on that generative ai, at least in current models, can't address. So even good code, the right answer, as you put it coming out, may not have the right kind of a place in that overall system.

And if the user experience is not getting a better, if you ask me, that's when the app application is not getting better and app dev is not getting any better. So I think a really important element in here that both guy and Steven have hit on is that the value in app dev is not in knowing the details of the individual APIs. The, the value is the ability to take that, um, set of data, that set of requirements to deliver some value to our users and turn that into an application.

So for AI to be changing things in active, and I think it will, uh, it needs to take away the bit of the undifferentiated heavy lifting. It needs to continue to take away the needing to remember what parameters to use, how many spaces or tabs to use, which of those is acceptable in this organization. Because again, standards and styles are very organization dependent.

Uh, for AI to be useful, it needs to take away a lot of that pain from developers of trying to do things the right way and the acceptable way and allow them more to focus on those higher level values. How am I gonna present this data in front of my users? How am I gonna solve the problems that the business is actually experiencing right now?

I think that's where we get to the change away from, I have to write all these hundreds of lines of code and somewhere in them I've written something wrong towards what is my business requirement here? What am I going to do to make somebody's life better, somebody's work better, this business better? And that's really what we want from AI in general.

We don't want AI to be making art while we are still at work, as the saying is, right? I want the AI to do the boring work and let me do the creative and an an interesting part. And that has to be where we, we find that AI is transforming things.

And I think we're quite a long way away from that at the moment. I think we're very far away from that at the moment because the science, the art and science of software development and software engineering is, you know, and this is what I was saying earlier, it is not in the coding, right? It's, it's in understanding the algorithms and the architecture and the implementation and the data structures and how you optimize those tools to develop a solution that meets all of the requirements that your users have.

And that's, um, you know, it's very similar to writing pros, text pros and English or poetry. It's, you can give somebody a set of rules, but you can't necessarily create art out of a set of rules, um, the syntax, right? We can, we can say, um, uh, you know, that the AI is really good at doing syntax and correcting the four spaces or making sure there's a semicolon at the end of every line or whatever it is.

And that's a very good, um, developer productivity improvement, right? And it takes that as productivity improvement, but that doesn't necessarily improve the app itself or app development, um, itself. If we have our developers lose focus on providing the correct user experience and if they become too much reliant on AI to do their job and they lose focus on what the app is supposed supposed to be delivering.

There's another aspect of AI too that I think we should talk about. 'cause it's not all co co-pilots. And I think what we're seeing now as well is a lot of, uh, uh, companies using AI well to do, um, other aspects of AP dev including, you know, fuzzing and, um, development, uh, QA testing, uh, automating the software development process with, uh, you know, uh, rollouts and all sorts of things.

I mean, heck, there was even, um, I don't remember which one it was, but, uh, it's, it's, it's, uh, doing mouse movements in the IDE on your behalf so that if you're commonly clicking here, clicking there, clicking there, it'll watch that and learn that and do that for you. Um, so, so there are other things that people are using AI for. Unfortunately, some of those things have me a little scared because essentially, you know, AI can do things that I might not want it to do, um, well against my code, you know, I mean, I could have a situation where it's, um, like I said, trying to see ways of exploiting or finding weaknesses, uh, finding, you know, buffer over overflows and things like that in my code.

Uh, that's a different aspect of AI that may not be improving the software development process, or honestly, maybe it is improving it. I dunno, Jack, do do you think having AI find exploits in my code is imp an improvement? Uh, yes.

I actually think that's a very important improvement about it. I, you know, and I come back at today, my last code development projects were actually developing hardware chips, which you, you know, you develop a chip in software and you simulate the chip. And part of that is the, the process is very, it takes a very long time because you wanna make sure you get it right because you can't just fix a bug and have a new chip the next day, it's three months and many millions of dollars later if you have to go do an, a revision of a chip.

So you write a lot of code to test your chip and you simulate it with a lot of random inputs, and you wanna run those random inputs until you've made sure every line of code and every code path has been executed. And that in and of itself is a very long and arduous process to get right. Having AI be able to do that or to augment that in some ways can accelerate that process and improve the development process.

Now I look at that and say, does that, that makes the development process better? Does it make the application better? That part, I'm not sure, uh, it can make it better from a security perspective because finding buffer overflows before that code gets, gets into production could be very critical to preventing, uh, a zero day exploit.

Um, but I, you know, i i, security of the code versus the application being better itself, I don't know, Code analysis is one of the cooler things. Um, code analysis summaries and annotations among the cooler things I've, I, I've seen AI do. But I have to admit that code analysis for, uh, security, um, also, uh, fills me with, uh, some level of terror and dismay, not because, uh, of false positives, um, or that sort of thing so much as, because it just seems so easy for the human to just go, well, I'd done the analysis, it's done.

And that's putting a lot of, so we're we're talking about, uh, what your, your, your dumb friend or I don't remember what Steven said. The, the four spaces thing. Uh, the, the person in the C classroom, whoever it was, um, now that know it all kid who said, oh, yep, the code's safe, or I found this.

It's really when they fi they find that that's the vulnerability right there. You gotta take care of that, and then there's something else that gets missed, but you're relying on, you know, something to help you through this admittedly tedious task. So that's where I get terrorized and worried, But I get delighted by some assistance to do tedious tasks because that lets me do the more challenging, the more interesting, the more useful tasks.

My kind of general thinking on this is, if we have a fixed amount of resources to spend on building this application, are we better spending it on all of the, the, the tedious processes that are, uh, repeatable and, and automatable in one way or another, whether it's AI or whether it's some, some other scripting tools, uh, or whether it's not generative ai, but actually predictive ai, or whether it's just straight conventional static code analysis. Are you using this version of this library? Are you using this routine?

Are you passing your data, you know, you're validating your data as you come through? Um, I think it's, the thing is that we've got a fixed amount of resource on where do you wanna spend your engineering effort, and I think Jack wants us to spend our engineering effort on making a better application rather than making the implementation of a poor application better. If you follow the differentiation.

Yeah, I, I would agree with that, with the caveat that we have to always understand the limitation of our tools. And I think that's what guy was getting at, is if we have very high confidence that the AI tools are producing, um, valid evaluations of the code, and they're catching all of the possible security issues and all the possible bugs, et cetera, et cetera, then it's okay to rely on them. If we don't have that high level of confidence where our confidence is misplaced, then we've got problems.

And, um, you know, that's, you know, there's a a data analysis question that comes into play there of how do you analyze and validate your AI is doing the right thing, and at what point does our reliance on ai um, cause us to lose the ability to understand exactly, um, what we're doing and whether the AI to be able to judge the AI as doing the correct things or not. So I, I think as, as we're looking at these tools, one of the things that, that we see in manufacturing where we've got an automotive process, you don't simply accept that everything coming outta that automated manufacturing process is good. You sample it and you audit the results.

And so this is an element of, there still is that human requirement to go back and look for something that doesn't make sense, doesn't fit, doesn't work. And so I think there's a, uh, an important element of just we need to validate trust, but validate along the way with, uh, these automated tools. And, um, GLA you wanted to leap in?

Well, I think that AI has an obvious and clear benefit. I just don't think it's the higher quality app yet. I don't think it's a better app yet.

And that's my sort of stopping point for saying it's not making app dev better. Um, I think this is because application development is a many layered thing, uh, a host of layers and the user experience functionality and so forth. It's just one of those layers.

And, uh, when we think about AI improving the productivity of, you know, marketers or knowledge workers or what have you, they're not dealing with many layered issues. Uh, uh, you know, a piece of writing is a piece of writing so to speak, or a picture is a picture. But with app dev, there's so much at play that I think that's why it can still be helpful.

It can provide something that's ultimately more robust. Um, but is it it better in this sense of does it do more? Does it, it does, is it more advanced?

And that's where I think we're missing for a while, what AI can do in app dev. But I think there's a range of things that AI can do in a software development lifecycle that isn't just about writing the code itself, that's more about the things that surround that code. And we see a lot of value in, in getting those things right, thing elements that are useful for integrations and those, those kinds of elements.

And one of the key things that we see through AI building AI tools and using AI tools is that garbage in, garbage out. You've gotta have good quality data coming in in order to make these decisions. Often it's you, you take that generic information that a, a large language model has, and you specialize it to know about your own coding styles, your own, uh, language, that very private, uh, chip description language, the VE log that Jack has been writing in order to write checks, this stuff you've gotta add.

So data becomes really central to all of this. Yeah, I want to tune in on that because that's one of the things that I've been pretty excited about seeing in the app dev space is companies working on some pretty neat things related to data, not necessarily to, um, to the, uh, actual act of coding. And this is one way I think that AI is already actually helping me.

So let me, let me be specific. Well, philosophically you might, you guys probably know apostle's law, right? Um, be conservative in what you send, be liberal in what you accept.

And that's one of those things that I encounter all the time when I'm doing any kind of, um, development, any kind of work, is I, I find that the things that my outputs are always, um, well formatted, you know, um, you know, if I'm outputting, uh, JSON, I make sure I've got everything filled in, et cetera. But the inputs that I'm using are often, let's say, less well formatted. And I spend a lot of time trying to futz around with RegX or figuring out a different API that I can use to draw on the data.

It's the same when you're talking about databases. Um, a lot of the time now, rather than having, you know, very nicely structured databases, you've got a data lake full of random stuff, and you might pull up a fish or you might pull up a boot in your data lake and, um, it, it helps to be able to try to get some of that stuff under control. So I've been looking at a bunch of companies that are talking about creating, um, metadata using, um, deep learning ai, and also structuring, uh, data in a way that's useful.

So for example, I'm already using, uh, AI tools to reconstruct data sets as JSON, and I find that that actually really, really helps me to, to, to know that I'm actually going to be getting the data that I'm looking for coming into the application. And then from there I can, I'm, it's much easier for me to use that data because I have strictly defined keys, you know, what's the title, what's the description, you know, that sort of thing. And, and all of, all of that input is, is is extremely liberal because basically the LLM is able to take kind of almost any garbage and structure it in such a way that it comes out in a nice format for me to use.

And I, I, I'm actually pretty excited by that Now that is that application development, I guess in a way it is because it's saving me hours and hours of, you know, munging data in order to try to get the right thing. But, um, I guess that's the kind of thing that people forget about or they don't think about when they're talking about using AI in app dev. Well, I, I like that concept of, you know, and, and I've learned that I always learned that as doing network programming of accept all input and gen, you know, accept all input, even if it's invalid and don't crash on it and always generate correct output.

And that, um, flexibility or lack of brittleness would be great if we could use AI to do that, not only in app dev, but also in the applications themselves. So it would be great if people could apply that to web interfaces. So I don't have to sit there and figure out that I actually, do I need to put parentheses around my area code when I enter my phone number in this particular website, right?

It's can the application take my address regardless of how I figure, put it in and figure it out without making me, you know, you have my zip code. Do you really need me to enter my state and my country and all the other information that you're asking me to enter? And can you not just grab the data in whatever raw text I give you, throw it to an AI that can analyze that and say, here is the valid correct format for this application to use without bothering the users.

And that sort of understanding of data and cleaning up of the data, that part can not only make the application development better, but can make the apps better as well. And I look at that sort of a two for one win. Maybe it's it's above, uh, or, you know, over my head, it could be over my head.

It's not really my area so much, but you're talking about, um, I know we're talking about masses and masses, terabytes and terabytes of data that's unstructured. Um, and, uh, a lot, there's a lot of innovation going on right now in turning those into relational like experiences where you can use sql, you can do regular queries, you can organize and all that sort of stuff. Um, so I still feel like we are dependent, Stephen, on the human, in this case, maybe the data science scientist, uh, or the, the human operator or administrator or what have you to ensure it's almost like, um, I, I can see it making things faster.

I can see it making things more complete, um, in partnership with a human. Well, I'm not saying that you cut the human out of it for any, for of course, but I, I will say that I've been pretty impressed by what's going on here. And so I, I, I can't talk about it today, but tomorrow I'm gonna talk about it because the product launch is actually tomorrow.

But one of the companies was showing me a, essentially an AI driven, um, it's an agentic AI based data lake, and they can scale to petabytes and even exabyte scale with, um, arbitrary unstructured data. And they can use basically AI calling ai calling ai, calling AI agents to scan through, you know, billions of objects, trillions of objects, and, um, structure that in a way that is useful for an application. Um, it is really cool stuff.

They also have a self-describing API. So as you hit the API, it, it, it re responds to you in a way that guides you to what you're actually trying to do with the API. Um, those things are kind of neat, right?

I, I, I'm gonna say that, you know, kind of, I guess in summary, back to the, the premise of this podcast, I think AI can help you in application development. It's just, I think people are so fixated on copilot, you know, contin, you know, finishing my sentence AI, that they, that they're losing sight of the other cool things that AI can do for you. My, my contention for a year and a half at least now, has been that ai, generative AI at least does two things for you.

Neither of them are improve your productivity. They make you more reliable in the way I described earlier, if you have writer's block, they get you, they get you started if, if writing is what you're doing. Um, and, uh, they improve quality because they allow you to do more rounds of review and attention.

Um, and I, I think that that's what you're driving at here. Um, I maybe there's a layer here that you're identifying that leads to a better user experience and a better application. Um, it seems a little indirect to me, but you know, it does make sense that if you use better, better data and you can use more data and you can be more comprehensive, that the outcomes should be better.

But that still relies on the human operator, scientist, developer, what have you, to make sure that it happens. So there you go. So I think we're coming to the conclusion that AI has a place in application development, but it's, it's augmenting humans, not replacing humans.

And thank you all for joining us today on the Tech Field Day podcast. Before we go, where can people connect with you and continue the conversation, guy? Well, they can find me on Blue Sky at Guy Courier's feed, uh, along with Tech Field Day, which, uh, welcome to Blue Sky Tech Field Day.

com. And Jack, you are not on Futurum, where do we find you? I am not on Futurum.

You can find me on LinkedIn, uh, and on efficiently connected. But of course, Mr. Foskett, where do we find you?

Where are the many places we can find Mr. FoST? You can find me pretty much every Tuesday on the Textron Gang.

Um, you'll also find me on Wednesday for the Gestalt. It rundown most weeks. And of course, you can find me at s Foskett on most popular social media sites, including the Mastodons, the Threads, the Blue Skies, and even the Ex Twitters.

And of course, you can find all of my guests at App Dev Field Day in Cobe Con this week. Thank you for listening to this episode of the Tech Field Day podcast. If you enjoyed the discussion, please subscribe on YouTube or your favorite podcast application so you don't miss an episode.

And do consider giving us a rating or a nice review that really helps other people find this awesome content. This podcast was brought to you by Tech Field Day, the home of it experts from across the enterprise, and a part of the FU and group. com/podcasts or view us on Techstrong tv.

Thanks for listening, and we'll see you next week.