AI in Developing IT Systems – Techstrong AI Podcast EP34

Hello, and welcome to the Techstrong AI Podcast. I'm Amanda Ani, and I'm excited to be here today with Scott Wheeler. He is the Cloud Practice lead for Asper Toss.

How are you doing? Great. How about yourself?

Doing well. Well, can you first of all share a little bit about yourself, your background, and the company, and what does Asper Toss offer? Sure.

Um, as you said, Scott Wheeler, I'm the cloud practice lead. I've been, uh, working in, uh, cloud since about 2008, uh, from the very, very early beginnings. Um, uh, my background is in, uh, software development for the past 30 years, and that kind of transitioned into cloud infrastructure and, and things related to that in that in the past, uh, 15, um, Asper Toss Consulting is a professional services firm.

Uh, we're focused on, uh, large Fortune 2000 companies, um, migrating to the cloud, uh, creating cloud infrastructure. Uh, now that we're years into this, uh, we really are focused on, um, increasing their capabilities in the cloud. Uh, companies now, generally, all of them have, uh, a cloud presence, usually with two or more cloud vendors.

Uh, so at this point we're doing things like making more effective, whether that's cost effective efficiency or, or things like that. Uh, so that's generally what we do. Okay.

Wonderful. Thank you for sharing. So, moving into our topic today, which is how the move to AI will help codify and document security compliance and architecture decisions made in developing IT systems.

So can you share how will it do that? Well, it, it's, it's on its way where we are in this, in this ai, uh, journey that we're taking. Uh, right now, AI is, and has been for a little while, uh, tremendously helpful in, uh, developing systems.

Uh, the, the use cases that are most popular right now in AI are related to software coding. Uh, software, uh, AI helps people find, uh, basically other code, uh, on the internet very easily, and AI puts it together and, and, uh, takes all that information and, and basically puts it into something that you're asking it to do, uh, from the body of of work that's out there. So, uh, AI vendors out there or, or vendors that actually do software and security and other kinds of things, uh, what they're using it for is they're embedding it into their products, uh, to allow, uh, customers to have, uh, better responses, quicker responses, more accurate responses.

Uh, it's basically taking, uh, you know, the old tech search or you think about a Google search and just making it a lot more intelligent, a lot more, um, appropriate to the, the questions being asked. What are some of the big roadblocks you hear business leaders talking about when they try to implement ai? Uh, the biggest thing that, uh, especially large companies deal with is the, the idea of security, right?

Um, you're, everybody's concerned about the, the, the, the AI being this, uh, big behemoth that's going to go and, you know, uh, find out your deepest secrets and, you know, take over everything. And, you know, and, and you have, you know, uh, you know, a, a, you know, like Terminator or something like that going on. Um, really right now, a AI as it exists with LLMs, and this is going to change, which are large language models, which is like your chat, GPT.

What it really is, is it's, it's a better tech search. It's a, it's a more relevant tech search. Uh, 'cause basically all these models do is break down whatever you give it in, in the case of chat, GBT, that was basically everything, you know, on the internet it could find that was relevant.

Uh, in the case of, say, an organization, it's, it's going against your, your documents that you have internally. Uh, right. And it's, and it's giving you back the information.

It's breaking that down in an algorithm, building it back up, and giving you the, the, the relevant information in a, in a, um, a much better format than say a search engine would. Um, we haven't gotten, although there's things coming out now like, um, chat, GPT Strawberry, uh, which are, you know, these systems are starting to do more reasoning around it, which is where you think of more of the intelligence aspect. But in this first generation, they're dealing with the things that we're dealing with in it now, which is, you know, analyzing logs and doing chat bots and, you know, helping with coding.

That's just this first generation LLM, which is basically, um, uh, a very sophisticated, uh, search engine to give you very relevant information. Do you have, in working with companies, do you have any use case journeys that you could share with us, perhaps? Um, absolutely.

Uh, a lot of the large companies, as I say, that we're working with are cautious at this point. Uh, the things that are actually happening now that larger companies are more comfortable with are the pre-packaged solutions that come with software. For example, Microsoft Co-pilot.

This is something that comes with, you know, your, your Microsoft 365, uh, you know, uh, software package that you can add as an add-on. Um, very good. You can have it go and look at your documents and suggest you know how to reword things, or you can tell it, make me an outline to a proposal, or, you know, whatever it is.

Um, those kinds of AI implementations are easily usable, right? You really have to understand the only knowledge you really need is how to understand, how to create a prompt, how do you ask the question to get something relevant back, right? Um, on the other side, there's, there's companies like say, you know, a Salesforce or an Oracle or, you know, other software vendors out there that are embedding AI into their, into their, um, you know, software products, uh, to get better answers inside their world.

So say your CRM data's in Salesforce and they have an ai, you know, uh, solution called Einstein. It'll go look at that data within those four walls of their Salesforce environment and give you answers on your, you know, customers and, and things like that that are obviously more, uh, compelling and, and, and better structured than say just a simple text search. As this technology is advancing so rapidly, what advice do you have for business leaders as far as how to best implement it and where, how do they start?

Yeah. The, the, the best way to implement is the same as we do with everything in, in it. You really have to understand what it is your end goal is, right?

What are you trying to get out of this, right? Are you trying to get better customer satisfaction? Are you trying to drive, you know, lower cost through efficiency?

Um, you really have to understand what's the goal? If you're just playing with it, okay, play with it. But, you know, don't put a lot of money into that.

It's, it's, you know, understand the technology and position yourself to be able to acquire it at some point, uh, the technology to acquire it at some point where when it makes sense. But if you really want to dive in and, and, uh, spend money on it of any significance, um, understand the outcomes you want, um, is it your end users want to, you know, uh, manage their email or do their documents, maybe copilots, it's okay for that. You need a pilot to understand what the return is gonna be on that.

If it's a, uh, GitHub copilot, which helps developers, you know, code better and faster, uh, do some work on that because, uh, studies have found that, you know, uh, for junior developers, people say they have, you know, 1, 2, 3, 4 years experience. It's not really a productivity booster. Uh, for developers that have, you know, 5, 10, 15 years experience, it's a huge multiplier, right?

So you really have to go through and understand, uh, what you're trying to get out of it, what's your end result, and then, you know, do a proof of concept to, to vet out whether that's really gonna happen. Um, the space is moving very quickly now. You're gonna see a lot more, um, uh, AI embedded in things, but I, my my feeling now seeing the market, is it shaking out where, you know, very few people are going to train their own models unless they have a very specific use case, like, say maybe financial trading or, or things like that.

Um, you, you, you're gonna see more and more people actually just consume products like copilot, whether that's GitHub copilot or Microsoft's copilot, or, you know, things embedded into software products like Salesforce and that, I think that's really where 90% of your, uh, corporate buyers are going to, you know, spend their money is on, on predefined stuff rather than rolling your own Mm-Hmm. That's an interesting thing you said too. It depends on the staff employee skill level as to how helpful these tools are.

Mm-Hmm. Absolutely. We're finding that, um, in a lot of customers when they're using say, Microsoft copilot, um, they're just giving it to 'em, saying, you know, you know, the sky should open up and they should just know how to do it.

And what they're realizing is, a, anybody who's used these things, either it's Chachi, TPR, whatever, um, the first time you enter in, you know, you know what is X, Y, or Z, and it just has no content context and it gives you back whatever it right. And you're like, that's not that accurate. It's like, okay, give it a couple sentences to describe exactly what you want, right?

Which is prompt engineering they call it now. Um, so, you know, people do need to be trained on these things. People think, oh, it's just ai, it's gonna do, it's gonna magically, you know, the, how computer's gonna figure out, you know, your deepest, darkest desires and give it back to you, and that it's just not the case.

Like with every technology, you have to understand how to use it, uh, you know, what are the limits of it, and, um, and then you can be effective. Yeah, absolutely. Well, if there was one key takeaway you could share with our audience today, what would that be?

Uh, don't think that AI is gonna solve everything for you. I think that's the key. Uh, it, it's not, it is an evolution, uh, in technology.

This is the LLMs are just a continuation of, um, what we have done with say, machine learning is a form of AI that preceded this, uh, didn't get quite the press. This is just another evolution in that step. So understand how to use this version of the technology like we have in previous versions.

Um, uh, be aware of it and, um, you know, never, as I mentioned before, never go in, uh, without understanding what is, what the outcome is you really want out of this. Um, that's the most important. All right.

Well, thank you, Scott, for coming on our show and sharing your insights with us today. It was great. Thank you very much for having me.

All right. And thank you to our audience. Stay tuned because there's more.