13. GenAI is Revolutionizing the Enterprise – Tech Field Day Podcast

Everybody's talking about generative ai, and it has really filled the entire conversation enough that business people, uh, executives in industries outside the world of software development and IT are starting to think about how gen AI can revolutionaries their business. But let me ask you, do you think that it can really move the needle? That's the discussion on this episode of the Tech Field Day podcast.

Welcome to the Tech Field Day podcast, where we're both on premise, and yes, this week on premises. Each time we meet, we bring together a group of it luminaries to discuss a single topic or premise. This week, we are here at App Dev Field Day recording an episode focused on the impact of generative AI on enterprise.

Yeah, I know, but bear with us. It's a really good discussion. Before we begin though, let's meet who's on the panel.

Hi, I'm Jack Poller. I am an industry analyst. I started my own firm recently, paradigm Technica, and you can find me blogging on LinkedIn.

And I'm Calvin Hendricks, Barker. I'm CTO and co-founder of Six Feet Up. Uh, we are a Python and AI for good consulting agency, and you can find me on Twitter or X at Calvin hp.

Hi, I'm Josh Atwell. I'm a Derel and marketing leader, free agent at the moment, and I am on X at Josh Atwell and on LinkedIn. I'm gonna rerecord my opening because I f****d it up.

Let's try that again. But you guys, you guys did great. So we'll just paste this around.

Here we go, Corey. Love to Ebony. So everybody is talking about AI and everybody is probably excited or worried or both about what happens with ai, but one of the things that we're hearing is that a lot of companies out there are trying to figure out how they're gonna benefit from generative ai.

And this week's premise is frankly not a bright spot for those companies. The topic or the premise that we're gonna be discussing today is the fact in our opinions that maybe, uh, enterprises won't actually benefit all that much from generative ai. Maybe it won't be really worth the effort.

And, uh, I guess let's start off with, uh, with you, Jack. Uh, you suggested this topic, so why won't it move the needle? Well, I think there's, generative AI is very much in its infancy, right?

And what we don't know right now is, um, exactly how to build applications that do anything more than what we currently do today, just faster, right? So is if we invest, if an enterprise invests in ai, are they simply taking exactly what they're doing today and making it work a little bit faster? Or are they going to transform their business?

And I think that's really the issue. If you're gonna invest the amount of money and time and resources that it takes to degenerative ai, what are you getting out of it? And I don't see it today.

I don't see any groundbreaking up, um, applications that are moving the needle in one way or another. I, I think Jack picked the topic. So we could say the word AI over and over again, uh, to make sure we hit all the, like, you know, hot spots.

I, I'll say that I think generative AI actually is gonna benefit the, the enterprise. I, I think there's a lot of use cases we're not seeing that are kind of working behind the, in the long tail of enterprise applications. I mean, just from a a application standpoint alone, the ability to have generative AI build, uh, unit tests and functional tests, automated testing for the applications is a huge time saver.

Just with that one alone, I think the next thing you're gonna see is gonna be the ability for generative AI to, to handle a lot of the, uh, the kind of go betweens. Whether we're gonna actually, you know, for example, building a query. If you had a large faceted search that you were building a query, and maybe it had hundreds, if not a thousand facets, that's an unusable UI by a human.

But it'd be really useful and interesting for people to be able to slice and dice their data in such a way that they may actually extract additional value. But if you can actually chat with that search bar, it builds some pre-made, uh, preset like limitations or kind of like filters it down for you automatically, or gets you to a, a usable point by a human. There's another big win there and potentially discovering things.

Yeah, I, I think that's a pretty reasonable take. I, I look at the current state on AI within the enterprise, it's a defensive posture, right? You've got a lot of companies now who they have employees, whether they're in the legal department, whether they're in the development team, whether they're in marketing, who are using these public AI frameworks.

And there is a real concern that, hey, do we have intellectual property leaking out into these platforms? Is what we're getting back or these teams getting back actually something that, uh, uh, we can trust? Uh, is it gonna be a risk?

Um, I have family members who work in the pharmaceutical industry and they're actively saying, okay, you cannot use these external AI model, but we are working on, you know, taking open AI or some other model bringing in, we're gonna create our own thing with our own data and do our own thing. No real timeline for some of those as to when that's actually going to, you know, net a valuable outcome. But I, I think right now it's, it's very much a defensive type of thing.

It's like the, the genie's out of the bottle. People know about these tools, they're going out and trying to use them, and they're trying to determine like, what is our comfort level and how do we want to govern and, you know, keep track of how this is getting utilized. Because I don't think, to Jack's point, I don't think at holistically there is a real understanding about how this technology is going to improve the way these enterprises operate and run.

Well, I think that that really kind of hits the nail on the head. I don't think a lot of these enterprises really understand what it even is. I think a lot of them are hearing about it.

They're hearing, I mean, AI is like a wave on top of a wave on top of a wave in terms of buzz. It is everywhere. It is.

You know, everybody's talking about it. Your grandma's probably talking about it, but do they really have any idea of what generative AI is apart from knowing that it's buzz, buzz buzz and they gotta get them some of that AI magic, I Guarantee their employees are using it, whether they've put a policy in place to say they can't or not, they've got their phone with them, they're opening up the, the chat GPT app and they're starting to talk to it. Yeah.

And the gain value. Well, and then also, you know, in, in earlier discussion, I don't think there's a, there's a person in an enterprise anywhere that is in the process of buying a product or renewing a product, particularly software where AI is incorporated into that product. Or is it at least being marketed and promoted as contributing to that product.

So there's also a level of, um, you know, how do I gain enough understanding to, to actually recognize whether or not I'm getting a real added value? And that maybe that maybe they're even trying to charge premiums for that value, or they're trying to differentiate there's a risk associated with that. And so, you know, that I think that's the biggest exposure that they're getting more than anything.

Well, and, and it's, it's the value part that, that I think is the missing picture here. And yeah, there's a lot of employees. You can get a little bit of use of it, and people are doing things, and I see that, right?

But when you, it's, yeah, I, I look at that different on a personal level where I can do a little bit here, a little bit there. And to me that all of that is acceleration of one form or another. I'm accelerating the business and there is value about that.

But is there anything else? If I'm, if I'm A-A-A-C-I-O who's leading an organ large organization, and I'm looking at it and saying, we're thinking contemplating investing, not, you know, tens of dollars, not hundreds of dollars, but millions or maybe tens of millions or hundreds of millions of dollars into an AI effort for the organization. Where is that leading?

What am I doing? What is transformational about that rather than, you know, revolutionary rather than evolutionary? And I, I look at it and, you know, we, we had an earlier discussion at lunch about, you know, uh, the interface.

And one of the beautiful things I think about, uh, the, the AI is the ability to understand things in English language right now, which means that you can go give a computer an English language and not a query, which is where most people are doing, not doing a data search, but do something else that's much more transformational. As a security guy, I look at it and say, I'd like to be able to set policies to say, Hey, you know, Josh, you, Josh can look at this set of stuff. He's allowed to look at accounting data because he's in the accounting department, but he is not allowed to look at engineering plans.

And that's all I should have to do as a security guy. And everything else should happen automatically. 'cause the AI has the ability to take that English language statement mm-hmm.

And translate it into things that computers can understand. And that means that the business gets this huge value of not having to do all of the empty, um, thousand steps that would normally take to get, to translate a business leader's commands into execution at the computer level. Well, and, and less error prone potentially.

'cause less humans potentially are involved in that understanding of, can see, can't see of a document or of a yes type type of piece of data. So there, there, there's clearly risk in it, but, you know, I look at, you know, the, the, the Microsoft and the Googles and all these other companies that are doing the copilots, right? Mm-hmm.

To me, that copilot is really just accelerating search and that's it, It can accelerate search. But I think there's also a flip side to that. And it, um, we can accelerate the ability to understand if we have incorrect permission set on, on, uh, documents or data sources inside the infrastructure.

I mean, most people's, uh, OneDrive or SharePoints are probably a mess of, uh, share this Land. Oh, correct. Absolutely.

To a specific person. It would take a human, an error or prone human, uh, a long time to discern whether they've got documents that are shared with the wrong people. Or you can unleash a model against that.

Actually, it's really good at classifying whether there's risk or not risk a associated with it. So it's more than even generative ai. It's like using AI to actually discern whether you've got, um, risks, outstanding risks.

Well see, I think, but that's exactly my point is it's more than generative ai. It's More than generative ai. Yeah.

It's by the generative AI part of it. The output part where we are today is we're saying, okay, take an English language thing and then use that to generate output, a picture, some text, whatever it is. Mm-hmm.

Right? It's taking, going much further beyond that and saying, how do I take this thing and apply it to solve these real world problems that then have the potential to transform in my company? Whether it's the security aspects of it or whether it's, how do I say, um, you know, look at everything, you know, I, Josh came in and talked to me and said he wants a loan to buy a house.

Right Now. What are all the steps involved in that? And how do I do that?

And what is it that an AI can do for me that changes that whole business model of a applying for a loan and getting a loan, right? Mm-hmm. And, and to that point, I think I wanna sharpen the conversation here.

I didn't say ai, I said generative ai, right? And what I meant was, and, and, and what I, what I think that Jack is kind of, kind of hi hitting here is that, you know, think about things like LLMs. Think about things like some of these other generative AI that can, you know, generate an image or generate audio or video or whatever.

Um, how are those things really going to move the needle? Not for the world, not for, but for enterprises, for the average company out there that is not in the software development business necessarily. Uh, just, just just any of the thousands of, you know, hundred plus person companies out there that are trying to do something and they've got this, this chat bot, or they've got this image generator and they're excited about it.

They know they need this thing because it's getting a lot of buzz. How is that thing really gonna benefit them? And I think that that's kind of where this conversation's going, because I don't think any of us are Luddites, I don't think any of us are saying AI is, is dumb or doesn't work or something.

But I think all of us are realistic in saying, hold on. I don't think it's quite what you think it is. Right?

Yes. I can look at a couple of scenarios. Like for instance, you have a, you have a company, a smaller company like you were describing who they are in the process of developing a new marketing plan.

And rather than invest millions of dollars to work with a firm to develop all the graphics and all, all the, you know, all the visuals and, and the copy working with a generative AI system and say, Hey, this is who we want to reach. This is our core message. This is the, the key outcomes.

Like, how would you frame out a marketing campaign? You can already do that, and it will give you samples for banners. It'll give you samples for, um, copy that you can put on your website.

Like, that's something that they can do and that has tangible value. And of course they can take that, make modifications. The difficult part though, and I've, I've seen this, um, with, honestly, it's a, uh, a bar in my hometown.

Um, the owner is using chat GPT to create social media graphics and to create menus and things like that. The problem he runs into, and I love this guy, but the problem he runs into is it doesn't quite give it to him the way he wants it. And making minor edits is very difficult.

So fi the fine tuning becomes challenging. So he, he's getting really great material that he can use to promote his business mm-hmm. And to grow his business.

But then there's a skills gap or a tooling gap between, it's like, oh, like it did that thing that really just looks off or weird if only that would go away. But the system has difficulty like fixing that, but it's improving. But I don't think that's the revolutionary change.

We're, we're kind of aspiring to here, right? Not, I think if you're talking about a large enterprise, if you can actually create some kind of a multiplier inside the organization that gives you an advantage, that would be a revolutionary usage of gene ai. And so I've, we've actually experimented with this a little bit in our small team, but if you can take for example, performance reviews or if you can do assessments across multiple groups and say, uh, plot data of like a sales team on the east versus a sales team on the west mm-hmm.

And understanding where their strengths are, that LLMs and large language models actually can do a really good job of consuming that, those results. Uh, consuming a framework for which to think about them and then actually allow you to now ask questions and understand like, what could team A learn from team B mm-hmm. And get these kinds of insights that may not be obvious when you're looking at a giant like, set of survey results, but you could actually create a multiplier inside the organization and actually build a 10 x team as opposed to like each team working in their own silos.

So, But I, I think that's going to end up coming, as I was mentioning earlier, from the vendors you're already using. 'cause that would be something I would expect Salesforce to then provide me as a service to be able to give me those, those, So they're gonna get it. They're, they're gonna, they're gonna get a revolutionary usage outta LL M1 way or another.

Yeah. Right. But, okay, I'm gonna challenge you on this one.

Is that really moving the needle? Because a lot of the things I'm hearing are things that enterprises may believe they're already doing, that they already have access to, um, that they can achieve in different ways. Uh, that, you know, is generative AI actually doing something you can't otherwise do?

For example, we saw an example today, a demonstration today of a system that you can query to see arbitrary information using rag uh, retrieval augmented generation, which connects a database to an LLM. We saw a system that allows you to query it about support, uh, you know, basically query your, your, your technical support database and, and about aspects of your product. Whether it does this, whether it does that, is that actually needle moving technology or is that just a better different search engine?

I mean, is it really revolutionary 'cause 'cause Calvin you just said revolutionary. Revolutionary, right? Almost Fire.

Is It really revolutionary? I think it is. I think humans can't hold the amount of context in their brains that the LLMs are capable of doing.

Even right now, like you look at the, the Gemini like million token plus like context windows, that gives you the ability, if you think creatively about the usage of it to feed it, a lot of information that we can't even comprehend as a human being, like that, you were talking about this at lunch earlier too, is that there's just such, it's such a big set of data and we can't make sense heads or tails of it, but the LLMs actually do a really good job with this. Well, volume of data doesn't mean value of data, but I think That they give you an opportunity to extract value from The data. Well, what it, yeah, because, well, what it gives you, and this, this goes to the other side of it as well.

Like, even as we find things like, the difficult part is going to be going back and validating whether or not what we are getting from these systems is accurate and the direction we need to take, right? It's, you know, a trust but verify. 'cause a computer can tell you whatever it wants based on whatever data it had access to.

Well, and, And, and I, and I'll put it this way, there was, I have seen, and I'm sure it was on, you know, today's news and yesterday's news and every week or so there's an article about, you know, AI is going to replace your job, right? And I think the last one I saw was AI is gonna replace the CEO's job. And I think that that's a very naive view of Ingrid, right?

It's just, it's just naive. And the reality is that the, particularly the current public large language models are trained on data in the, that's available publicly on the internet, which is then used as Josh said, in this perfect example to generate new publicly available live data on the internet. And very soon we're gonna be in this recursive situation where LLMs are being trained on their own output.

Mm-hmm. Which We're actually, I would point out, we're actually already in, so as of this recording, we're in May of 2024. That is a massive topic in the AI industry because I'm, so, I record an AI podcast.

I have been very focused on the nuts and bolts of how these things work. Synthetic data is a very big topic in the AI industry. It is very widely used.

And it is interesting to think about where that goes. And again, I will come back to it. Are these use cases really revolutionary?

That's Just hype. That's, that's, that's my point is from, from the very high level, from the 50,000 foot level of the enterprise, looking at it and saying, I'm investing in ai. Do I go and build my own AI engine?

Do I go and train, you know, how many millions of dollars does it cost to train an LLM for my application specific environment? Well, and the answer there is Abso absolutely not. I'm just gonna tell you right now, get this outta your head.

Enterprises will not be building their own AI models that is asinine. And no one's gonna do it. And I don't think they'll be fine tuning 'em either.

They may be fine tuning, they may be doing transfer learning, probably not. Yeah. Alright.

They'll be relying on things like the technology like rag and, you know, vector databases and embeddings. Right? I think there's a lot of value.

Quick, quick easy wins there. Especially when it comes to retrieval of like, you know, if you could have the knowledge of a, of a, of an enterprise more evenly accessible to the whole enterprise, I think there's some gains that'll be made there as well. A lot of repetitive or reproductions or, you know, someone re reinventing the wheel can, can be hopefully avoided.

I guess the, the cautionary tale I have is, is really much more of understand what you're doing, j just go jump into it and yeah. And say, okay, now I can do this with AI and now I can do that with ai. It is not the silver bullet that is going to magically transform your, and today big data that's going to transform is the silver bullet, right?

That's gonna transform your organization. To be honest, I think big data has a much more likely more likelihood of moving the needle for many enterprises than generative AI specifically. Well, the combination of the two, if you've got, if you have your big data story down and you've got really well structured data, you now have got an advantage to actually extract bigger value out of it.

Yeah. I, and I'm gonna, I'll, I'll provide a counter position here. I actually do believe that we have had this really great outcome of this movement around DevOps in that now everything has an API and APIs are something that these tools eat up.

Mm-hmm. And it makes it extremely simple to get 80% of what you need done using one of these generative AI systems. And when you think about that, now we're hitting a point where, you know, the time to market to put out certain features or capabilities will continue to shrink.

Or more importantly, the integrations between different components and getting them to work more effectively together. 'cause these tools, they like the structure and of a, of an API. Mm-hmm.

They're able to consume it, get a general enough understanding to produce just good enough for a proper developer or an admin or someone to fine tune it and make it work specific for that environment. So when I think about the, what I expect over the next couple of years to make an impact that we look at, that's the one that I'm, I'm, I'm most bullish on. I think that one's got the biggest chance.

So I guess, uh, let's wrap this thing up with, uh, I'm gonna go back to you guys, back to the, uh, original premise of our discussion. Is generative AI specifically going to move the needle for the average, you know, fortune 2000 Enterprise? Not for many years.

A hundred percent. I see the next couple years. Hmm.

Yeah, I think it's gonna, I, I think we're gonna start seeing it as I, as I outline, it's gonna be in the tools that they're already buying and in, you know, very specific API driven things, but I, I don't think they're gonna be called successes for several years. Well, there you have it, folks. We don't have an answer but Par for the course.

But this was actually, I think, a really interesting discussion because this is the question that our, many of our listeners, many of our WA viewers, most of our, you know, the tech field a audience are wrestling with because we deal with people who are IT decision makers. They're out there in enterprise, they're trying to do their jobs, they're interested into technology, they're excited in technology, and they're talking to business leaders who jump on the hype train here and jump on the hype train there. And they have to try to help those guy, those people, uh, kind of talk 'em down from the edge and say, no, no, no.

Uh, this is what this technology can really do. Whether it's virtualization or blockchain or big data or digital transformation or ai, uh, yes, these things could move the needle. No, they're not gonna do it on their own because no technology's gonna do it on its own, right?

It's all about how you use it. So thank you very much for joining us, uh, for this episode of the Tech Field Day podcast, recorded live at AppDev Field Day, uh, on premises. Uh, before we go, I'm just gonna go through, uh, tell us, uh, where we can find you, where we can continue this conversation, huh?

Thank you. You can find me on LinkedIn and, uh, starting my own blog soon. You can find me on LinkedIn as well.

I'm Calvin HP in most places on the Internets, And best spot for me is on LinkedIn. It's Josh Atwell. That's Pretty easy.

And as for me, uh, I'm Steven Foskett. Uh, you can find me at s FoST, uh, pretty much everywhere. And of course, you can find me here on the Tech Field Day podcast every Tuesday, unless it's Tom Hollingsworth.

Uh, you can also find us on Wednesdays for the, uh, gestalt it rundown, and of course, you'll see a lot more tech Field Day coming up in 2024 and beyond. Thanks for listening to the, uh, tech Field Day podcast. If you enjoyed this podcast, please do subscribe.

You'll find us in your favorite podcast application. You'll also find us on YouTube. Uh, you can also, uh, visit our website.

com/podcast. Thanks for listening, and we will catch you next week.