21. AI is Just a Fad – Tech Field Day Podcast

Although AI is certain to transform society, not to mention computing, what we know of it is likely to change pretty quickly. This episode of the Tech Field, a podcast brings together Glenn Deck, hazer, Alistair Cook, and Alison Klein, and myself, Steven Foskett, to discuss the real and changing picture of ai. Welcome to the Tech Field Day podcast, where we bring together a group of IT experts to discuss a single idea about key concepts in our industry.

This podcast features a variety of perspectives from members of the Tech Field Day delegate community, even though we often record it in association with one of our tech field day events. This particular episode is just a group of delegates getting together to talk about a topic. Today's topic is actually AI and the general feeling that maybe AI is kind of done.

Well, AI as we know it. That is. So, before we get into that discussion, let's go ahead and meet who's on the panel today.

Uh, Glen Decker. I am a global principal at Equinix. I'm Alistair Cook.

I'm a CTO advisor at the Future Room Group. I'm Alison Klein. I am the principal of the tech arena, And I am Steven organizer of the Tech Field Day event series for the Futurum Group.

So, as I said at the top, um, I think a lot of us have been watching AI very closely. In fact, I know that the four of us have, and you know, I feel like there's been a lot of fads in our industry and what we know of AI and what AI has been up until this point seems very faddish. Now.

I think it's maybe a little crazy to say AI is just a fad, but I, I do think, I do feel confident in saying that what AI has been to this point, especially in terms of hardware and software and infrastructure, a lot of that stuff does seem very faddish. Um, I guess, Glen, I wanna throw this to you first. You see a lot of environments.

Am I completely off base? I can understand why those, that there are a lot of people out there that would say that, um, you know, the hype is just too much. All the money being thrown at AI is just nuts.

Um, I can say that, um, the investments are real. That kind of money doesn't go into it unless people think they can actually generate real business outcomes. And we are seeing companies that are really generating results inside their company.

However, um, there's not a lot of knowledge out there in the, in, in the general public about what the components of AI are and the different reasons that all this infrastructure is being laid out there, right? You've got training, and when we hear about all these huge clusters of GPU and all the power necessary and all this stuff, um, that's a, you know, obviously it's a real use case and it's getting used by not just, uh, you know, the big companies like Meta and, and Google and, and OpenAI and all those guys, but also academia, right? And, and government.

But you also have, what enterprises are typically gonna be doing is more on the side of inference and, and, you know, retrieval, augmented generation, rag and tuning, um, also requires some of that stuff, but not an, not as much. Uh, and so, um, there's, there's that dynamic where, which part of AI are you talking about? How you, you know, which parties are using which pieces.

Uh, but then there's also, okay, well this is what AI looks like today, but what is it gonna look like tomorrow? Uh, and as, uh, as will always happen, constant optimization will occur. Um, and will we need all this infrastructure later?

Will it will a better way be found to do this? It, in which case, it may not be that AI is a fad, but perhaps the way AI way AI is done today, um, could be seen as that, or will that be seen as a fad if we look forward 10 years and look back and wow, look at what those goofballs did, right? I don't know.

Um, there's a, there's, there's some evidence to suggest in new stuff that that could change stuff That to me, Glen, that really hits what I think is really, uh, my concern about generative AI solutions that we have at the moment, these massive farms, hundreds of millions of dollars being spent on the infrastructure to build a single foundational model that just doesn't feel like it's a scalable way of operating. I felt for a while that we were, we were hitting one of these hype cycles of these wonderful possible solutions. There's really dramatic things that are being done and getting lots of coverage.

So things like, uh, so generating videos, uh, that look mostly lifelike, that kind of stuff is incredibly useful for getting views on websites and on, on news, but it doesn't deliver a lot of business value to a lot of organizations. I think that huge cost is, is something that's, uh, challenging to get a, a, a return from, and that there has to be a better way, there will be a better way that we'll see wider use. So I think definitely this fad of, of using massive farms of resources to generate something pretty, but not business valuable, just as a, as a part of a land grab will pass.

And we will see a lot of business value being derived from getting really good insights out of large amounts of data, which is what AI is really good for. And I think there's, there's another place to separate this as well, because you talked about that separation between training and maybe fine tuning rag solutions and inference. There's also an element of conventional, as I think of it, AI versus generative ai.

It's a huge amount of business value out of non generative ai. And there has been for a long period of time, we still see that in sort of, um, process automation, the use of video and the analysis, automated analysis of video. So I think we will see a, at some point there will be a sea change in the way generative AI is implemented, and we'll continue to see this value from non generative AI that I think is getting so much less attention.

But this idea of creating these massive scale out farms, it just doesn't seem like it's a, um, scalable solution along the way. I recently had a conversation with someone who reminded me that the first laptops required a wired network connection, um, and could had about a battery life of 45 minutes. And I think that we are kind of at that moment with ai, you know, the, the mad rush of investment just really underscores the value of opportunity that this represents both to enterprise and to society at large.

And that's why the large players are chasing this AI supremacy, but monetization is gonna be the key, and that is going to come by people looking at these massive compute clusters and saying, this is a laptop with a nick and a, and a power cord required for it, and we need to out engineer. So I go back to Glen's earlier comment, how can we do this more efficiently? It's not even the, the financial investment, it's the power investment that's going into this and that they're, there just isn't grid capacity to meet the requirements to build these gigawatt and multi gigawatt data centers.

I think it's interesting what you're saying is that essentially we're kind of running out ahead of ourselves here in a way. And, and I, I, I can reflect on that too. I mean, think about, um, well, way back in the dark ages when people were building a 100 based, uh, GPU clusters and now, uh, they're really, uh, surpassing the performance Perot with H 100 GPU clusters and already looking forward to the, uh, you know, the, the, the next generation, the two hundreds.

Uh, and they're saying, well, that was fun. You know, I, I do wonder if, um, you know, there's gonna be what kind of value proposition there's gonna be for outdated AI clusters, if any. And, and frankly, I would say that there, it looks to me like there is gonna be a business model in building massive scalable ai, you know, GPU clusters.

I think though the business model for that is gonna be sort of a hosting provider business model, essentially companies, uh, a few companies are gonna be successful by basically having a massive cluster that you can use to train your model if you need to train your model. And that is one business model for ai. But, you know, to your point, Alison, I think it's, it's brilliant because, you know, it was actually the invention of wifi that made laptops useful.

You know, it wasn't anything related to CPUs or screens or storage or anything. It wasn't even battery life. It was wifi.

And the same thing I think is waiting to happen with ai. Essentially, we have this incredible promising technology. What are we gonna do with it?

How is it going to affect us? How is it gonna impact us? And, and I think that ultimately it's gonna be, in a way that is gonna be a little bit surprising, it's gonna come a little bit from left field and it's gonna make a lot of what we're doing right now seem, well, maybe not as valuable as we think.

It's, Uh, you know, so you, you made, you, you, you made a comment about the eight, the H one hundreds and the dark ages. And, um, today we do see, uh, customers looking even for the, uh, the L forties, you know, the kind of the, the lower end, the GPUs. I, I just myself bought an RTX to throw into my, uh, my desktop to run llama eight B runs.

Great. Um, so you know that, that, you know, those H one hundreds, I think over time, right? They, they still take a lot of power, but it, those ones that take a lot of power in a few years as nvidia, one of the things that doesn't make sense to me is they promise to kind of double their GPU power every year.

Um, how do you even deal with that from a depreciation cycle financially, right? Um, it's like, how can you make your decision in, in 12 months and, you know, you can get, uh, you know, lower power and, and twice and twice the, uh, and twice the throughput. But, um, you know, the enterprise is not ready for training, uh, for full training models.

Um, I, I think I can say that, um, it's not an absolute, but you know, for the most part, the first step in getting to training is getting all that data ready for training. And, uh, my observations have been that most enterprises aren't anywhere close to that. Um, you know, trying to get their, uh, you know, and again, for gen ai, it's mostly unstructured data.

So getting that stuff in some sort of a, a format that can be ingested, um, by the LLMs, uh, you know, using an LM modeling techniques, it's just they're not ready. And so, uh, they're all, from what I'm seeing, you know, using existing models and doing either rag or they're doing tuning, and really all tuning is, is taking a model that's trained in training. It's some more, it's all it is.

Uh, so, uh, but these, every enterprise I talk to that's gone down this road, um, either with RAG or with tuning, even with a quote unquote consulting company, um, they're all stumbling and having trouble because they expect X, but they're not getting it, right? The Gen AI is not yeah's point, it's not the point, you know, you, I I, I think what'll happen over time, and, and we're starting to see some of it, is that Gen AI will be, will, will have the logic built into it or develop it to ask the questions of the regular AI that needs to come back to the Gen AI to come in and in, you know, language context to be transmitted back to the person who's originally asking the question. But we've seen Gene AI really stinks at, you know, math and, and, and, and, and that kind of stuff, right?

Coming up with logical answers, um, that require comput that require computation, ironically, right? So until that's fixed, you're not gonna be able to trust this thing with, with real work, right? It can, it can generate some content, it'll, it'll, you know, be able to make weird pictures.

And we've seen some of those today out of, out of rock two. Um, but, uh, you know, this is where the hype cycle is gonna meet a little bit of reality. Uh, it's, it doesn't mean that it's, that it's just a fad gonna go away.

Um, it's just people are gonna get more realistic about what this can do today, and it'll drive that innovation towards more outcomes that makes sense. Um, which means it's gonna be, you know, it's kind of the multi-domain, uh, approach. I mean, you see, like you dot com's doing it a little bit, right?

They, they're coming, they're coming at it that way. They're not quite there yet. You can choose this or that, but, um, as this, you know, we're getting the, the, the copilot laptops now with the built in, uh, and, and, you know, the new MacBooks are gonna have it, and the new, um, and the new MacBook, the new, uh, iOS chips are gonna have it.

So they'll be, I don't think you're gonna have a lot of inference at the edge. That's like a big deal. I think it's, it's gonna be your end points sooner or later, especially if this new, uh, stuff that's coming out with, uh, the Matt Mul free, um, techniques of, uh, of not needing matrix multiplication to do a lot of this stuff and to, and to figure out a tension, uh, in, in these models is necessary.

I think if they can do it in 13 watts instead of, you know, 300 or 400 watts, now you're in a form factor in a laptop, right? So the changes happen fast. Um, and I, I've never, I've never seen a technology change so fast and adopt so fast, and then still we're like, I think we're in like the fourth, fifth generation, and it really only started a year and a half ago.

I, I think you hit this sort of speed of change, which indicates it's not ready for your mainstream enterprise business who doesn't have the, the resources to dedicate to this. I think that's a, a really significant element and that, that leads towards the idea that possibly this is gonna end up as being a cloud service. That's, that service delivery model of being able to keep up with the pace of innovation is certainly we see cloud providers better at innovation in new products, and we see a lot of enterprises.

Uh, I do think we are that sort of crossing over that, that, uh, peak of overhyped expectation at the moment and heading towards a bit of a trough of disillusionment. I've been hearing some noises recently about maybe this isn't as as wonderful as we thought. Um, but I think it is important to come back to some of the sort of central principles.

One of the things that we, we heard at, um, cloud Field day 20 was from Bobby Allen at Google saying, AI is not the thing, it's the thing that makes the thing better. And so AI is, is going to become a feature of, of your business process. It's not actually going to be the thing you want to do.

It's, it's going to be part of making the thing better. Uh, adding wifi, as you say to a laptop, adding AI to your application, the application is what's important. And we will, I think, see a lot of use of these large language models for interacting with humans because humans are, uh, unpredictable and behave somewhat weirdly, rather like an ai.

Uh, but as you say, there's real problems with, um, hallucinations, with inability to actually tie back to source document and say, the answer I got was because of this input information. That's the places where with tools like RAG come, come into this. So I think we will see maturity over time, but it does feel like at the moment, enterprises aren't up to the job of, majority of enterprises aren't up to the job of deploying these massive clusters and gaining actual business value out of them because there is just so much work to get your data and get your data hooked up into these models.

Although rags seems retrieval, augmented generation seems fairly simple. You take your business data and you vectorize it, um, just get numeric representation of it, and you couple that with an existing large language model, there's quite a lot of options and, um, choices to be made about how you do that vectorization. So it's still a data science project to do the vectorization, let alone, as you say, that data preparation, getting good enough data to be able to, to generate the stuff.

Uh, this does feel like a place where I'd rather go to an expert organization with that small number of very smart people who actually understand this rather than trying to upskill my internal team, unless there's some huge business value that's gonna come out of it. You know, in the early days of cloud computing, there was this trend of, that I think is happening here, which is kind of layered, um, adoption of technology. Um, if, if you remember when IT organizations were trying to get their arms around the cloud, developers from different business groups within those companies would just pull out their corporate cards and start buying Amazon services.

And, you know, it took time for those companies to get everything, um, into a unified strategy for the cloud if they ever did. Um, I think AI is a little bit the same way. I think IT organizations are looking for the use cases that they would like to deploy AI planning, POCs working with, um, consultants and service providers on these things.

And within business groups, you've got folks who are actually utilizing tools in real time to deploy them to real business problems, um, as kind of a wild west application of ai. And I think that, again, those two forces will come into bearing. And I think that that also speaks to the power of this technology.

People are seeking it out. They're looking for ways that they can apply what they can get their hands onto because they know that it has a direct impact to making them more efficient or addressing a real business problem in real time. I think that what really gets into my head and, you know, rag and, and different fine tuning approaches help with it, is how do you get the enterprise resilience that is required to start deploying this at scale?

And, you know, we've seen a few examples in the news where companies have gone a little bit sideways, um, with their application of ai. I think that that's going to be really the, um, asymptotic moment, if you will, about broad scale deployment when we can get that confidence that AI can be deployed in, in these types of use cases with the same credibility, resiliency, reliability, that other enterprise applications demand. You know, it's, and, and I want, I want to, both of you guys said something very important that I think, um, goes to the original concept of, of, of what we're talking about, right?

Or the fad thing, right? So, um, AI isn't the thing, it's the thing that's gonna make the thing better, right? Um, I, again, uh, going back a few technologies ago, um, this is what I think we need to be careful of in this world is that first of all, you remember the days of application performance monitoring or network automation or, um, I, I used to do a lot of that stuff in, in, in my old career, and it would always, you know, the network guys would always want it, you know, the, uh, the server folks, they would always want these tools that would help them become better at managing the environment, and they would budget it and would always get, you know, cut off, cut off the bottom of the budget in favor of those things that helped the top line, like if, like a new ERP system, something that was gonna make us more competitive, right?

So, um, if from an IT perspective, right? I would say AI's got some problems because nobody, I've got my very, I find enterprises don't really spend a lot of time and money making their IT people, their IT environment work better. They pay for it anyway, they expect it to work.

They'd rather put that investment money in business related stuff. And so if AI is expected to go and do a lot of that, um, I, I think that's going, that will fall into fad world. And a lot of people are gonna be disappointed when their budgets don't get approved for that kind of stuff.

'cause I don't think top level management sees value, even though there is value in that. Don't get me wrong, it should be done, but they've got other things on their mind and ai, they want AI to be a, uh, a top line multiplier, creating new opportunities to create revenue, um, in the, in the realm of digital transformation, right? We hear a lot about that.

Um, so, um, and to something you were talking about, Allison, uh, you know, we're talking about it, uh, with the R hole, that kind of stuff. Most of the time we're not seeing IT driving AI opportunities in enterprise, it's line of business, right? They may be asking it to do stuff and, and, and, and the, and the light, the, the journey of this is like, they're taking like one to six opportunities, you know, business processes and, you know, applying the ai, you know, AI technologies to it, gen ai, it may not even have to be gen ai, right?

And, and they're trying those things out. And, and you're right, Allison, that happens in the cloud first, right? Um, but eventually once you get beyond 6, 7, 10, that's when the economic, that's when a, it has to be brought in to make it scalable and to create this, this shared service across.

Um, and the problem is those, these different lines of business need different things. So, um, one solution doesn't necessarily work across the all six use cases. So lots of problems here.

'cause ai, I think that means, 'cause AI is not monolithic, it's just not. There's the, across the, a single enterprise, it could mean multiple things in our organization. We use, you know, a couple different tools for different things, um, that don't really work together.

You have to kinda use 'em separately. So, um, yeah, I, I just, um, from, I, I think from a fad perspective, it, I can see the, you know, what they call the trial disillusionment. That's what's gonna cause it, right?

Is people gonna go in, they're gonna find good use cases, realize the value they get isn't worth the money they gotta spend to get there. Or, uh, it's just too darn hard. Um, and, and not scalable.

To your point, Allison, uh, over different use cases. I think it's interesting that, um, you know, I I wanna bring up, um, many of you probably, I, you guys probably heard of this, I hope our readers have heard of this, but, um, there was a very controversial, um, question asked back in September, 2023 by Sequoia, uh, v VC firm, uh, VC there, um, he called it the $200 billion question, which was basically, where's all the revenue from ai? And that we are, you know, spending so much money, we as an industry, we as a a human race are spending so much money on AI hardware.

Where's the revenue gonna come from that's gonna make up for that? And, uh, in June of 24, he, uh, re you know, updated that and called it the $600 billion question, because yes, that's what happened between November and June. Now we have a $600 billion hole, essentially that we've dug for ourselves looking for revenue.

And, and I don't mean us, and I don't mean any particular company, I mean the whole industry. And it's only getting worse, and it's only getting deeper. And even if you look at a company like OpenAI that is making money, um, you know, their revenue numbers are really solid.

4 billion in revenue from the, from a new technology, you'd be like, heck yeah, that sounds like a great business, but what if I told you how much they spent to get that revenue? You might not think it looks so good. Um, because essentially they're, you know, buying dimes for a dollar, and that only lasts you so long.

And, and, and so as a, as an entire industry, we need to look at it, you know, where's the revenue gonna come from? I wanna point out, for example, that the airline industry has never made a dollar. If you look at the, uh, the, the money spent to build and fly planes around the entire industry has never made a dollar in the history of the human race.

That doesn't mean that all airlines have been unprofitable. I mean, Southwest was wonderfully profitable for a while. Um, and, and even some of the incumbents have been wonderfully profitable for a while.

But the problem is that it's such a capital intensive business, and it, and, and not to mention the operational costs of running it, that it ends up being non-profitable overall. Well, AI puts the airline industry to shame. Uh, we are spending so much money on this stuff that it had better transformed the entire universe or else it's never gonna be worth the money.

And I think that this is one of those questions that we have to ask. So, uh, you know, to the point to the premise of this discussion, AI is just a fad. As I said at the top.

It's not that AI generally is just a fad. I think that, uh, you know, you look at what Apple is doing with on-device inferencing, they, by the way, they trained all their models in the cloud, they rented it from Google, trained their models, and they're done. They didn't buy all that stuff.

They're gonna do inferencing on device. Hopefully they'll find a, you know, hopefully for them, they'll find a business model for it. You know, you look at what Google's doing, you look at what Qualcomm is doing with on-device processing.

A lot of these companies, I think are going to find a gold mine here. But I, I, I, I wanna get back to the premise and that's that the money that we've poured into these AI supercomputers as an industry, is that a fad? And is there ever gonna be payback for that?

If you look at the GPUs as a service companies, almost all of them are backed by the GPU company who they use, right? It's like, I think it's, it's either some sort of a consignment model or it's, you know, they, they go belly up. Uh, you know, Nvidia just takes their stuff back, I guess is is the deal for, for some of them, um, or some of the other hardware companies are backing them.

So, uh, I don't know what happens if to them, if the demand, or, you know, if, if it all dries up because people either figure out, I can't do this, it's not worth it, or some new thing comes along that well of a sudden, guess what? I don't need all these things. Um, it's a, it there's a lot of risk out there.

I'm gonna completely agree with you on that. Um, so, uh, is that a bubble? Um, I don't know.

I think, uh, you know, there, there's a chance it could be, but, uh, for now, I think everyone's all in and, and it's gonna continue to go. And as long as everyone's, uh, partying with tulips, let's, uh, you know, let's, uh, I'll, I'll take red and yellow please. I think that this bus is one that you, if you miss it, you risk losing a tremendous amount of market share.

And so they can't get it wrong. Um, and I think it's unfair to have a question of when is the monetization gonna happen. I think that we just talked earlier about the fact that Chachi BT launched in November of 22.

Um, it's been a, you know, it's been a little under two years. Um, and the world's developers are coming up with some very interesting use cases to apply LLMs to. I think that this time next year, we will be seeing a very different environment where we're gonna be seeing enterprises actually showcasing some of the ways that they're actually de deploying this technology.

That's, that's my view. And I think that the monetization train will begin. I don't think that it's a, it's going to be overnight what people had thought it would be, but no technology really ever is.

Um, it takes time to ingest and, um, for companies to really understand how they're going to use it. And I think we're gonna see that play out. I, I'm gonna go with, I think this is a fa and that I think we are going to see radical change hopefully over the next 12 months in how AI is implemented, used, and delivers business value.

I think we are in a, a position where we've sent spent $600 billion on tulip bulbs because we have to get those bulbs before anybody else can. And that's Allison's point is, is you are not gonna get business value by not getting into this. Well, I'm not sure I agree with you because leaping into the, the technology too soon, you end up overpaying for the business value that you get.

Are we actually getting to the point where AI tools are giving us business differentiation? Are they making me more competitive? Am I going to lose because my competitors have this?

What I'm hearing is that it's really hard to get business value and competitive differentiation out of using AI tools as they are now. And so, radical change, I think is going to come before we actually see this. And I do think this, this is a $600 billion problem, and it's an ongoing, uh, cost and materials and, uh, environmental cost that I, I just don't see how this is, is something that's going to continue growing at this rate for the next year or two without there being some catastrophic event, hopefully a positive catastrophic event because new technology turns out, new methodologies turn up that make it much more cost effective and environmentally effective to achieve the outcomes that we want from these AI tools.

Well, we get the small, the smaller language models also, don't forget, I think a lot of these companies are starting to realize that I, that, that they can get good enough results outta their models with sm with the smaller, you know, smaller token set, right? So, um, that that's also gonna drive a lot, you know, some, some of this, the squeeze perhaps of the infrastructure. You know, not everybody needs a trillion, you know, token model.

I mean, they haven't proven that it's that much better. Some, some do. 0 right?

Uh, to 4 billion I think it was. So, uh, and they're getting just as good results they found. And, and it, it's the constant optimization of this, again, over this compressed timeframe.

The, the acceleration of the acceleration is just mind boggling. Um, and so I, I don't think we can predict what this thing's gonna look like in 12 months, but it, it's kinda like if it gets a lot better, like if we, if we get, if we develop new technologies that are much more efficient, um, that has a bad impact on the infrastructure side of it, right? I mean, all these, all this stuff that you just bought, maybe you don't need.

Um, but if it stays the way it is, right? I can see from a regulatory perspective, I mean, are they just gonna eat 80% of our power on in the grid for doing this, this, this training? It's ridiculous.

Uh, you eventually, governments are gonna have to step in and say, no, we're gonna, we're gonna ration this stuff out, or you're gonna generate your own, which I see a lot more of, but that's gonna even raise the cost more, right? So, um, it, it's like the, the, the rate of change and the rate of acceleration of change is just, is dizzying and mixing makes it impossible to figure out where the puck's going. Yep.

Yeah. And I, and I, I agree with you there, Glen, that uh, yeah, you look at four oh, and you look at what, like I said, like what Apple's doing, what, uh, IBM meta, uh, you know, all these companies have come up with fine tuned, you know, reduced models that work really, really well. And ultimately I think that that's where we're gonna, we're gonna see AI go, but we've already gone, uh, with this podcast unfortunately long enough.

I think we could talk a lot more about this subject and I think we will talk a lot about, more about this subject in, in future, uh, field day events and future episodes. But before we go, let me just give you all a chance to give a shout out. Where can we find you and continue the conversation about AI Glen?

So you can find me on threads at GEC hazer or on LinkedIn. co nz. You'll also find me at AI field day five in a few weeks time, learning even more about AI as we go through.

net and Allison Klein on LinkedIn. Um, I will be embarking on a, uh, series of content about the sustainability of AI data centers over the coming weeks. So check it out.

Well definitely look forward to that, and I look forward to seeing you all at, uh, future Field Day events too. Thanks for listening to this episode of the Tech Field Day podcast. If you enjoyed the discussion, please do subscribe.

You'll find us on YouTube as well as your favorite podcast application. And don't miss an episode. We have different topics every single time.

This podcast is brought to you by Tech Field Day, home of IT experts from across the enterprise, which is part of the Futurum Group. com/podcast, or you can find these episodes, uh, streaming on Techstrong tv. Thanks for listening, and we will catch you next week.