Open-Weights, Data Sovereignty, and LTS Models | Utilizing AI Episode 43

Even as the media wrings its hands about the advanced hacking capabilities of OpenAI and Anthropic models, similar capability is now available to all from a variety of sources. On this episode of "Utilizing AI," Brad Shimmin, Olivier Blanchard, and myself consider the implications of the modern global AI stack. Welcome to "Utilizing AI," the podcast focused on practical applications of artificial intelligence and part of the Futurum Podcast Network.

Every Wednesday, we explore news and use cases of the ways in which AI is transforming enterprise IT and the industries it serves. I'm your host, Stephen Foskett, President of the Tech Field Day business unit at Futurum Media. Before we dive into this discussion, let's meet who's on the panel today.

Hi, everyone. I am Brad Shimmin. I'm VP Practice Lead for Data Intelligence, Analytics, and Infrastructure at the Futurum Group.

I'm Olivier Blanchard, also Research Director of the Futurum Group for Intelligent Devices and the Intelligent Edge. And as I mentioned, Stephen Foskett, I run the AI Field Day events, and often I'm on the Techstrong gang, pretty much every Monday episode. And one of the things that came up there, and one of the things that has been certainly coming up in our world, is the proliferation of AI tools and the fact that everybody's got AI tools now.

I know that there's been a lot of hand-wringing in the popular media of the prevalence of rogue AI hacking into other companies and things like that. Well, if you thought that that was going to be contained, you clearly haven't been paying much attention. Now everyone can have that capability.

Thanks, China. And we can all hack into anything we want at any time. Welcome to the dystopian future.

Brad, are you the president of the dystopia? Oh, I am, absolutely. I would much rather rule in some place that's hotter than normal than serve somewhere else, for sure, and that makes me an open source advocate, by the way.

But yeah, I think that you're right. " Because yeah, we have this proliferation of models that are coming out of China that aren't just open weights. Some of them are, but some of them are also fully open sourced under the MIT license, which is one of the most permissive licenses that actually lets you do business on top of that model, which is kind of interesting.

But anyway, we have so many of these right now that are flooding the market, and I think that's the only way to put it, with cost ratios that make your Claude code $300 a month Max subscription look like a kind of a bad investment. Yep. Oh, you're waiting for me to say something.

I don't want to rule anything. I just want my peace. I want my technology that should be making my life easier, that has been promised as a solution to make everyone's life easier, so we can let the AIs work and let the tools do things for us, so we can have more time to do more things, either to be more productive or to pursue things that are more interesting to us, like painting and writing and travel, instead of the opposite, which is a lot of this technology, unfortunately, making everything more difficult.

Even if it gives you back time in one way, takes away more time in another, and where it's just complete chaos. I'm frustrated because this is my profession. And I'm really trying, guys.

I'm trying to stay optimistic and upbeat about all the potential. But unfortunately, it just feels like just tornado after tornado of chaos and mayhem, and it's not great. Which I guess is good for us because it gives us an endless supply, first of all, of things to talk about, but also as analysts, problems to solve and to help our clients solve.

But I don't necessarily need to be working that hard. I'd like to not. So China, if you're listening, yeah, just calm down.

Calm down. Well, no one's going to calm down. And it's funny because you're right.

It's an interesting position to be in for those in our profession, because usually when we sort of survey a group of enterprise practitioners, for example, we're not going through the exact same thing they are, but we are right now. And as you say, I'm not sure what it means to even be an analyst anymore. Am I an analyst or an AI shepherd using the persona of an analyst?

I'm not sure, honestly. And I think that this is a train ride that we've all gotten on, and no one's going to calm down, Olivier. The procession of innovation is such that we'll find new ways to use these things.

And like you say, some of them are going to be very beneficial for us. Some of them are going to take time, as you say, which is one of my biggest frustrations with any AI model right now is it does something really fast that I could never do. It's like taking a huge corpus of information and making sense of it.

That's a very valuable tool. But what it generates from that might require me to spend almost as much time making sure that it's right than it would for me to actually do that going through that corpus on my own. Yeah.

Yep. Just something I just have to note because I'm obsessed with Odysseus and "The Odyssey" as many people are, but when you talked to us about being AI shepherds, I feel like, are we the Cyclops with our little AI sheep and some meandering band of barbarians comes into our cave to slaughter us every time that we get our AI sheep in order? Is that the metaphor?

Or are the Chinese models holding up in a big wooden horse that we just pulled into our citadel? I think you probably could make a case of that. We were talking on Techstrong gang last week about the mystery model that everybody's all excited about, the anonymous mystery model, and how people are jumping on it.

And it's easy to do that if you've got a model router, or one of the more modern platforms. " But you're literally giving your data to someone- Why do you think it's free? you don't know and asking them to work on it.

Now, their work product could be perfect, but they still have the data. Yeah. And that's actually a big concern, I think from a lot of listeners, a lot of corporate folks who have thought about this, is one of the challenges with AI as a service is that you're literally handing over your data, and that has a whole bunch of sovereignty implications.

3, which is, by the way, what caused me to start asking this question this week, no matter how good that thing is, you're still giving it your data or some of your data. And there was also the news that Stripe may have bought OpenRouter. It did.

There again, no matter how much you love Stripe, and I know everybody loves Stripe, are you really okay handing over your business data, your business plan, your, I don't know, confidential medical information, whatever it is, to a router that's then going to hand it over to someone else to do some work on it? I think that there's a case to be made that this is a big wooden horse. Of course.

And that's why Stripe wanted OpenRouter, because OpenRouter is the horse. Yeah. The cynical case is that it doesn't matter, that's what's happening everywhere, and you're giving away your data to everyone, and whatever hackers are going to get to your information.

So it's sort of like it's not a how, it's a when. And so if we can save some money, we'll just fix the problem later if it happens. That's the cynical view.

And I think that might work for a lot of SMBs that don't really have the budget, and for a lot of companies that are like, "Yeah, no, it's private data," but honestly, it's not necessarily that big of a deal. I think that it is a moat, however. So the flip side of that is that it is a moat for the big models, the big companies in the US that financial institutions especially are used to working with.

And that's what would keep... And I don't want to harp just on China. It's not just China doing it, but on any model that might be malicious or that might not have the sort of security framework that an important organization with a lot of very private and sensitive data would need to have.

So it's not necessarily the performance, it's the sort of assurance that your data is not going to end up in a weird place. Yeah, bang on. It is assurance.

And this has been from day one. If anyone was going to use OpenAI's ChatGPT or any of the GPT models, they did it through a virtual private cloud running on Microsoft Azure, their long-lived beloved partner. And that's for good reason, because they have a lot of the controls, and there is one person or one company to go to if there is some sort of breach or some sort of problem.

And that's what's so dangerous about acquisitions like the Stripe acquisition of OpenRouter, because like you said, Stephen, I know that OpenRouter has great controls for... I'm not going to use DeepSeek directly from DeepSeek's hosting service because that would be stupid. But I would use it from OpenRouter because they do have controls over things like data retention, for example.

And so I want to trust them. But when there is an acquisition like this that puts the company in the hands of a company that you may not understand, and may eventually be something that's not good for you, that's not what you want. And it's funny because it's the same pattern that you see with a lot of open source projects that either the engineer gets hired or just they acquire all the engineers, acquire them, or a plug-in vendor for a browser makes a great plug-in.

That person sells the company to another party, and the other party just happens to be a state actor with bad intentions. You've got to look at the entire supply chain of your AI, and that's really the critical part here I think we're talking about. Yeah, exactly.

And whether it's an open weight model that you're running internally or a router or a third-party service, there's a lot of hands in the data, in the chain these days. Yeah. Yeah.

And the one software. I think that's why edge hardware and having your own servers or your own AI box on the side of your desk is becoming much more important than you might think. I think that a lot of organizations in the past, or at least before AI, sort of moved towards cloud and everything cloud-based, and now we're seeing this shift again back to a healthier mix where you can run AI models increasingly locally, and without necessarily having massive investments in servers.

Well, on that note, also, we just learned that Apple introduced the new Mac Studio with a M5 Pro Ultra or M6 Ultra, I don't know, and a half terabyte of RAM, over a terabyte per second of memory bandwidth. That thing's an absolute monster, and I think Apple's going to sell every one they can, and I think people are going to be running these exact models on that hardware. Let me put it to you both.

Let's say you do that. Let's say you go out there and you buy that new super-duper Mac Studio for, I don't know, $20,000 or however much it's going to cost in September, and you run the latest Qwen model on that. Can you wash your hands of this whole discussion?

Are you free? Yes and no. So tell us which.

Yes, in terms of you have the ability to actually audit that trail that we were just talking about and have some sense of control over it. You can, for instance, look at your network and see is anything being ex-- What am I trying to say? Sent back to the company that built Qwen, Alibaba.

And by the way, it's not. Qwen does not, and none of them do. But there are still facets of deep learning models that we're talking about here that are unknown, and there are lots of 4chan-style conspiracies going around that can never be proven or disproven, simply because they sort of hinge on the fact that you can never know what a model is thinking or doing.

And I'll give you the example I'm talking about. So if, for example, you train a model and you have reinforcement learning to align that model with things, guidelines like don't make bombs. Great guideline.

You might also have baked into that a sort of code phrase that if it ever shows up in the prompt that goes to the model, that it has a pattern in there, which is to take a certain action. And do you ever know what that is? You kind of do, if you're able to have a fully open source model, which isn't just the open weights, but the actual recipe to build and reinforce, learn, train the model, as well as the dataset it's built on.

" So yeah, it is, but it's not guaranteed. Yeah, I mean, everything that Brad said. The only thing I would add is tokenomics.

You have your own token factory right there by your desk. So you've front-loaded the investment. You've paid for your hardware, and now you don't necessarily need to pay for tokens the way that you do if you get your AI from a service in the cloud.

So the advantage there is if you're only an occasional user, which is most people in an enterprise, yeah, sure, you can host a lot of stuff on servers. But if you're a power user, and you would run through your allocation of tokens very quickly in a matter of days, for the month, then it definitely makes sense to have that kind of machine. Yeah, 100%.

Not just because of the data privacy advantages of it, but just the economics/tokenomics of the whole thing. And that's going to be increasingly a factor as well. Yeah.

Yeah. They're equal to the security, I think, honestly, because you're limited by how many watts you can pull in your fancy new Mac. And so if you're just wasting tokens on thinking tokens, what are we doing here?

And so I'll give you a good example of that. 8 is very chatty, in terms of if you have the thinking set to medium or high, it is going to have this string of thoughts that are like, "Wait a minute, should I do this? " And that's great because it can use that to sort of reason its way through very complicated requests.

But it's expensive in terms of the tokens that get burned by that. Because that's open source, you can see those thinking tokens and have control over them. Not all open models, like open weight models, like Meta's new Glimmer model, give you that transparency and auditability or control, I would say, more than that.

So you can't really see what the thinking tokens are doing or how they're spending, and that's always been the problem with the frontier hosted models. So if I use Claude or a GPT from OpenAI is really the one I'm pointing a finger at here. You really can't see and control what the spend is on those thinking tokens.

That's a problem. Also, yeah, not for the Mac specifically, but generally if you have a GB10 or especially a GB300 system in your house or your office, it could heat. It could be a great space heater also in the wintertime.

So you need some proper ventilation in the summer, especially if you live in southern climates like I do. But in the winter, you could probably heat a room, if not your whole office or house, with one of those if you use it enough. So that's either an advantage or a disadvantage, depending on what part of the world you live in.

In Iceland, it might be an advantage. There was a Bitcoin company that literally made a baseboard heater form factor of their Bitcoin miner, and a tower heater form factor, too. " It really is.

They market it as that. Buy our baseboard heater and we'll take the mining. By all accounts, what it wasn't, was productive.

Yep. Indeed. So yeah, the other challenge, and this is something that I found especially with my use of some of these more advanced models.

Now inside at Futurum, we use a lot of Anthropic models, Claude. The latest Claude tends to be extremely persistent, let's say. It's like a dog with a bone when I ask it to do something.

It will not give up. It will keep working on it. And it's funny because it's made the whole experience slower, and maybe a little less rewarding too, because I constantly have to remind it, let's do one step at a time.

Let's not try it. I don't need the full answer without having given you any of the context. Yeah.

Hold on there, boy. And I think that that's a sign as well, and what I'm seeing from a lot of these other models too, is that as you're talking about, Brad, the supposed reasoning models tend to spin a lot of wheels on the way to coming up with an answer. And that's one reason actually that their answers are so much better now.

Because, for example, Claude, if you assign it a task that it can verify, it will actually go ahead and spin up a tool or verify the answer or check the math. And I am all for it checking the math, but I'm not all for it checking the math before I even gave it the inputs that I wanted to use as part of this, because I end up intuitive a lot. Just choosing what you wanted and delivered.

Yeah, it's funny because, with some models like the Qwen 27 billion dense model versus, they have one that's a mixture of experts that only loads a smaller number of parameters into memory. But the dense models, a lot of people are finding that it's best to just turn thinking off entirely, for code generation because it's faster, cleaner, and you get better code. And if you've ever taken the time to go through all the reasoning traces from these current models, you will see a lot of dead ends and a lot of self-doubt that doesn't need to be there.

Maybe they need therapy. And also a lot of spurious web searches. That's the other thing that people are noticing.

I asked it to write a quick JavaScript for me, and for some reason, it's searching 4chan and Reddit. What is it doing over there? Those kind of things.

Well, I don't want it there. Who knows what it's doing. Maybe it was curious.

I don't know. The Earth is hollow. Let me go check 4chan out while I'm doing this task for Steven.

But I guess my question for you all is, what's the prognosis? What are people supposed to do in this situation? So, as I said at the beginning, essentially everybody now can have access to really incredibly advanced AI models, whether it's online or local, pretty mostly, as a service.

Some of these are even free. Some of these are even anonymous, and you don't know who's running it, who's doing what with your data. What should people be thinking at this point?

Well, what do you mean by people? Which people? Our listeners.

What should enterprise tech folks be thinking after listening to us all rattle on about how bizarre this world is? I think the thing that's frustrating for me is that we're constantly in a reactive mode with this. These models, they don't come out of nowhere, but there's just so many of them coming at us in rapid succession that it feels like we're constantly testing, evaluating, helping train them, that we're always on our toes.

We're always continuously having to adapt and train, and there's no firm ground between our feet. And that makes it very difficult to just really build any kind of strategy to have continuity in even to the project from quarter to quarter. If we're constantly having to change tools, learn new tools, evaluate them, figure out what works best, everything that you've talked about, right?

Learning all the quirks of all of these new versions of these new tools. And I think that organizationally, and we don't necessarily have anybody has the resources to do this. It's almost like you need a development group that focuses on evaluating, testing, sort of filtering through all this, and then everybody else.

Because if it becomes everybody's job to become an expert on how to adapt to the latest AI models, there's just no time left. And we're all going to get burned out. Which is already happening anyway, I think, to some extent, with just everything else.

If on top of that, we have to do this and keep up with this, it's just not sustainable. I'm sorry. It's not.

If I put my practitioner hat on for a second, I face this all the time, and Olivier is absolutely right. What I hate the most is that from any sort of running of this product, that from day to day, there's always a new challenge that I have to face and figure out. And a lot of them are down to simply how the model behaves.

Because as Steven said, these are coming at us pretty fast. But even in between the releases, there are changes that take place with those models in terms of how they respond. That's down typically to things like optimizing the inferencing costs that the vendor that's hosting them go about.

So you've heard the term nerfing a model. That happens sometimes when a hosting provider of a frontier model downgrades the quantization or the hardware that it's running on, and to such a degree that it now becomes less capable. And you would never know that unless you suddenly looked at your outcome, your output from your product and went, "Crap, that sucks.

" And then how do you go back and figure that out? That is the real challenge, and I hate that about AI right now because I want consistency. Like Olivier said, I want a product that works from day one to day 30.

If I don't make any changes to the product, I don't expect it to work any differently on day 30 than it did on day one. But that's not the case with AI, especially when you're using hosted frontier models. I will say, to their defense, sometimes the way they're managed is good.

I have been trying to migrate from a plethora of Gemini models. 1 flashlight, for example, for really lightweight tasks. 5 Flash for the in-between stuff.

And what a pain to manage that. 7 Flash now. I'm able to actually simplify that entire landscape.

So I have one model that I can manage by just changing how it goes about managing tokens, so the thinking and effort levels. And then I can use that across a lot of different use cases within this one product. So I'm very grateful for that.

All I want now is for them to not touch it for six months. Good luck with that. Yeah.

Yeah, how's that working out for you? We'll see. It's only day seven, I think.

Yeah. No, I just feel like we're just mice in a little maze and having to constantly learn how to spin the wheel differently to get the cheese. It's frustrating, and I would like to find a solution to this.

So in an organization, I think, that has the types of resources that they could devote to this, you could have a development group that basically that's all they do. " But you almost need to have people dedicated to this full time to be able to pass on their knowledge and their findings to the rest of the organization. And that could be a great idea for a business, a service, where you do this for SMBs, right?

And then just subscribe to that, and then you know how to optimize your stuff, and you don't have to figure it out internally. Brad? A great another subscription.

You've invented the analyst industry, man. Yeah, maybe that's where it needs to be. But honestly, there's a need for that in some way, shape, or form at this point.

There is. What we really need is for AI to be treated like software. " And those channels all run on very different schedules.

So if you're on the stable channel, what you actually are using and what you update to is something that's been proven, that does what it's supposed to do. Why can't models be like that? And furthermore, I wish that there were LTS models, like operating systems, where- Yeah ...

they would give you a guarantee. This model will be available and will function the same for X number of years. Man, wouldn't that be incredible?

Weird. As if nobody ever thought of that before. Well, unfortunately, we do have to run.

It's been a fun 30-minute discussion here on "Utilizing AI," just like it is every Wednesday. Thanks for joining us, both of you. com.

com for more of that. And do check out our other sister podcasts, "Agents of Dev," "Security Boulevard," and of course, the rest of the Techstrong gang every day at noon. Before we run, Brad, where can we continue the conversation with you?

com, but I'm also over on LinkedIn occasionally under Brad Shimmen. And for me, the same. Not so much on LinkedIn, although I'm there, just not very active on it.

But I am active on X still, so I'll never stop calling it Twitter. But I'm there occasionally when something important pops up. Excellent.

Well, it's good to know that there's some good content still up over on X. And thank you everyone for listening to the "Utilizing AI" podcast today. If you enjoyed the discussion, please do give us a subscription, a rating, a review.

You'll find us on YouTube or in your favorite podcast application. This podcast is brought to you by the analysts and experts from the Futurum Group, where insights meet AI. ai, the "Utilizing AI" YouTube channel, or the Techstrong TV app.

Thanks for listening, and we'll catch you next week.