Are AI Wearables Just Trojan Horses for Corporate Surveillance? | Utilizing AI Episode 33

The success of the AI industry depends on consumer demand, not just technical capability. This episode of "Utilizing AI," featuring Futurum Group analysts Olivier Blanchard and Brad Shimmen, focuses on the next generation of AI products, including smart glasses and non-traditional interfaces. Welcome to "Utilizing AI," the podcast focused on practical applications of artificial intelligence from the Futurum Group.

Every Wednesday, we explore news and use cases of the ways in which AI is transforming enterprise IT and the industries it serves. I'm your host, Stephen Foskett, President of the Tech Field Day business unit here at the Futurum Group. Before we dive into this discussion, let's meet who's on the panel today.

I'm Olivier Blanchard. I am an analyst at the Futurum Group, and I specialize in any kind of AI devices. So we're going to talk about that a little bit today, but anything from little wearables all the way to smart cars and robots.

And I am Brad Shimmen. I am also an analyst at the Futurum Group, and I look at data intelligence, analytics, and infrastructure, so all of the stuff that just sits where you never see it, behind the tech that Olivier writes about. So, as we were discussing recently, there's a lot of different products coming out, a lot of different attempts to try to find something that's going to find market traction.

And since we've got Olivier and Brad here, especially Olivier, I want to kick this thing off by talking a little bit more about some of the consumer-facing AI products that are out there. We've certainly seen quite a lot of attention focused on sort of the ChatGPT-type consumer-facing AI technology, but there's a lot more than that. We recently had WWDC, where Apple introduced, this time for sure, Apple Intelligence.

We've also recently seen the introduction of smart glasses from Snap, as well as a new, kind of under-the-radar, introduction of smart glasses from Meta. And this begs the question, what is the market for these? Who is demanding these?

Or is this more of a, let's say, supplier-led rather than a demand-led product? So Olivier, I want to start with you because this is your area of focus. Talk to us a little bit more about this new world of consumer AI devices.

Yeah. So that's a really good topic. And I think, in a way, it's a little bit of a microcosm of the greater AI gold rush, I guess, where we have this new technology, we have all this new IP built around AI, and the promise of, well, AI is finally here.

If you're a science fiction fan, you grew up in a world that has been influenced by science fiction. Everybody understands that AI can do some really cool stuff, and it would be great to have robots and to be able to talk to our cars and to have the cars talk back, actually, is really the more important point there. And so we have all these devices that are sort of like the interfaces that we have, right?

So it started kind of with our PCs and ChatGPT, and then our phones were able to do certain things and became more and more capable of handling some of those AI interactions. Obviously, anyone who has a smart speaker or an Alexa device or a Fire TV is also very well aware that they can have conversations with their appliances now and that it brings some value to our lives, right? You can ask what the weather is.

You can ask your Alexa device, for instance, or a Google device to search things on the Internet and to help plan your day. So there's some good utility there that we've already seen. And I feel very strongly about this, or at least up until now I did.

Now I'm not so sure, that the next iteration of AI interfaces would be the smart glasses, right? And I'm not talking necessarily about augmented reality this or mixed reality that. I don't really care if there's projection on the screen and if I have sort of this mixed reality experience.

My focus, for the purpose of this discussion, is AI glasses that are essentially the microphones, the speakers, the cameras, and the way that you interface with assistants and with agents using your own natural language conversationally with gestures, with pointing at things, maybe tapping on the side of your glasses. For instance, you're walking down a city and you see a building and it's beautiful and you want to get some information about that. Instead of pulling out your phone, you can just discreetly look at it and say, "Hey, what am I looking at?

" Or, "What is this tree? What is this plant? " And immediately, the AI is going to understand the query, look for the information somewhere, bring it back, and give you the information that you asked for.

Maybe ask for more context or the way that we interact with AI now, which is like, "Hey, would you like me to tell you more? " And all these things have a lot of value theoretically, right? And as the capabilities increase and improve, the idea or the promise, I think, of these AI products is that they will bring more utility and more value to your life, right?

They will give you more context. They will help you with-- It's just a great educational tool, if nothing else. It's a great navigational tool.

It's everything, and it's on your face, and you don't have to really touch it, and you forget about it, and it's sort of magical. It's the kind of stuff that 50 years from now, sci-fi writers would have either assigned to James Bond Right? Or to some kind of space technology.

And here we have it. Now, here's where I get a little bit negative or critical. I feel like we are squandering this so quickly.

We've only been really interacting with the closest thing to AI that we have, because I don't consider any of this to be truly AI, but we can call it AI, it's fine. We've only been interacting with it since COVID, essentially. It was 2020, essentially, that the AI revolution really started in earnest.

So it's only been a few years. And the form factors are nice. We still have phones that keep getting better and thinner, and the battery keeps getting more high performance.

We have smart glasses now, and Meta just released its own glasses, the Adventure, the Fury, and the Starfire Kylie edition, which are branded by Meta this time. It's not Ray-Ban, it's not Oakley. They have their own thing to keep the price low.

But essentially, they do the same things that the other glasses do. So we're getting more styles. We're getting more personalization in terms of the form factors, so you don't have to wear glasses that aren't necessarily your vibe.

But we're not really getting that much better functionality. We're not really getting more utility. It just seems to me that we accelerated really quickly to a point where the technology was very impressive, and then it plateaued.

And it just became like a lot of other technologies, it became kind of safe, and it became sort of like, well, this is what we have. This is what you can get, and now we're going to spend the next few years telling you what different colors you can get it in and different sort of skins you can get it in. But it doesn't really add anything to your life necessarily.

Right? The question that I start to ask is, why would I spend money for this if it doesn't give me something back in return and doesn't keep giving me something back in return every quarter or every six months with a new software release, with this new update, it's going to do more things like technology used to? And I'll add one more thing.

I'm sorry that I'm kind of monologuing here. There's a sort of a pro and con equation at play here. On the one hand, there's what the technology brings to us, this value of what can I do with this?

What can this help me with? What problem does it solve for me that this becomes indispensable to me, that I want it? You don't even have to market it to me.

I just want it. The other part of the equation, the other side of that coin, is what it takes away from me, what it takes away from consumers. And in the case of AI, in the case of smart glasses particularly, there are significant privacy issues that arise from this.

Not just from other people necessarily filming you or taking pictures of you and recording you without your knowledge or your consent, which is one big problem, and then using AI to alter images or audio of you and using that in all kinds of nefarious ways. But there's also the notion that when you have cameras on your face, the cameras can potentially record, and the microphones can potentially record everything you do, just like your phone does, but more effectively because now it sees what you see. It's not in your pocket.

You can't tuck it away somewhere. And so whether these devices can essentially learn from you and help an AI somewhere build a psychological and a sort of habit matrix of everyone who wears them, they could also be used to train AI without your knowledge or your consent to be able to perform tasks that you can perform. And we've already seen that some businesses use them literally to train, to have their human employees train robots to do the work that they do so that they can replace them in a few years.

And so that's the part of the equation that's taking away. And so my broad question, to come back to your initial sort of theme, or our theme today, is there's a tug of war between products that are built for consumers, which is how we used to kind of do it. Let's build something that people are really going to enjoy using, that's going to benefit their lives, that they're going to love using, and that's really going to be in service of the user, versus products that are sort of pushed onto the markets with a few bells and whistles that are cool, and the notion that if you don't have one, you're missing out and you're not one of the cool kids.

But ultimately, it's really sort of using the consumer to subsidize a data collection exercise that will ultimately benefit everyone but the user, or anyone but the user. And so now we're getting into the realm of Big Brother as opposed to Big Butler, where the technology is serving itself or is serving to take something away from the user, as opposed to being in the service of the user, and driving demand as opposed to being pushed onto the market. So that was just a lot of ideas.

It was a big answer to a very concise question. But that's what is bothering me right now is I don't see how a lot of these products are being marketed in a way that makes any sense for consumers anymore. I'm really glad we're talking about this, Olivier, and it is a big answer because it's a big question.

And when I try to step back from the little industry that you and I and Steven look at, and try to think about these trends in terms of why we're doing what we're doing and who it's for I agree with you in that I feel that what we're doing right now with these smart wearables is really an extension of what we've been doing, or merely an extension of what we've been doing for the last four or five years, and that is building these data collection and monetization devices, that I think present more issues than they solve in many respects. Not in all respects, because you don't have to look too deep to think about the Apple Watch as an example of a smart device that uses AI to detect if you've had a fall or if you're in atrial fibrillation and you need help. And we've seen many stories about people who have found that quite useful.

But at the same time, just as we're seeing with the way social media works, with the way our devices work continuously in interacting with our world and gathering the data about us, that it's not there to really help us plan out our day better. That is ostensibly what it's doing, for example, when you look at how AI is being built into our user experience for productivity software. That is the stated goal, is to make it easier for us to manage our busy lives.

But I fear that there is a great cost in doing that. And I will give up some of my privacy and security to know if I'm having a heart attack, but I'm not quite sure I would do the same to plan my day a little more effectively. And it seems to me that it's not just the people making these devices that are missing that, but also the consumers of those devices that are missing that.

" Is it a case, as you said, Olivier, that if I'm recording everything I'm doing in my reality, in my day-to-day life, I should say, if that is of benefit to me, how is it of benefit to me, and what are the risks to me of doing so? That's the conversation we're not having. Yeah.

It is so true that already we are living in sort of a corporate panopticon here. We worry about the government spying on us, but the government really isn't, for the most part, spying on us. It's companies that are spying on us, and to a greater extent than people are aware.

I was traveling with a group of people the last couple of weeks, and they, at dinner one night, brought up the whole, "My Apple and Amazon devices are listening all the time because I get ads for things that I was just talking about" question. And I tried to help them to understand that, no, it's not listening to you. It doesn't need to listen to you- Right ...

because you live in a data mining universe of signals. And so the fact that you get ads for canoes doesn't mean that it heard you mention a canoe. " Right.

These things are building a world where you're constantly being watched. Now, Apple, at WWDC, again reiterated that they're trying to protect the consumer. They're trying to be on the consumer's side.

Many people have criticized them for having a small-minded vision of AI compared to many of their competitors, or that they're late. But Apple tried to counter that by saying that they're trying to protect user data. They're trying to protect the sanctity and build a system that would allow people to get use of this technology without giving up the data to everybody else.

That is an admirable goal, but at the same time, Apple just recently revamped the App Store to use literally every signal, including pauses, cursor movements. They would do eye tracking if they could to build a customized app store pitch for you. They're drawing fire for including ads in more things, but certainly fire from me because I'm sick of all these ads everywhere.

And of course, the rumor is that Apple is also about to add additional AI-powered features and also additional sensors and capabilities to things that live in our houses, to our AirPods. Maybe there's an Apple clip device, that sort of thing. And all of this, I guess it's one of those things where, okay, if we take Apple at their word that they're trying to be consumer sensitive or have been trying to be consumer sensitive, at what point does that fall by the wayside in light of literally maybe trillions of dollars of potential revenue monetizing people and monetizing things?

I for one don't want to bet my future on Apple's good graces because I've seen that get burned too many times before. So the question is, are we entering a sort of a panopticon corporate world where everybody's watching us all the time or not? And actually, I'll throw one more thing in here.

I've been reading "The Murderbot Diaries" by Martha Wells. I strongly recommend it. They're really fun, short books.

They're really well-written. And one of the underlining themes of this entire series is exactly this, and I know that if you've watched the Apple TV show about it, you maybe enjoyed it, but you didn't know that that was one of the themes. In the books, it's very much the theme, that corporations are constantly spying on us and using everything and data mining and using every bit of information they can possibly get ...

in order to exploit everybody in the world. And I feel like that's the world we live in. So Olivier, I'll throw that to you.

Talk to me. Are we building the corporate panopticon of Murderbot? Yeah.

We've been building it. We've been living in it. We just don't have Murderbot yet.

So unfortunately, yeah, I think we're cooked. We're in it now. So, where we go from here is a whole other thing, and I don't want to get into dystopian cyberpunk world-building theories.

But we are living in a version of exactly what you were talking about in the early days. But to bring our-- And I think it's inevitable, right? I'm still hoping that we do a show that focuses on the flock cameras and actually just focusing on automotive a little bit, because even in upcoming vehicles, there's some new technology as well, that in some cases, like Brad mentioned with when you have a heart attack or something, you fall, having intelligent devices or intelligent appliances that can help you survive those incidents and call help faster is-- These are literally life-saving applications for this.

So there are similarly valuable applications of this technology that is intrusive in vehicles and sort of like the flock camera network, potentially. Unfortunately, those are the exception rather than the rule. They will save a few lives.

They will help solve a few crimes. But for the most part, they're misused, and are definitely not a net positive for users in general. But I think the question that I have, because the hardware vendors here, whether it's Meta building glasses or Ray-Ban building glasses or the hardware vendors that provide the hardware platforms, right?

So the chips, the processors, the memory, all of the things that actually make this work, the silicon, I don't have an issue with what they're doing. I think that they're enabling amazing technologies. The problem is in how they're being enabled in the real world.

And so the question and I guess the problem that some of these tech companies, these tech giants have to solve for, assuming that they want to solve for it, is I'm not sure that they realize just how negative an opinion of them is sort of forming outside of their headquarters with all these technologies. And I don't want to single out anyone in particular, but I think that the real threat for an Amazon, the real threat for Google, the real threat for a Meta is that this surveillance state and this panopticon of surveillance by these corporate entities that collect data from us and promise us these wonderful products that are going to, the one pair of glasses to rule them all, are actually Trojan horses of data collection and surveillance. The game is up.

I don't think that people will continue to accept this unless they're forced to accept it. And we're seeing this with the sort of adoption with AI, which is being pushed in the corporate environment. Like you have to use AI, right?

Whether you like it or not, because it's the new thing, even if it doesn't solve a problem, and even if it creates more problems than it solves. It's the same thing for this. And the one thing that Apple, and I'll commend them on this, they were late to AI, and it had nothing to do with privacy.

But they did, I think, make a choice to put privacy first and at least make that part of their branding and say, "Look, we're going to be different from everybody else, at least outwardly, and at least for now. We're going to do the things that Google said it would do, and then didn't. We are going to talk about privacy at least, and build privacy into our business model more so than the others.

" And it's sort of like this contract that they have with consumers, which they need to. It's a major differentiator. And these other companies are not learning from this.

And I think it's going to come back to bite them. Because at some point, for economic reasons to begin with, because the economy's not doing that great, and so people aren't rushing to buy things they don't need, I think that's definitely a brake on demand for all these technologies. But the bigger brake for demand for these technologies is that if I buy this, I'm just inviting more surveillance, and I'm inviting more intrusion and risk into my life.

And why would I do that? And by not focusing on this, and by not addressing this directly and making it part of their corporate identity and branding, they're missing a critical piece of this, which is trust. And if consumers lose trust in your company, it doesn't matter how good the tech is, they'll just go somewhere else, or they'll just not adopt your technology at all.

And that's bad for everybody because then IP doesn't really move forward. We're not building things for consumers anymore. We're not building things that improve the world.

And so demand essentially dies on the vine. That's why you have the Kylie glasses, Olivier, so that you can build that demand for fear of missing out. Yes.

I like what you're saying there because going back to what I was talking about with that conversation, it is contingent upon us as consumers buying these products to think about that trust and with whom we are making that contract when we buy those glasses. And For me, the wake-up call for all of this has been slowly building. The frog boiling in the pot is the realization that even if you are privacy-focused, that the opportunities for that attack surface to expand beyond your control are very easy to get out of control without you even knowing that that's happened.

And you need look no further than the VPN marketplace as a good example of that. And as the FBI even mentioned in a letter they wrote a year or so ago, if you use a VPN, yes, your telecom provider can't spy on you, but the provider that you're contracting with might be selling your data on. And how do you control that?

And think more recently to the face ID that we are seeing both legislative and also pressure from the tech community to implement. And I think Anthropic just announced that you're going to need face ID to use Claude, coming pretty quickly here. And did you know that Anthropic is not going to be managing that data and those data retention rules?

They have contracted out with a third party to manage that. Do we think we have trust in that third party as well as Anthropic? No.

No. Well, it's funny that one of the things Olivier said, and what you just said there, it really kind of, in my mind, the core question, again, back to the central tenet of this whole discussion is, who are we building this for? Yeah.

And are we building this because consumers want this, or are we building this as a way to force adoption of something that the companies want, that they want us to have? I think that Google proved decades ago that the data is the ultimate value of basically everything in the mass market. Meta has built a massive company around that concept.

And now, with these real-world interacting AI systems, essentially, that it's getting even more data, providing even more value to these companies. I guess the question, it reminds me of televisions. If you've bought a television recently, you might have noticed how cheap they are.

The reason they're sold at below cost is because, hello, they're spying on you. They have essentially a system that samples whatever's on the screen, figures out what it is. And so even if you don't think they're spying on you, they are.

And then they sell that data upstream and use that. Fox just bought Roku. What they bought primarily was ad-supported streaming, but also a new incredible source of consumer data that Roku had built.

Roku basically is, you look at their numbers, they were making more money off of the data from consumers than the hardware that they sold to consumers. And I think that that's fundamentally their business plan. And if we look at a lot of these systems, a lot of these things, again, we're not here to throw stones at Meta.

We're not here to throw stones at Snap or anything. But Meta and Snap are consumer data companies, ultimately. That's really their product is consumer data, and they have built something that is going to generate just tremendous volumes of real-world consumer data if it sees uptake.

And so it would be in their best interest to price these things as low as possible, maybe even below cost, to get them into consumer hands to build more of that data. That's, I guess, the sad world that we're heading to, right, Brad? We're already there.

Yeah. You mentioned a panopticon, and for those of us that really worry about that kind of thing, which I think is everybody, if you've ever found yourself in a conversation with family or friends or just talking out loud, censored yourself because you thought, "Maybe I shouldn't say that out loud," you are already in the panopticon, and you've already changed your own behavior to adapt to a world in which privacy is something of the past. And I think that that is true, and it's something that I think we all need to come to terms with and try as best we can in our consumption habits, in what we choose to buy and who we choose to partner with as somebody we buy our products from.

If I can look at Sony or LG and say, "Okay, I believe that if I turn Wi-Fi off, you're not going to be selling my data because I don't want to do that, and that's why I'm buying your TV," then I think we stand a chance of at least building around each of us according to our own ability and desire for privacy, security, and capability, and the balance thereof, that I can maybe go with those TV providers and feel good about it. And that's why I do as well, guys, applaud Apple for their approach to AI in particular, and I wish them luck with that, and I think that history will bear it out. And as we've seen with Apple, in particular in dealing with requests for data issued by governments, whether leaked or purposefully put out, that these companies do understand what you're saying, Olivier, that if we stop buying their products, they're going to suffer, and that they need to build a better relationship with us.

So I'm hopeful. I really am. I'm hopeful that we can, each of us, build relationships with companies that we trust and that those companies will likewise try to build trust with us.

I hope so. I feel like the model has to change, though. First of all, they have to address the opacity of what they're doing.

You put in our chat that you are always the product. And so, if you treat Meta glasses the way that we look at how Facebook is essentially just a data collection experiment that is addictive, so we want to use it. And so that's sort of the value exchange.

We get to enjoy the addictive nature of these products, and companies like Meta get to understand us a lot better than they ought to without our knowledge. I think that with smart glasses specifically, the contract has to change. I think that on the one hand, the opacity has to become much more transparent.

It has to be like, "Look, we're Meta. We're company XYZ. " Right?

" Okay? Everybody understands that. But even having that dialogue, I think, helps.

But also giving the consumers the opportunity to either opt in or opt out, essentially treating your customers respectfully as partners in this endeavor, where you're like, "Look, this is what we're trying to do. So here's a dashboard. " Basically, sort of like any dashboard where you check for your privacy, and you decide who can see your things and what has access to what.

Same thing. Make it really easy on startup. This is what I'm going to allow.

" Right? "We'll help you monetize this. " All the training that we do as consumers for free with the devices that we bought with our money recruits these customers into the practice of training AI and using these devices to essentially build better products in the future.

I think that's a much more ethical approach. I think that's the kind of approach that would help consumers trust a company more than the others. And I think that's great marketing because, as we all know, we do business with people we like and with companies we trust.

And that is ultimately the foundation of loyalty, right? And I know that I'm talking about marketing more so than product development, but I feel like if this is going to continue to work for these companies, and if AI can turn the corner on this momentum of negativity that the public is really starting to feel about AI in general, they need to do something, and this is a good place to start. Showing that AI can be ethical, that these companies can be ethical, that they can work with consumers to build a better AI future as opposed to feeling this tension of sort of like we're buying things that are going to spy on us, and there's no way out, and it's only going to get worse.

It's on them to change that. And other than Apple right now, I can't think of any company that's doing that or is doing that well. And it's too bad because I think that there is a huge opportunity here.

This technology is really incredible. And people actually have good feelings about many AI-adjacent products. Again, my travels with people who are not technologists showed a lot of people very enthusiastic about things like Google Lens, for example.

I want to know what that is. Google Translate. These are AI features, but they're features that consumers like and consumers trust and consumers want to use, as opposed to the overwhelming sort of negativity and dread even about the robots coming for our jobs and that sort of thing.

And I feel like a lot of this messaging has been effective at saying, "Hey, Anthropic and OpenAI are absolutely killing it and making this amazing technology," but they've been really negative in terms of turning the world against them. And I'm not sure they realize that, that people are genuinely very angry and upset about this technology, that honestly they don't have to be because frankly, the technology could be used positively. I would love to see that world, Olivier.

And Brad, thank you so much for this conversation. I hope that we can continue this. I do want to dive into the question of cameras and vehicles and that sort of thing.

Maybe we can do that in the next episode. But for now, let me check in with you both. Brad, Olivier, where can we continue this conversation with you if we want to in between episodes?

Brad? Yeah. com.

And I actually am going to be not traveling for the next two and a half weeks, which I don't quite know what to do with. So we'll be very interested in trying to put everything that the last six or seven weeks on the road, putting it in some sort of perspective. So I would really encourage everyone, if you're interested in AI, look up Olivier and I and what we're writing about right now because it's been a busy travel season, and a lot happened.

Yeah. And for me, same thing. So I'm on LinkedIn, although I'm not on LinkedIn very much, to the chagrin of many of my colleagues.

But I am still on X, and I occasionally post angry tweets or posts on X about technology. Obviously, you can also find all of my writings and reports and such with the Futurum Group and Futurum Research websites. And I do have some new research coming out, I think, next week or this week.

Especially if you're logged in and you're a client, we have a new IT decision-maker survey that focuses on intent to purchase for PCs and all sorts of other intelligent devices in the enterprise. And it's yielding some really interesting insights about pricing and growth curves of very specific devices that everybody uses every day. Yeah, let me just reiterate that.

That stuff is just incredible, the results of some of these surveys. I've seen the previous one. I haven't seen this one yet.

I can't wait to see it, Olivier. Really incredible insight into, to the point of this episode, how people are feeling about this technology and what kind of demand there is. So thank you so much for sharing that with us.

And thank you everyone for listening to this episode of the "Utilizing AI" podcast. If you enjoyed this discussion, please do subscribe. The best place is on YouTube, but you'll also find us, of course, in your favorite podcast application.

And please give us a rating and a review. This podcast is brought to you by the analysts and experts from the Futurum Group, where insights meet AI. ai, the "Utilizing AI" YouTube channel, or the Techstrong TV app.

Thanks for listening, and we'll see you next week.