Edge-Based AI Sensing – Mark Hanson, Sony Semiconductor

This is texturing TV. Hey everyone, welcome back to techstrug TV. Continuing with our show today.

Our next guest is Mark Hansen Mark is with Sony semiconductor. And as you can tell from the Sony logo there that's pretty recognizable logo. It's affiliated with the Sony whether you're watching TVs or whatever.

You probably familiar with Sony. Hey, Mark, welcome to Tech strong TV. Thank you.

It's great to be here. Great to be great to have you on so Mark I as I say, I think all of our audience of course familiar with Sony but let's talk about Sony semiconductor, maybe something we're not familiar with. Yeah, actually I'm working with the team in Sunny semiconductors that is doing some unusual development work.

Actually, we just recently launched but we're we're not only selling chips these days. We are developing a sensor-based platform that makes Visual AI easier lower cost more effective to implement than the current, you know. Traditional what we think of moon launch activities that folks are doing with visual AI today.

So and we call it atrials. Oh, that's the Atrius. Got it.

That's the atrious part of the platform. Yeah. so yeah, no just say so you're probably pretty familiar with what semiconductors does as a whole we're in about whoops.

Let's let's put it in the right place. So we're at about 60% of all the world's cell phones these days. So we ship billions of image sensors to on a mobile handset folks camera folks Etc.

So we kind of democratize picture taking for the world, right? That was that was our sort of claim to fame in the semiconductor side. But now we want to try to do the same thing for visual AI, you know businesses that want to get real world data into their data stores and make better decisions because of things that are actually happening in the physical world not to certainly the things that are in their data centers.

So we're trying to figure out effective ways to do that at scale. So that's our big objective right now. Got it.

So Mark Luke. I'm sure when times does their man or person or thing of the Year. This year AI is going to be the winner.

Right. It's like AI this chat GPT and Ai and And all of this is just blowing up. You know, I I was I told you I was in Las Vegas this week one morning.

I turned on I think was Good Morning America and they've got a whole thing on AI and iterative, you know chat and and I'm thinking to myself when I start seeing this stuff on Good Morning America. I know we're in hype cycle. And you know and and beware the hype sometimes.

I want to separate what you're doing from the hype. Okay, right and and so what I'd like you to do and if you can what do we mean by visual AI? How would you describe or Define that for our audience?

So we we have some context here. So so the traditional kind of visual AI is often done by placing lots of cameras within an environment. Somebody let's say a retail store as an example and they're usually using a lot of streaming cameras and feeding that all inside servers and then running AI models that will do people recognition or particular stock recognition or you know, identifying traffic patterns of people walking or check out lines or things like that.

But those are pretty cumbersome. They have lots of challenges. You got to deploy lots of dumb streaming cameras.

You got to lay lots of CAT5 cable you use a lot of power when you think about you know, I got a thousand cameras. I got 50 Edge servers each one of those at 1500 watts, you're burning through a lot of power. You got some latency issues because by the time you're actually, you know, identifying an object.

It may have moved especially when you're talking about like Fast removing object. Acts like in a factory automation scenario, or maybe you're doing Smart City and cars are moving around. So it's a little more difficult to develop those models and and output those and then the other big thing is privacy, right?

I mean, hey, I got a thousand streaming cameras in a public location either in a smart city or in a you know retailment. We got lots of people you're streaming a lot of video around. So what we sort of looked at as we said look AI developers are trying to figure out how to do a better job of More effective deployment lower costs better time the market.

So we started off with as we built this teeny little chip. It's about a it's almost the same sizes the chip that are found in these cell phones. And what we did was we used some of our core technology where we move a lot of the logic onto the back side of that chip.

We actually stacked a little AI processor in it in the back side of that pixel format. So we've kind of moved all of those things that you normally would have to do with that servers and cabling and all that to the back of an image sensor. So we kind of extended what I would call Aji to the extreme Edge.

And by doing that we solve a lot of those challenges that I just kind of described for you. One we can make teeny tiny little cameras like like this. And these cameras are 12 megapixel sensor cameras this out.

What happens to be made by Lucid but this particular camera actually doesn't send any video or images. It runs a tiny ml model on the back side of that image sensor. And this is all you need to identify an object and determine what that object is and you merely are sending the better Json or or bounding box information or the confidence level through the network.

You're not sending heavy video. You can do these cameras on Wi-Fi or run them on battery if you want. So there's no heavy lifting.

You don't have to update your network. Um, it's super fast because there's no there's no distance between the Chip And in other words the pixel inside of the Chip And the processor side. So it instantaneous within each 30 second frame.

We're identifying an object and outputting that data. So we're able to do things very very quickly and optimize. So like moving cars you want to put a bounding box around where those cars are moving.

It's instant. It doesn't have any problem keeping up with it as opposed to some thousand dollar GPU CPU enhanced smart, smart cameras will struggle with that the Carl move, they'll put the bounding box and it's already behind the car just because it has to go through a whole processing layer a bus structure Etc. So that's sort of the core enablement that we did.

We did this development on the courtship itself and provide sdks for OEM and Camera guys to build into their equipment into their cameras this kind of capability. So if you can kind of imagine this it's like a visual iot sensor. The output data coming out of the sensor is looks more like, you know shocking vibration data or temperature data or coordinates data.

It's not sending images at all. So the nice thing about that is it really manages private privacy actually at the very edge of the network. So there's no video streaming around the environment that you're working at.

It's actually just processing that video One frame at a time looking at it determining if it identifies an object puts the confidence level bounding box coordinates, whatever requirements and that's all it's sending like a text stream. That's it. So it's it's a little unusual in that there isn't a you know, there's lots of different effective ways to work with CPUs and gpus but to put it on the back side of the image sensor requires a whole platform to enable that capability and our long-term vision is we want to democratize this visual AI capabilities.

So we built a simplified platform that actually it's an Enterprise level platform that sits on top of Microsoft Azure who's our strategic partner and basically through a console through user you can easily just take like a QR code put it in front of the camera and it'll automatically connect the camera to the network to as your to atrials and then you can control deploy models to the camera identify where the camera is understand the status how much data is flowing through it, etc. Etc. Those types of capabilities, so it's not unusual for you know, like an IT manager to run that.

Capability as a normal it manager deploying equipment and managing it over the network. So that in a nutshell is kind of what Sony Sony's working on from the semiconductor side is we're trying to build out a sensor platform. And today we're doing it with RGB cameras book color cameras, but eventually we could do it with swear cameras.

We could do it with ir cameras. We could do with polarized cameras. We could even do them with like time-of-flight enabled solutions that provide depth information to do determinations as to what's out there.

So that's that's what we're trying to do as a team. Yep. So, let me let me dig in here a little bit and see if I can't make this.

I mean I have questions. I'm sure the audience would you sure so first of all that camera you showed from Lucent? Yeah, it's Lucid manufactured but it's running this new Sony.

Let's call it visual AI on a chip or visual ai ai on a board kind of thing. Right? Yeah.

It's running that other than Lucent other people making these or plants for other people to make yes, we're we're in discussions with lots of other folks right now to do make different types of devices plus we provide sdks. So let's say somebody that's doing maybe visual inspection and Factory and wants to implement it as part of their their equipment manufacturing equipment. We could give them the capabilities to do that as an example got it and now and the beauty of it is because it's all on a chip or on a card, whatever you want to call it.

That you you've removed the need for the edge and all of that background. Computing power which not only added a lot of money to the equation but also added a lot of latency and time. Yeah.

Well latency is time to the equation and so we've cut that out as well which just makes the whole thing damn near instantaneous. Yeah, we call it virtual instant. Yeah, I recognition.

Yeah, it's it is that quick and if you think about it the other the other thing that's kind of interesting is, you know, we've got a couple of Companies that have been looking to deploy a couple thousand cameras like a few retailers and what they want to do is they want to do stuff like on shelf availability, right? They got they've got scenario. I'll give you this interesting scenario.

So this this retailer At an issue where they they did a whole deal with one of these big consumer goods companies and spent all kinds of money promoting it marketing and getting inventory into the stores Etc and and a month later. They're looking at the data the POS data the inventory data. We got the inventory there Poss.

We ran all these ads. We didn't see any movement. There was no impact to this effort that we put out what the heck happened.

Well in this particular case what they didn't realize was the merchandisers remember put the inventory in the right place in the store. So you didn't have any way to get him a way to verify it. Right?

There's no input about what's physically happening in the store. So these are the types of things that we want to bring to businesses to help them and we're doing that through Partnerships. We're we're doing it with Microsoft.

We're we've got a lot of isv Partners who are Building Solutions that don't have visual data as part of their Solutions, but we've developed sdks to enable them to quickly add this kind of capability to their Their solution so they can drive that that kind of awareness and another, you know another way that this is more. Sustainable, if you will green, whatever you want to call it is that you're not actually transmitting visual data. You're not transmitting a picture you you're reducing the picture to diffs to text and just you know, which is a lot of it.

It's a much easier uplift than sending out a series of jpeg or whatever, you know format and and that makes a beautiful I mean, look, I'm listening to this saying wow. This is blue sky, right? This is truly a green field opportunity.

I haven't even thought of all the ways we could use this. Right. I think it's very kind of scratch the surface.

Yeah, and what's real interesting is that that's a great point out and this is a little bit of a paradigm shift. You know, you look at you. Look at folks like at Stanford's AI lab FIFA Lee's lab or you look at mits AI lab and you know there they've been taught sort of the traditional way of thinking about visual AI because they didn't have this kind of Court technology capability to enable this.

So so one of the things that we're doing because to your point is exactly right, there's so many potential opportunities. So what we did was we partnered with Microsoft and developed a lab scenario in Asia in US In Germany in China, and basically these are both physical Labs where you could where we're a AI engineer or or a company that has a particular solution. Like let's say a retail automation solution or a factory visual inspection solution and they want to get the stata.

Well, we have as a free program that can enable them to spend a week with Sony Engineers Microsoft Engineers either physically in the lab or virtually we give them equipment. We give them some of these cameras we can access to atrials and Azure and they can run a one week scrum to prove out that particular idea or use case that you're referring to to figure out whether or not this is a usable capability for what they want to try to achieve we have those kind of services available because it is so new. We need to get a bunch of people trying these kind of new methodologies of doing this.

well Look, I I think this is some great technology. That may wind up with some, you know, killer apps, right it always takes a killer app to make these things Rock and you know, I don't know if we've even discovered the killer app for this yet. But this technology certainly gives you enough to play with right?

Yeah do some interesting things and we'll see where that goes. Mark for isvs or companies who want to Maybe get involved in this where where would you suggest they go? Well, number one.

You could go to Atrius. You look up Atrius on a Google search go directly to the website and you can you can effectively get more information that way the other the other way you can do it is we have an iot Labs link where you can apply for a application to run one of these one week scrums which will give you that information as a link and you guys can anybody who's interested just fill out a quick 10 minute application and then we'll we'll just review it give you a call and arrange to to do a quick scrum and test one of your use cases be great. Absolutely.

Hey, you know, I should mention for people who are watching this, you know on live texture on TV on on YouTube or Facebook or LinkedIn or whatever if if you come over to Tech strong dot TV and watch the the on-demand version of this in the notes Mark, we're gonna put links to all of these including including a presentation that you've done that kind of really helps kind of explain this a little bit deeper as well. Yeah, give some great visual examples to explain, you know, the high privacy the low power consumption the ease of deployment. It's it's quite helpful to actually visually see what's going on.

It's pretty neat. Absolutely. Hey man, this sounds like a new breakthrough from the great Folks at Sony who always have, you know, great technology anyway, so thanks for coming on and telling us about it man.

It's exciting stuff. I'm interested to see where it goes. Thanks, Alan.

We're great being with you. Great talking to the talking about what we're doing and love to hear from your viewers about what they want to try to accomplish who's in this kind of tech. It'd be great.

Well, it's up to you viewers out there do it. Sony semiconducting the important name to remember his atrious a i t r i OS go check them out Mark. Thanks for coming on and telling us about this great new technology.

It's good to see this some real stuff beyond the hype around AI. Let's see where it goes. Thank you.

All right, we're gonna take a break here on Tech strong TV. We'll be right back.