Physical AI Needs Operational Context, Not Just Robots

Hello, and welcome to the latest edition of the Techstrong Leadership Insight Series. I'm your host, Mike Vizard. Today, we're with Sanhansu Ga, who's senior vice president and deputy GM for research and development at Hitachi America.

And we're having a little chat about, well, where is AI going to show up in all these physical environments that we're talking about because, well, it's just not robots. Hey, Sanhansu, welcome to the show. Yeah, thanks, Mike.

Glad to be here. It seems like everybody's obsessed about physical AI and robots, but the reality is maybe a little more nuanced than we're starting to see AI show up in everything from managing our trains to traffic to all kinds of interesting use cases within factories. What's your assessment of what's going on here, and where are we going to see the impact first?

Yeah. So I think when many people think about physical AI, they look at the physical embodiment of AI in terms of robots, humanoids, and drones, and that's a fair assessment. These entities are going to be part of the physical AI.

But from our viewpoint, physical AI story is much broader. When you look at AI agents like the AI models from the frontiers lab, they can do incredible job. But they have not seen your factories.

They have not seen how railway networks work. They have not seen your energy grid, how it operates. They don't have the tribal knowledge that comes from operating these infrastructure pieces.

So from our viewpoint, the understanding of the actual operational context when it comes to these different infrastructure pieces is very important, and that is the gap that we see when we talk about physical robots and drones. They cannot operate in isolation as isolated IT gadget pieces. They have to really be grounded with the operational context, and that is the broader story, and that is the broader theme we are working towards.

So where do I get that operational context? Where does it reside, and how do I expose that to what probably is going to be a series of AI agents? And that could number, I don't know, there could be hundreds of them, and they all need some level of context, right?

Yeah. So think of operational context as the meta prompt for your AI, right? When you work with ChatGPT, you give a prompt, and there is some system prompt that it acts upon.

But when it comes to, let's say, a factory, now the AI agents, they have no idea what your factory looks like. Of course, they have some idea how, let's say, automotive plant operates, but they have not seen what are different pieces of equipment, how they are related with one another, what are different pieces of equipment, how they are related with one another, what are the operational constraints, what are the guardrails, what is the maintenance record, for example. What is the tribal knowledge around operating these different pieces of infrastructure?

So in order to build this operational context or the context layer, you can-- Of course, just like enterprise agents, it's mostly RAG. You run it on the documentation that you have. But there's also, oftentimes you'll find whatever is documented, it differs from the real world how things work, right?

There are always nuances. So this is where you want AI agents to sort of scan through different data sets, create their own understanding, attach a confidence level, how confident they feel about particular understanding of some aspect of your operation. And wherever you see a gap, you loop in the human expert.

This is how we sort of build the contextual layer and keep it, like avoid making it go stale, right? So it has to evolve with the time as your operations evolve. The thing about AI, and particularly generative AI, is that it doesn't seem like they're designed to do the same thing the same way twice.

And a lot of these physical AI workflows are deterministic in the sense that they're supposed to be done the same way every time. So how do we kind of find a balance to use the generative AI capabilities that are probabilistic in a deterministic environment? That's a great question, and I think that really points us to the what is the right architecture for your physical world, right?

The deterministic side of infrastructure, it gives you the right guardrails, interlocks to make sure nothing goes out of the order, right? You need that part as the overarching sort of execution layer. But within that layer, you need to figure out, based on your specific use cases, where you can bring the desired reasoning layer that generative AI provides within your deterministic infrastructure.

I'll give you a simple example. If you look at automotive plants, right? So typically, when they produce a product, it goes from one process to the next one and so on.

And at each process, there are certain measurements, and currently we apply more like a rule-based logic to decide whether this product goes from process one to the next process. Now, when we bring in the generative AI piece, we can remove these rigid rules where you just look at rules-based logic to figure out whether the product should proceed to the next process or not. So the overarching execution is still deterministic.

It defines how things move. But within each layer, we can have that flexible reasoning layer to give the desired sort of connect the dots across different operations and bring more richer insight in terms of foreseeing potential issues that you may not be able to see, bringing more operational efficiency. What is your assessment of what's going on here with all these AI agents that recently went, quote-unquote, "rogue"?

Because we don't seem to understand whether or not, is it really the AI agent, or was it just the fact that we didn't have, to your point, the right controls in place in the first place? Yeah, that's another thing where I feel we are not paying enough attention, right? So there are essentially two things at play here.

You have the frontier lab models or open weight models. That's one side of story. The other one is the contextual layer, right?

The right context. Now, oftentimes, we are looking at whether we can trust the frontier models or the open weight models, and that's a fair thing to worry about. But I think the thing that gets overlooked at is the operational context layer.

Now, just imagine if someone tampers with it, what will happen? Your frontier AI models will be reasoning with this tampered context layer, and you will get outcomes which, at the minimum, are not producing optimal outcomes. At worst, they may lead to some safety or some critical issues.

Similarly, it's not just someone tampering with the context layer. What if the context layer doesn't evolve with your operation? It goes stale.

You'll be making decision based on information which is not up to date, right? So there has to be very thorough thought process paid into how you bring a combination of the model, pick the model. Again, on the frontier model itself, right?

When we talk about the paradigms are cost versus efficiency, right? What you can trust. Token usage is becoming more and more critical.

You want to see where the next dollar goes, right? Where to dedicate it. But when you work with open weight model, you also need to worry about the source of those models, right?

So yeah, I think there are a lot of nuances in terms of ensuring right outcomes. And on industrial side, the stakes are much higher, right? You are connecting these pieces of IT technology to control or manipulate the physical world, right?

So which you can't undo if things go wrong. So the physical world is not static. So how will we train the models and the agents, once they're deployed, that there's been a change in the environment and that somehow or other that needs to get incorporated into whatever workflow we're automating?

Yeah, that's an interesting question, and there are two different philosophies, right? So the one is you can bring these models, you can bring the smaller version of these models and try to train based on your world's knowledge, whether it's a factory or railway operations. But the problem with that is, as you said, things change, things evolve, right?

So what you will end up doing is you have thousands of these models, you're trying to sort of maintain them, evolve them over time, which is not very economical or practical. Another approach which I think has more promise is don't touch the model. The frontier labs are working on it.

These models are advancing every few months. Bring the latest capabilities of this model and pair them with the operational context. Of your ground operations, of your work, right?

Now, this operational context layer, that is the one that needs to evolve. And the way you can implement it, again, you can have these AI agents which are continuously scraping the data that's generated by on the production lines, on the maintenance records. They are the ones who bring in the subject matter experts as and when needed to make sure that the context layer never goes stale.

Right? So if you focus on that aspect, then you'll be set for success in future, yeah. We also know that there are a lot of different kinds of AI models these days.

So when we talk about physical AI, everybody seems to be talking about generative AI, but there's these world models. So, when it comes to physical AI, is the world model more important than a generative AI model? Or what's the relationship between all these things?

You need both of them, right? So if you look at world model, these are essentially your-- Basically, they provide the rich synthetic experiences, right? So they allow you the ability to simulate something in the physical.

And before your AI is connected to the physical world, you are able to sort of run multiple hundreds and thousands of what-if scenarios to catch any issues before you connect these outcomes to the external world, right? So the way I look at it is when it comes to physical AI, there are three, four pieces, right? You need the frontier generative AI models.

You need this contextual layer that captures your business and operational context. And the third thing is you need this simulation environment, right, which can generate synthetic, very rich experiences mimicking your operations. And you need an entity that orchestrates all of these different pieces of technology.

And each piece on its own may hallucinate or may cause issues. But you need a layer that ensures that the final outcome can be trusted. It will produce useful outcomes.

It can be audited. It can be rehearsed. It is traceable.

You need to ensure all of that before things get interfaced with the physical world. So yeah, it's not this or that. You need both of the capabilities.

So what are you seeing people doing out there these days that just kind of makes you shake your head a little bit and go, "Folks, we need to be a little bit smarter about this," when it comes to physical AI especially, but I think we're all in a constant state of experimentation here. So what have you seen? Exactly.

I'll go back to how we started this discussion. So physical embodiment of AI, the robots, they are getting the media attention, they are getting the VC attention, right? If you look at the investment pouring in since last year.

But again, if you really want to scale those experiences, bring the robotic infrastructure into your operation, you don't want them to end up as isolated pieces of infra, right? You want this to be integrated into your workflow. You want this to be integrated with the operational reality of your environment, right?

If you look at our Hitachi Rails factory in Maryland, we have deployed robots and drones. And more important than that, what we have done is build this foundation layer that connects the business context, operational context, and provides this contextual intelligence to these pieces of hardware so that they operate as one infrastructure. And that is something I think we need to pay attention.

And I'll also mention, repeat myself here, the security aspect, right? You need to ensure, because everything you are banking on is the operational context layer. You need to ensure this is safe, secure, and is kept fresh as the time evolves, your operations evolve.

Well, speaking of security, what is the correct approach here to securing all these things? " And there's no shortage of ideas now, it seems, for securing these things. But in the context of physical AI, how do I approach that?

Yeah, for physical AI, again, the stakes are much higher, right? So since we are dealing with the physical and critical infrastructure. The way to secure, again, I think first of all, you need to have the right team set up, right?

You need folks from the IT side who have been involved with the enterprise agent architecture infrastructure, bring the best practices from there. But then also you need to have folks from the OT side, the operational side. They need to have a seat on the table to work with the IT team to figure out how to build this operational context layer, how to keep it fresh.

And maybe you need to think of ways-- Just imagine, if someone attacks your infrastructure and they get access to this operational context layer, they have the entire blueprint of your infrastructure. Right? So this becomes a really, really critical piece of technology or a stack that you need to secure.

There are many, I think, things that we can borrow from the cybersecurity world, right? Like not putting everything at one place, having more fragmented approach in terms of how we implement this contextual intelligence layer. But then there is a trade-off in terms of latency.

How do you process data? How do you ensure that the entire thing is not available to the outside hackers? So again, yeah, it's a very, I think, interesting topic to be discussed, and it's an evolving story.

We are still sort of figuring out what is the best way. But having the right team set up is really critical. Will these physical AI systems wind up interacting with each other?

And so, it seems like nothing in the physical world, at least, is isolated from the rest of the world. Maybe with a couple exceptions here and there. It could be the secret recipe for Kentucky Fried Chicken is probably not readily accessible.

But everything else seems to be interdependent and interconnected. So, when there is an issue or an incident, how do we figure out the cascading effects? Because, as one wag once said, "It's one thing to be wrong.

" Exactly. I think that's another important question. And the way I think about it is, if you look at, let's say our current operation, I can go back to the factory example without robots, right?

You have the workforce. On a particular day, some folks don't turn out, or you have a new staff who doesn't know how to operate things, right? So there's a certain way that humans work and sort of figure out how to bring the desired flexibility to the operations.

We'll have to think of the similar layer when it comes to bringing, once we have these different physical AI hardware pieces, robots, drones, et cetera, in taking care of the infrastructure. You need to build this skill map, right? So what are the different capabilities of each entity?

What they can do, what they can't do, and what are the jobs that they can do on behalf of human workforce, for example. Right? So you need to have that sort of inventory of skills, and then execute it dynamically based on, again, you want to map it, what task is critical also, right?

So it's not just about who can do it, what could be the outcomes if things go wrong. So this is a broader sort of issue that needs to be looked into. On the same topic, I'll say another thing that's challenging is if you look at consumer-side experience, when you unbox a robotic vacuum cleaner, all you have to do is connect it to Wi-Fi, and that's pretty much it.

It can sort of discover its own world. We have not reached that level on the industrial side. There's still a lot of SI, system integration, work involved.

There are a lot of human stakeholders involved to make it really scale and work. So these are, I think, a lot of things are interconnected. We are heading in that direction, but yeah, it will take us some time to get to a point when we start thinking about these entities talking to each other and sharing stuff and filling the gaps, et cetera.

So ultimately, what do you think the, quote-unquote, man-machine interface is going to look like in the future when we have millions of AI agents crawling over these physical environments, of which humans are still in their mix somewhere, but how do we kind of collaborate with each other? Yeah, I think the bigger impact will be-- I can cite you some trends we are observing right now, right? So if you look at, let's say if you need someone to work with your ERP system, it's a very complex system.

Now, but with generative AI, we are bringing the natural language interface that reduces the bar, the skill level bar to sort of fill those gaps. But then you have things like, let's say forklift drivers. There's a huge shortage, right?

So what the physical AI, the impact it will have is more on the redesigning the workforce or the work itself, right? There are a lot of jobs which are tedious, dangerous. Those will be filled by physical AI.

Physical AI will also help augment the capability of human workers performing different tasks. But it's not going to replace human workers when it comes to making a decision that affects critical infrastructure. So humans will still be involved in terms of decision-making for anything that's sensitive and critical.

But a lot of tedious and repetitive work will eventually be-- Physical AIs will probably come in and help with those gaps, yeah. But again, and this will happen over a period of time. It's not like there's a date and then everything changes, right?

So we are on that journey, yeah. All right, folks. Well, one way to think about it is to say, hey, there's a lot of tasks in the physical world that many of us would prefer not to do, and going forward, we're not going to be replaced as much as we're going to find ourselves managing a small army of AI agents that do the tasks that, well, we didn't want to do them in the first place.

Hey, Sudhanshu, thanks for being on the show. Thanks, Mike. Glad to be here.

AI Leadership Insight series. You can find this episode and others on our website. We invite you to check all those out.

Until then, we'll see you next time.