Why Most Agentic AI Projects Never Leave the Pilot Phase – Temporal Ep 2

Hello, and welcome back to this Techstrong series with Temporal. We're missing Alan today. Miss you, Alan, but we'll have him back in a future episode.

In a moment, we'll get to introductions, but the last time, I want to remind you that we talked about durable execution, what it is, and the value that it brings. With that foundation, today we're going to lean in on AI, and specifically AI agents. We've seen all the studies that there's all sorts of cool stuff happening in the demo space, but we're struggling getting those things to production.

That's what we're going to explore today. So with that, I will move on to introducing our panel here. First of all, my name is Cornelia Davis.

I'm a principal technologist here at Temporal. I have a long history in distributed systems, kind of grew up through the microservices era, and you'll see in a moment that when we're talking about AI systems, we're talking about distributed systems. So with that, Samuel?

Hi, I'm Samuel. I am best known as the creator of Pydantic, now run Pydantic Labs, the company. I'm Tom Wheeler, Principal Developer Advocate at Temporal, and I've spent half my career as a software engineer and the other half trying to teach developers how to do distributed systems the right way.

And I'm Joshua Smith. I'm a solution architect here at Temporal. Spent a lot of time as a developer and architect, and now I help people build systems with Temporal, and recently it's been a lot of agentic systems.

Excellent. Well, thank you all for being here. We're going to have a great, fun conversation here for the next 30 or 40 minutes.

" There's a lot of promise there, and we're seeing really great demonstrations. I'd love to have you, maybe Tom, start with, what is it that you've seen that really excites you, and why do you think these demos are so darn compelling? Well, I think that agentic AI brings about a whole new way of developing, and it makes the impossible possible.

It reduces the cost of software development, and there's a whole lot of things that just weren't practical to build before that all of a sudden have become practical, and we're starting to see those things. We're also seeing the reduction of friction that we face as people trying to do things like booking travel, for example, where agents can do that kind of thing for us, and that's really exciting to be able to take advantage of that. Yeah.

Now, Samuel, you just introduced yourself as the creator of Pydantic, and you've created a whole bunch of other tools around that as well. So maybe you can talk a little bit about the tools that are supporting these agents and the amazing things that they can do. Yeah, absolutely.

I'd say we have to start by kind of splitting agents into two categories. We have the coding agents, which really come to fruition over, I guess it's a year, but really, I think you look at the graph, it's like October that they really started to spike. We can talk about them a bit, but I'd put them to one side for a minute.

When I talk about agents, I talk about agents that are going to be almost universally deployed into the cloud and are going to do, in general, a more specific task. In terms of the tools around them, memory is obviously a big part of it. Structured outputs, which is obviously where Pydantic comes in, is important.

There's obviously stuff like evals, observability, and AI observability. I've got to get that in, because that's what we do. But obviously also durable execution and how do you manage agents that, let's say, go on for longer than 30 seconds, and therefore can't just be assumed to complete on one node or within a request cycle, and which are expensive enough that we don't just want to throw away the result and restart if we do have to change node.

Yeah, I remember from years ago that it used to be that processes stayed up for days, weeks, months. And these days, I think I spent a lot of time in the container space, and I think there's studies out there that say that an average container is alive for seconds, maybe minutes. And so our infrastructure, it's not only failures, but it's just our infrastructure is cycling all the time.

Now, one of the things that I think is really interesting, and I might maybe come over to you, Josh, is that I also think that we're in this Wild West phase. And so we're so blown away by some of the things that can happen. It's not always the safest thing.

So I'm thinking OpenClaw? Yeah. There have been a lot of people handing a lot of their personal tokens and keys to OpenClaw and letting those agents, which aren't exactly deployed agents, which is a really important thing to talk about, but just running on their laptop and doing things that aren't coding, but trying to automate their lives with OpenClaw, and to mixed success, I'll say.

" And that actually, those long-running independent agents just make me think back to what you were saying about when you have deployed agents, you want those same things. You want to know what the agent did and why it did it, and not have it do some unpredictable things over and over again and burn up all your token budget. In particular, a year and a half ago, we were in a situation whereYes, sure.

In theory, we could have multi-turn agents, but in reality, it was like we were getting to three or four turns, and we went, "Wow, it actually managed to make two tool calls in succession and return the right result. " And that took 10 to 15 seconds. We are the victims of our own success, or the success of the models, which can now run for maybe an hour and complete a very complex task.

But that means that if you happen to cycle a container after 55 minutes, you're pretty sad about what you've thrown away. And so the necessity of durable execution has increased as, in general, as technology has advanced, things have got faster. So far in the materialization of AI, things have got slower.

We've gone from things that were going to take 15 seconds to things that are going to take an hour. Now, presumably, that's not going to be a linear continuation. It's going to get more complicated, but durable execution is obviously more important, as you refer to tasks that take longer periods of time.

Yeah, and we will definitely spend a lot of time on durable execution. But one of the things that I think you've all touched upon in your comments is this is as old as software engineering. When we're building the demo, we're focusing on the things that we can do, not the things that we can't.

And so we focus on the happy path. And I would suggest that there's even tools out there that are designed to support people really, really well in the demo phase. So for example, some of these agent SDKs, they are there, and they're designed to help you get to that first success very, very quickly.

But under the covers, they don't cover some of the, as you keep bringing up, the durable execution, which we'll drill in on in a lot more detail. Or the other parts that you need for an actually production-ready system, I think. That's right.

Yeah. And what other parts? We talked about evaluation.

We talked a little bit about durable execution, but what else? Well, I would say that we used to call them agent frameworks back in the, long ago, last year. Now we talk about harnesses the whole time.

" Eg, we slop generate our own micro framework on point and hope it works. Obviously, I'm biased in this, but I think this idea that we don't need one, it makes absolutely no sense. " So I think that the agent framework is a relevant part of it.

Structure, data extraction, evals, the model that you use, the memory, a whole bunch of different components come together. " The model providers love the idea that you start with their agent framework or their SDK, because now you can't change when someone else has a better model. But I would suggest being able to change model is pretty important.

Yep, absolutely. And of course, we're seeing tremendous changes in the models over and over. And I think one of the things you talked when you were talking about coding agents earlier was that the models really weren't up to snuff, until late last year, early this year, and then the models got good enough.

So okay, so the models are really good at what they do, but what else is there? If the models are so good, why are we still struggling getting things into production? The models are a lot better.

They can reason, and we're better at preparing them. But I'm an architect at heart. I love well-designed systems, and I've seen a lot of agentic systems that look like toothpicks duct taped together.

They just don't have the durability that you want out of an actual system. You want structure. You want the ability to maintain that state over time.

I work at Temporal. I'm a big fan of durable execution. So when I build agents, when I help people build agents, or when I help people build the same agent but better, we use the principles of durable execution to make sure that the human in the loop, that the state management, that the way we call tools, and the way we load data, and the way we call models is all in the framework of durable execution.

And that enables people to think at a higher level. One of the things I learned about Temporal when I was learning from Tom is Temporal lets you think at a higher level. Durable execution lets you think about things at a higher level because you don't have to deal with all the nuts and bolts and building your own harness, building your own bits and bobs.

And so what excites me with the newer models and the more capable models and the durable execution architecture is you can really build some very powerful agents that can do a lot of stuff without a lot of code and without a lot of overhead, so people can focus on what's their business problem and building a better system. Yeah. Now, I think that's really interesting.

Tom, did you have something that you wanted to add? Yeah. Going back to what Samuel had said earlier about models running for longer, and if you have something that runs, it's supposed to run for an hour, and then it dies, it crashes at 55 minutes.

Well, your customer who's depending on that thing to have worked didn't get 90% of the value. They got 0% of the value. You just paid 90% of the cost.

So you've got a bunch of wasted tokens, wasted opportunity, wasted time for them, and that erodes the trust. And so, that is why the things aren't going out into production, is because they're being tested in an environment where you have a safety net, and then you get out into production where the reality hits, and you've got rate limits, you've got containers that are crashing, you've got servers that are going offline, and all sorts of other problems that you face in production. And durable execution is the foundation that gives your applications a solid backing.

And so the problems that Samuel was talking about before, it would be crazy to write your own web server and to write your own HTTP parsing logic. Those are solved problems. But durable execution is also a solved problem.

So the companies that are innovating, the ones that are getting ahead, they're not trying to reinvent that wheel. They're using a foundation that's already out there, that's trusted by lots of big companies, and they're building on top of that. And so they're able to innovate while everybody else is debugging their shipping.

Yeah. I think the other thing to say is that, Andrej Karpathy talks about spiky intelligence. The models are very good at some tasks, and you have very close tasks that seem very close, but they perform very badly.

And that is in particular because they just haven't been basically put through the training loops as strongly on those particular tasks. And so you cannot just assume, look, we've got this model that's amazing, superhuman at doing some task, therefore it must be superhuman at all tasks, and I can just leave it to run. You will find some task very nearby where it's going to fail very often.

And so obviously you need evals, you need the observability to go and find out what those tasks it's not performing are. And you might want to go and use a sandbox to be able to run some of the code that it's writing because LLMs are very good at writing code, it's a very good way for them to express themselves and a bunch of other things. But you cannot just assume they are clever at one task, therefore they are clever at all of them.

And obviously, you can guarantee that when an LLM provider or someone building agents for you demonstrates a task, they're going to have chosen the thing it's good at, aren't they? Yep. So rather than the thing that's in the trough next to the spike, which it's doing really badly.

Yeah. And so what I think we're getting to here is that the models can do a lot, but you still need an awful lot around the models. Mm-hmm.

So durable execution, the model itself, durable execution isn't in there. It's providing durability around all of the processing that happens around the model. So one of the things, unfortunately, we know that Claude, that Anthropic released the source code of Claude, which I have not looked at out of respect, but I have seen the reports.

It's a half a million lines of code. It's not just the model. And that half a million lines of code is doing things like compensating for some of the deficiencies that still exist in the model.

It is addressing some of the durability concerns that we've talked about here or trying to, and that half million lines of code is not running in one monolithic process, right? Yeah. It's running in a distributed system.

Yeah, excellent. I'd really love to-- One of the things I do is I look at different agents that people have built, and then my job is to help them use durable execution. But I am really curious to see, of the major desktop tools and web-based agentic frameworks, how all their code bases compare.

I also have not looked at the source code, but as a technologist, I'm really curious to see what they're all doing behind the scenes, because it's got to be very complex and there's a lot of systems in there. There's lots of hideous things. I've read numerous articles about it.

I have looked at some of the code. They have 500-line functions. The main function loop is- Yeah ...

500 lines. They have a whole bunch of detecting whether or not you're p****d off with a model by basically a regex for expletives. That LLM company who discovered, whether they discovered or whether the LLM wrote it.

But yeah, they're not kind of pantheons of perfect code- Yeah ... that we would've expected. They're almost proud that they're basically not looking at any of their code, which- Yeah.

Yeah. No, it's true, they are. I admit that I'm not quite there yet.

I still look at the code, at least at some level. But what I do think is interesting is that I do think on the positive side, it isn't just that the LLMs maybe are writing all of the code, the 5 million line, or 500 million lines of code or whatever it is. But I think it's also an acknowledgement that there's certain things that are happening even in the agentic process that just don't make sense to hand over to an LLM.

There's still a lot of processing that can go on around that, and I think that's an acknowledgement of that. " And then we talk about what would you want to do that maybe you don't want to just delegate to what's really just a sparkling API call. You're just handing a bunch of stuff to an LLM, having it come back to you.

Maybe we should have some structure around what tools make sense in this context, or what context you want to give to the model, and not just say, "Solve my problems," and then send that to the LLM. And the way you get data back out and the way you look at it. There's this maturity curve where I think the more you-- We call it harness engineering, like you were talking about earlier.

But it's really just what's your architecture around this really very special and fancy, but it's just an API call, and you shouldn't trust it to do everything because it's someone else's model most of the time, and you want to know what it's doing and what it's giving back to you. I'd also say that, look, the big model providers are trying to work out their business model, right? They do not want to be in a world where they compete entirely on the quality of the model and price and latency and uptime, right?

They want stronger hooks that keep you. That's a completely understandable thing to do. Complete respect for them.

I think in many contexts for your business, it probably doesn't make sense to go all in with them. So it probably doesn't make sense just to use Claude Code SDK in your platform. It probably doesn't make sense to use all of the built-in tools that they provide.

So they'll do like MCP, just give us all your auth stuff and we'll just do MCP for you. Let us do your code execution. Let us do all of your web search.

I think that there's an awful lot of value in thinking all the way through about how do we have the flexibility to go and-- Even if it's just the flexibility to have a pretty hard-nosed conversation with the OpenAI sales rep next time they come around, or whatever it might be. Yep. You definitely want model flexibility, and you need the visibility to know what all the different models are doing and evaluations.

But I have never seen-- That works pretty well for on-desktop coding agents, those kinds of things. But for deployed agents, which is where I think the real future and the real magic is, you really don't want to be sending all of your company's data through a system that only you can deal with one vendor. That personally gives me lock-in feels, and I don't like it.

I want to have a framework that's flexible. I want to be able to use multiple models, because this is my company's intelligence. We're building an intelligent business here.

We want to not be locked into a vendor that can have all of our data and then tell us it'll be a lot more money than we were used to. So totally agree. And I would say also, we saw what happened with, again, deep respect for Anthropic.

They're not my customer. Maybe they're yours, maybe I haven't had, but you know. Yeah.

We saw what happened with them and Figma, right? They had someone on the board until the day before. " They are looking at every industry out there and saying which ones are most profitable and most replaceable with AI.

And so, basically another reason not to completely go all in with one particular model provider, use their SDK and be done with it. I would strongly advocate for an agent framework, whether it be Pydantic AI or one of the other inferior ones. Rubik's.

Yeah. Outstanding. So actually, let's stay with Pydantic because I want to drill in on a couple of topics.

We're not going to do a deep dive on all of these things, but I want to pick a couple of topics, and since you are sitting here, Samuel, and you created Pydantic, let's talk a little bit about structured output. Because LLMs, we know that they do languages, but one of the things that all of the different model providers do in their APIs is they support structured output, either in connection with tools, where you can send in tools which have a schema for the API effectively that the tool is fronting. But then there's also the ability to just send in to the LLM, "Hey, here's a query," and instead of just giving me back unstructured data, because it turns out that LLMs are pretty darn good at parsing stuff.

And so maybe you can talk a little bit about the structured output and the impact that that's had on this agent space. Yeah, I think that everything that isn't just a very simple chat interaction between a human and an LLM requires structured outputs. And even if you're ultimately doing something that feels like a chat interface, you're probably going to go and want to do tool calls.

Those tools are going to want particular parameters, and so you're going to end up doing effectively structured outputs somewhere within your application. " You want to be going and parsing the JSON and validating it against a schema, and you want reflection. " It's amazing how efficiently that works at basically keeping models working.

So yeah, there are kind of three ways of doing structured outputs, to go into detail for a second. You have tool calling, and you can use tool calling either for what you would presume to be a tool, or you can do what Pydantic AI does, where we have a final result tool if you're doing structured outputs, where we'll use a tool call as the way of doing returning structured data from the entire agent run. Many of the models now have native support for structured outputs as the output type.

And then finally, you can do what we call prompted output, where you basically say, "Hey, here's the JSON schema. Please return data that matches this JSON schema. " And we'll do a bit of cutting off the triple backticks around the code.

But basically, we take that JSON, and then we pass it to Pydantic. We're very lucky that all of the models are now basically trained on the validation errors that Pydantic generates, and so returning Pydantic errors, even if you're not using Pydantic AI, obviously Pydantic is the underlying validation layer for all of the agent frameworks. So you will get it even if you're not explicitly using our agent framework.

Mm-hmm. Excellent. And there's equivalents in other languages.

But yeah, the principles stand basically the same. Okay. And that's so important.

I've seen a lot of, getting back to your earlier question about demos, I've seen a lot of what may be just demos at enterprise companies who built an agent, and they're like: "Look at what the model can do. " But they don't have that structure that's necessary, so that when they're calling out to their enterprise systems to take action based on that agent, with hopefully a human approving it, that it's not garbage data when they try a different model or a model gets upgraded or anything like that. It's so important to have structure like that in building these real agents that deliver business value.

And to your point on the human approval, you also want a permission system where fundamentally the model cannot do anything that it's not allowed to do. Right? As in, you still need those logical, procedural, deterministic constraints on the outside.

" Yeah. You would have constraints on what they can go and do, and the same is even more true of these stochastic machines made by companies who have made their money out of breaching IP regulations. You don't want to just let them do whatever they want to do.

Sorry if there are companies- Your cynicism is so great. I love it. But yeah, there are lots of privileges I don't have at my company.

Like if I was in high school, I can't change my grades. You don't want to-It's horrifying what people have given their agents access to do without any approval. They're just like, "Oh, they have the authority of the president and the board.

" I think what's really interesting is that our mantra as a company is AI is still just engineering. Exactly. And I think that it sounds very much like-- It's the reason I'm probably in the room with you, is because we kind of agree on that, whether we use the same mantra or not.

I think that there is a certain bit of, dare I say, Silicon Valley, who likes to think that AI is so profound and so different that we can basically abandon engineering practices, and we can basically YOLO it. Mm-hmm. And so in particular, one of the things that causes people to think is, I don't need to learn real engineering.

I can just go and like, today it would be install my OpenClaw. A year ago, it might've been get going with LangChain, or whatever it might be, and I can ignore the engineering practices, and I can just go and do stuff. Mm.

And it'll work because it's AI. And the whatever it is, 30 years of wisdom, maybe hundreds of years of wisdom on how do we do permissions within organizations, is abandoned by people at the altar of, it's so profoundly different that we can abandon what was known before. And I think we are in a moment of basically sliding back towards, oh, it turns out some of those things we learned in the past do matter, and we're sure that it's going to look slightly different, but it's still engineering.

It's just engineering with a new tool that is slightly different to use. And I will go so far as to say, I feel like I do more engineering now than I did before this. Mm.

My son is also a software engineer, earlier in his career, of course. And as we've been navigating this AI thing, he's probably more sad about losing his need to write code than I am. Maybe in part because I'm slightly ahead of him.

But what I tell him all the time is that I thought in the beginning, I love writing code, but I don't write code all day every day. " And now the coding agents are doing that. And so I was a little sad myself, and then I actually started learning how to use the tools, and I find I do way more engineering.

I think we used to use code to do engineering, in part. That was the language that we used to organize our thoughts and set up our architectures. But now I find that I do engineering in a very different way.

So I'm totally with you, Samuel. We're still engineering. I think there's a parallel problem that's coming when we get to these business deployed agents, where people are going to hear about this and say, "Oh, no.

" But the actual reality, the real optimism that we can all have that's grounded in reality is these agents can gather an enormous amount of data much faster than any of we can. They can summarize it much better than humans can and present it to a human to make real decisions. And I've seen that happen and it's super powerful.

And I think what's really wonderful is humans will have that experience you have in your work where you're doing actual engineering instead of wondering what's the data like on the other side of the country or any of these 15 different data silos. They'll do actual decision making informed by all of the information they can get from these data sources that are currently locked behind silos. I've seen it happen and it's really powerful.

Yeah. In particular, the AI can basically go and do the methodical bits of my job for me. Mm-hmm.

It can go and implement all of the React components and the five different endpoints I need for this thing, which are completely procedural. I love that. Yes.

And I can get on with trying to work out what the really hard, unique, novel thing is that the agents can't do, and maybe won't ever be able to do, or at least maybe that's further off. Yep. Okay, so we've gone to a number of places, and I want to kind of use all of these threads and pull us back into the durable execution.

So we talked a little bit about structured output, and we talked about rigorous systems. We talked about the breadth of things that happen outside of the model, the size of what the agent logic is. And we talked about OpenClaw and how scary it is, and permissions, and those types of things.

So let's talk a little bit-- And we also talked at a fairly high level about how durable execution at the principle level of pick up where you left off helps in that. But I'd love to go in a little bit, kind of at the same level that we did with structured output. Let's talk a little bit about durable execution, specifically in the context of that we've just set up with the earlier part of the conversation.

Yeah. So, I love durable execution for agentic use cases. That doesn't surprise you.

I'm a Temporal fan, and I've been building with this architecture for a while. But being able to write code that does agentic things, and then it just runs like it runs on my laptop, that easy, but it has this durable execution architecture wrapped around it to track workflow state, to track every step that I've ever done, to not recall the things that I've called already, to automatically handle errors for me, that has built-in timers and built-in human in the loop. Those are just all the tools that I need to build a distributed system, and you guys made the wonderful caseAgentic systems are distributed systems.

It's just they're long-running and complex, and I need to be able to think at a higher level. And that's what durable execution lets you do. It lets you think about what do I actually want to have happen, and the architectural framework of durable execution supports me and my agent and makes that safe and reliable and scalable in really powerful ways.

Yeah. And whether it's you writing code or whether it's an agent writing the code, applications so often get cluttered up having to deal with-- There's the small little happy path, the thing that they're ultimately trying to do, and the code gets cluttered up with all the things that could fail along the way, all the different negative adverse consequences that could take place, and you have to write code. And this is when you're first learning how to code, it's a lot of fun because you don't have to worry about production support.

You don't have to worry about any of that. You're just exploring and making things work. And that's why it's fun to put together demos.

But when you try to push them out to production, you discover that it's a cruel, harsh world. And in order to sort of survive that world, you have to have lots of try catch blocks. You have to check the return codes that are coming back.

You have to build manual application checkpointing into the application. You have to do all sorts of things that just obscure what you're trying to do, and you wind up in a situation where the amount of error handling and mitigation code that you have is triple the code related to the happy path. And durable execution allows you to clear all that out because you're able to sort of push that out of your application down into the platform and delegate it in the same way that you're delegating the low-level stuff about writing code to the agent.

You're able to delegate all of this crash recovery and failure handling and all that kind of stuff to a platform that deals with it, so you don't have to. And when you do that, magically, the happy path shows up again. Yeah.

You can see the clear path ahead. Yeah. I- It becomes procedural code again.

I think that's one of the points to say is like, there are two arguments for graphs, right? We have to talk about graphs here because there have been those who have argued that you need a graph to run an agent. There are two arguments for graphs.

One of them is graphs are an excellent way of doing snapshotting and giving you infrastructure that allows you to snapshot so that you can resume from certain points within the graph. It turns out that graphs are terrible for doing that, actually, and they involve you doing a whole bunch of stuff, particularly talking in Python. You basically cannot have graphs and parallel node execution and type safety.

And type safety is incredibly important for LLMs when they're writing code because they basically can use it as this very fast side effect-free way of working out whether they're writing logical code. The other reason for graphs is developers like graphs. That is completely reasonable.

As someone said to me, I was having a conversation with someone about what the arguments for and against graphs were, particularly having had a conversation with Maxim about why graphs are never a good idea. " And I think that is a perfectly good answer. But the point is, if you get rid of the graph needing to stand in as a kind of pale substitute for durable execution, now your graph can-- You can actually go and do parallel node execution, and you can keep the type safety because you're basically handing off the durable execution to some low-level piece of magic that does that for you.

" They're separate things. They should be solved in separate ways, and you can have a much better durable execution solution and a much more elegant graph implementation if you separate the two out. Yeah.

And I would argue that part of the reason that some developers like graphs is because for them, it's a simpler programming model. It's a way of them organizing their thoughts. But that's really, with durable execution, we talk about the runtime behaviors.

We talk about the fact that we're pushing that error handling and those types of things down into the platform. But the way that we're doing that, and you alluded to it just a moment ago, is that we have a very simple programming model. The programming model allows developers to code as if it's all in one process.

And durable execution, the durable execution process turns that into a distributed system and handles all the errors along those. " Let's bring them back into the conversation because one of the things I think that's interesting about this agentic space is that you're right, that the coding agents are getting really good at coding, particularly if you give them a programming model that allows them to just program the business logic. Turns out humans are better at it.

Turns out agents are better at it, too. No surprise. It's very satisfying as someone who's cared about DX for most of my career that now I don't need to have to argue it's DX, it's for humans.

" Now it's like- Then people listen. Now then people listen. So type safety, durable execution, and simplifying workflow, avoiding endless like try catch.

These are good principles that have been around for a long time, but now we can say, "Oh, no," they are important for the alter of AI, not just for us humans. Yep. I'm very fond of sharing one of my mantras, one of my one-liners within the organization is, remember, we're not just teaching humans anymore because we do a lot of teaching of our customer base, and our customer base aren't just humans anymore.

Josh? Yeah, I was just going to say, it's kind of working. I've known that this future was coming for a long time, that durable execution is code first, that's great, at least with Temporal.

Coding agents need to be good and understand this architecture and rely on it, and I've been playing with coding agents for a while now, and whatever it is you're doing, it's working because they seem to be getting a little bit better at understanding, "Oh, right, I can code like Tom told me to, and I don't have to write all of this extra boilerplate code. " And that's really a great moment when weProbably all have used a coding agent that's generated lots and lots and lots of code. But it's much nicer when you say, "Use Temporal and just write these steps," and then it's easy to read.

It actually supercharges the developer experience of coding with coding agents, because they are really good at generating code, and if you give them the right context and training, they can write durable execution code really well, and then me, as a human, I can understand it. Yep, absolutely. And it's not only we are starting to get the models themselves to understand Temporal.

There's more code out there, there's better code out there. But back to the earlier conversation of the tool sets that are evolving around these things now include a little something called agent skills. And so one of the things that we've done here at Temporal is we've created an agent skill that has a bunch of Tom's knowledge in it, and it has some Josh knowledge in it, and captures all of those best practices.

And these agents are really good. Back to what are the things that agents are doing other than just invoking a model. Well, they're also doing things like navigating the context- Mm-hmm ...

that's available to them that is going to service them in doing their jobs better. And so agent skills are a very kind of formal specification for that. Yeah, it's pretty funny.

At least to me it's pretty funny, because I'm a guy who keeps links to everything. So, two years ago, Tom in a community post, I'm using you as my reference a lot, but this happens. It's because Tom has done a lot of community posts.

Tom is a very clear writer, so I write down, okay, when a customer asks this about how to use durable execution, this is a public post that I can cite, and I have that all in a giant file of all the things I know about Temporal. I turn that into a markdown, and now when I write Temporal code, I can just pass it all of the wisdom of all of the people who have used durable execution so well, and the agent has that knowledge because it's so good at reading all that information and making sense and reasoning about it, and using it as context. So it's really powerful.

Yeah. Really awesome. All right, well, we need to start rounding it out here, and what I'd love for each of you to kind of chime in with is, we've taken this conversation on a little bit of a story arc, and what would you leave everybody with?

Any nuggets of advice in helping them? Again, the task here is, yeah, we're building demos, they look all fine and snazzy and all of that stuff, but then what do we do specifically? What are we going to do Monday morning to get our agents closer to prod?

So for me, if I could leave some parting advice for the audience, it's that I see the construction of agents going down two paths. There's the tragedy of poor architecture, where they don't have the things that we've talked about today that you need: structured outputs, visibility, durability, durable execution. And it's just a tragedy because there's so much power in combining these agentic models with good tools and good problems to solve to help people, and that feels like a tragedy to me.

But the other side of that coin for me is grounded optimism, that if you build agents with good architecture, and I would say that architecture should be durable execution, then you have the whole world in front of you because you can build simple agents, they're easy to understand, and they really do have superpowers. Yeah, very well said. I would say, if you look at what you ultimately need to have in production, you need the ability to survive a crash.

You need the ability to withstand failures in other systems, because you're dealing with systems that don't have the benefit of durable execution. You need the ability to look back at the history and understand exactly what happened, and everything that led you to a certain point. Forensics and audit trail, that's super important, and it will become even more important as time goes on and we delegate more and more important things to these systems.

Having the ability to go back and see exactly what they did and how they arrived where they did is going to be very important. And I think durable execution gives you all of those, and it's definitely the foundation that I think people should be building agentic applications on. I think you can think of it as the tortoise and the hare.

There are two approaches to go about this. You can just forge ahead with the first tool that you find that seems to work to start with, and you'll probably get to something that's 80% complete. It runs, it's really impressive.

Wow, it solved our problem on this. Okay. Oh, I know it failed the first two times, but the third time it worked.

That's really awesome. Or you can take the kind of tortoise approach of like, I'm going to start at the bottom of the hill, to change my analogy, and I'm going to go and choose the right-- I'm basically going to build a system by which I can hill climb towards the right solution. And that is choosing your framework, including model agnosticism, working out how I'm going to do structured outputs, working out how I'm going to do evals, working out how I'm going to do durable execution.

You get all of those primitives in the right order, you build something very simple, and now sure, you're way behind your colleague or your competitor who just went straight out and tried to do it, go all the way, but you're actually going to get to a way better thing. It's a bit like rewind to 2000, and you have two ways of building a website. One is you go into Microsoft Word, you drag the pictures into the right place, you put the text in, and now you do export to HTML, and you've got this WYSIWYG way of building a website, and you get to something in a few seconds, and then making that look good is basically impossible.

Whereas the other colleague who started by writing out the HTML and trying to work out how to build a website, sure, they're going to be behind to start with, but they're going to end up actually building a decent website. Starting to build with the wrong tools is like trying to build a website with a WYSIWYG editor. You will get so farAnd you will fail.

Apologies for the mixed metaphors, but I think that's mine. It's perfect. And to travel with that metaphor a bit further, because I've seen a lot of people, I was a web developer back then, and I saw a lot of people take that approach.

They wind up with something that can't be made to look good, but more importantly, is completely unmaintainable. So they create something, and they've painted themselves into a corner, and I think when you're going out to build the applications of the future, you want to build something that ultimately you can maintain and support over a long period of time. And so you have to think into the future that way, and that's where the investment in building it the right way really pays off.

We honestly have a consultancy we work with who basically make their business out of going to companies who have started to build POCs with one of our competitors, run into trouble, and basically have these failed, paused POCs. Very much like those- Yeah ... WYSIWYG websites we all remember.

And the consultants come along and they basically start from scratch with Pydantic AI and build the agent, and often with Temporal, and do it correctly. But they're making lots of money out of it, and you don't want to have to have consultants come along and restart your thing from scratch. You want to start with the right primitives.

Yep. Yeah. And I love that.

And my parting words will be, we sometimes will talk to somebody who says, "You know what? Durability is only something that I need in production. " But I would suggest that that's a false dichotomy.

The durability or rapid prototyping is a false dichotomy. You can do both. We talked about the simplicity of the programming model being one of the benefits of a durable execution framework.

And so do not kick the can down the road. Start thinking about that. And there's no reason that it should slow you down.

So, all right. Josh, Tom, Samuel, thank you so much for the conversation today. It's been a real pleasure, and we'll see you next time.

All right. Thank you. Thanks.

I enjoyed it.