Developer Frustration, APIs, and the AI Tooling Shift | Agents Of Dev Ep. 6

Control. This is agent dev. I'm in position.

Copy that. Dev. Stand by for, go.

Standing by. Hey everybody. Welcome.

Welcome to another episode of Agents of Deb Podcast. I'm Mitch Ashley. I lead the software lifecycle engineering practice at Futurum Group, and also a practitioner myself, my co-host, Brad Shimmin.

Hey, Brad. Good to see you again. Hey there, Mitch.

Good to see you. I, uh, I understand we, we are gonna, we're gonna chat about, uh, a little bit of anger today in the developer community and try to make some sense about it or of it Yeah, that's one. You know, that's, that's sort of one mob.

You don't wanna get angry at you because, because they rally fast. They'll just make a new language. Tell folks about what you do at, at Futurum, by the way, or did you do Oh, yeah, sorry.

Sorry. If for those of you, uh, haven't, uh, or aren't following us, and if you're not, we, we would love you to, to, um, tune into our podcast. Mitch and I do this weekly, and, uh, we would love for you to join us on that journey.

Um, mm-hmm. So for me, I, I'm just, uh, like Mitch, I'm very similar, uh, and in that I'm an analyst and I cover data intelligence, analytics, infrastructure, and I'm also a practitioner, uh, you know, sadly beating his head against the, the impenetrable wall of data science, uh, on a daily basis. Yes.

Data, the, the world sometimes is an island and in the data world, but it seems less so. But maybe that's a topic for another podcast. That is a good topic.

Yeah, we should, we should talk about that. I'd love to talk to you about that. Get your thoughts on it.

Say, uh, so the controversy, um, which I didn't see this, you told me about it, uh, was about Anthropic. So changing their, um, use of their API to subscription customers who get the 200 all you can eat Buffet Yeah. Uh, of, uh, of Claude from any tool you like, but actually they changed it.

Talk about what happened. Yeah. So developers that were using, uh, a number of other tools that, that can basically use, uh, a harness to, to access, uh, anthropics models as you just described, using OAuth to get there.

So you basically just log into your anthropic backend via this third party front end tooling, like open code is, which was the most vocal, uh, of the, uh, communities to, to get angry about this. Uh, I think it was on Thursday or Friday. Um, but Yeah, either way.

What day? I think it was the ninth is when it was, whenever that was. Yeah, I think, I think that's Friday.

And, and, um, so they woke up that day and booted up their favorite open source friendly CLI tool that that does a lot of things. That cloud, uh, co code, which is anthropics front ends tooling does not do, um, uh, such as at the, you know, they're catching up. Okay.

I'm not saying that it's, it's inferior. I'm just, I'm just saying that, you know, their open source tools like Open Code have demonstrated some great ideas, like natively incorporating language server processors or LSPs or, yeah. Language service, service protocol.

Well, sorry, guys, to, to basically allow models to sort of look for syntax errors without having to ask, did I do something wrong? They could just fix it on their own, which is great. Mm-hmm.

And so anyway, open code developers woke up on Friday and, and went to incorporate their claw backend, uh, to use sonnets or haiku or, or any of the, the models they have, honestly, with their plan. And they were kicked out, uh, with a, you know, cannot log in warning. And, um, when pressed about it, philanthropic basically responded to say that, well, you know, this was really an inappropriate use of our API backend and one that was not in line with our security, um, requirements.

And so we, we have turned it off and you can make of that what you will. And, and I, I think their their right to protect their customers by ensuring the, the proper use of their backend services that is their responsibility after all. Right.

But, you know, it, it, it sort of irked a lot of people because it has, it has become very common practice right now to use the front end of your favorite front end, let's say Google's, um, anti-gravity and be able to access anthropic, uh, and their terrific models for coding. And suddenly you can't do that. You have to go right to Claude to do that with their own software.

So Anthropic with their own software, You know, I think several things kind of tick people off. One is, uh, your Ralph Wiggins, we'll talk about Ralph Wiggins, your Ralph Loops that were running overnight suddenly stopped running, didn't complete, um, yeah, because there was no notice too. So you, you were logged in or you went into open code and you both, you couldn't log in then, but also anything that happened to be running was already at a commission.

And, you know, you're figuring out what to, to do. People are complaining this is a, is a walled garden. I don't, I don't see it necessarily that way.

Um, you know, philanthropics got a business, they, they want people to use their models, but their tools, you know, they obviously want to have the, the best access to their, to their models, and they're giving away a lot of stuff with their $200, uh, subscription a month. Yeah. Not the 200.

Well, you know, you know what, man, I, I think that is one of the la drivers of this decision is giving away. Because when you allow people to just use their a la carte, eat all, all you can eat, you know, license in another tool, you being can't control how that a anthropic front ends. I'm using their model router, for example, to select which model to run against whatever task i I give it.

So if Haiku is gonna save them a lot of money, that's what you want. Mm-hmm. Yeah.

Well, exactly. And yeah, I, yeah, I get that. Of course, OpenAI immediately responded and say, Hey, we don't have such limitation.

Come on over the water's fine. Is it? I I, I, I, I was, I was looking to see, you know, because Codex is open source, okay.

Um, but I don't see an easy harness to bring in external models like we have in, in other tooling packages. Mm-hmm. Even Claude, it's, uh, you know, Claude code itself and, and Gemini and others are semi open, like Gemini is open source, but it's not an open harness.

And it's the same with Codex. Right? Um, and, and that's a big difference when you're talking about your tools accessing the model versus their tools, accessing their own models.

So I guess bring your own AI isn't quite totally a hundred percent. Just bring your own ai, whether that's models or tools, there are limita, you know, some limitation reply, some, some, some assembly required. Yeah.

Well, I mean, you could still use an API key, and that's, that's how a lot of people do that. But you don't get the same benefits that you do in using an OAuth just log in methodology for that, for that subscription, that unlimited subscription. So, you know, it's not that they're turning things off entirely, it's just that they are tightening the constraints around how you use their licensing programs.

Um, you know, for better or worse, I guess if, you know, and we've seen this many times with lost leaders, have we not Mitch, where oh, s will, will, you know, we're, we're in the, in the, the best one. The fav, my favorite one is, um, we're in a research phase, therefore this is free. And what that means is we're we're going to harvest every single interaction you have to better train our models, uh, to, to, you know, later on monetize, you know, o Open code does this with their Zen, um, API or sorry, model, uh, service itself wherein they have, uh, an proprietary model called Pickle, big Pickle Pickle.

Uh, and that's totally free. You can use it all day long. Uh, but you know, you're, you're giving it everything that you're, you're asking it to do Well and often, um, you know, the, they're feature limited or the previous generation is free.

The newest thing you have to pay for. Uh, I think the big thing here was you could, like you said, you could still access cla the models. You just have to pay the API token price, which can rack up, rack up some big dollars pretty quick if you don't, aren't careful about Yeah.

Especially with Mr. Wiggins at the helm. Helm.

Well, so what, so you ready to talk about, about Ralph? Yeah, I mean, okay, if we're talking about Gemini C Alive, talking about open code, talking about even anti-gravity and Zed with the, the sidebar, uh, a any of these frontend COIs that enable age agentic development are under the hood, basically just a, a four loop with, you know, an a break or an exit clause in there upon success. And this idea that, that we're, you know, chuckling about is, is, um, a, a methodology based upon a, a Simpson's character that, uh, was, was very helpful in that he was constantly, you know, screaming, I'm being helpful, I'm being helpful.

Um, and it, it, it's sort of a, well, okay, if we, if we take this seriously and we apply that to Agentic development, and we, we just force the model to, to keep chewing on the problem that it has, instead of, uh, either refusing to comply, which happens, or failing and stopping, or early stopping or getting lost, which also happens, then, uh, we, we can actually get some good code out of it. Uh, but as you, you mentioned, uh, and so, so rightly so, is that, um, that could be quite expensive and might be running for quite a long time. Hence everyone getting a little upset on Friday when Ralph refused was, was, you know, found dead in the street.

Yeah. It's actually in the, uh, in the Claude Library, if you Yeah. In the Claude Library, you can, you can access this routine.

So what, what I find fascinating about this, I, I think it's Jeff Huntley, who was the person who coined Yeah. Uh, created this, he called it a, a core loop originally, and it was a bash script, which basically essentially did the same thing without ai, but now, now it's with ai. Um, but what I really find fascinating about it, it gets around several problems.

One is this, this end state completion. How many times have we done a prompt and it says, oh, I'm done. Here you go, here's your answer.

You're like, yeah, I said there were five things and you did too. Or, you know, whatever. And it, and it likes to stop early and claim success.

Um, so, so this idea of having an end state that you defined, and then that's what keeps the, the core loop keep going until it actually does satisfy whatever the end condition is. But the other thing that goes along with it is every iteration, you know, we have, we have things like context window issues, right. And mirror issues.

Yeah. Oh God. Yeah.

Yeah. And We want to do long running agents. You gotta solve that, those problems.

One ways you can kind of get around it, I if it's solving it, but every loop can be another session, another instance. So you, you're iterating and you're not, you know, you're not flooding the zone of your context window. Yeah.

Nice analogy. At some point it floods over the window. It does.

Yeah. And that's, that's context stuffing is not the answer. And, you know, just, do you remember when meta released their 4 billion, you know, um, token context window model, Or was it 10?

Oh God, it, was it even more than that? Sorry. 5 I think, at the time.

And, uh, you know, you can't just take that for granted. As, as you and I have talked about, there's this, you know, u shape, you know, attention mm-hmm. Deficit within that context window to begin with.

So, you know, what you're talking about with this, this Wiggins technique is, is, uh, I think an important sort of point that we should, you know, we should be talking about. And so I'm kind of glad that it's bringing that to light and, and that is proper use of, of the context window. And, um, one of the predictions that, that I have for this year is, uh, this sort of emphasis on, um, optimizing that memory system for models mm-hmm.

And using a number of techniques intertwined, like semantic caching, for example, where you're not trying to save the, the entire history. You're just trying to, to save the meaning of it and reference that not the whole thing. Yeah.

Because you have, you know, anytime that you inject something into early into the process, it can carry that through, you know, it may have gone down a wrong path, and not, even though you're told that that's not the right thing, it keeps some of that going. Right. You, you're like, Hey, this is a Right relearns To fail every time.

Yes, Exactly. Um, so it, it's, it's an interesting concept. One, one of my predictions too, um, for 2026 is the emergence of the new development part paradigm of how we're creating software, uh, I think, I think is emerging this year and will, not that it would be codified and finalized by the end of the year, but I think we're gonna be talking well beyond vibe coding and just AI powered or AI native type coding.

There's a lot of these things are, are being invented and learned as as we go, of course, well, as, as the models are, the tools are changing themselves. But we'll see more things like control planes and guard guard rails and things like that being a serious part of not only the architecture, but how you architect it. And, uh, my thought is that, you know, we're, we're going from software development to software engineering, and that's because that's what this upper level thing of not writing all the code, but directing the code, reviewing and assign, and how things get you example in, in an innovation somebody creates that gets around an architectural Limitation.

Technical limitation. Yeah. You know, it's, it's funny, um, I, I think was thinking about all of these layers of abstraction that, that we're gaining with generative AI in particular, and I, I don't know where I even heard it, but there was this quote about every layer of abstraction is a bet on the, that in the future, at some point, you won't need to understand what's happening beneath that layer.

Mm-hmm. Terrify Thing to think about. Yeah.

Yeah. Well, maybe at some point if you wait long enough, it's true. But I'm sure there are many, Well, because you have another layer Just about the third layer under the second layer, the first layer.

Yep. Good point. You know, it's, it's interesting.

Um, it's just such a fun ride to be part of this whole recreation and how we're building software and, uh, yeah. Getting to experiment with it. You know, you and I aren't living in code every day.

You're, you're doing a lot of projects too there, um, with our signal work, so, which has been fantastic. Done a really fantastic job of that. So it's, it's, it's good.

You're, you're able to keep your hands in it. Me, I get to dabble in a lot of things and do that without having the commitment of a project. Yes.

The e evil scientist, yes. You're able to build that tower with hoist the, the sail and gather electricity from the storm. I'm the mad scientist.

Exactly. Exactly. You're like, I gotta ship product.

Uh, yeah. Right. And, and does, is that is a bit of a conflict, is it not in, in our industry where you have companies saying that you, the way we're going to measure success is how many vibe coded lines you've generated in this mm-hmm.

Past week. And, you know, forget the fact, and this goes to the whole, you know, technical debt thing we're talking about with betting on abstraction layers, but forgetting that just the fact that, you know, developers can't take the time to actually think about what they're doing. Instead, they're just waiting on the prompt to come back with the solution that they're trying to solve.

And that's where they're spending their time, is formulating their intent and their question and not musing about, well, maybe there's a better way to do this. And instead they're using AI to chase the solution. I, I'm not saying that's wrong, but I, but I do feel like we lose something in that.

And if all we're doing is, is chasing these, I'm not saying arbitrary, but they're arbitrary measures mm-hmm. Of, of value and quality. Well, and, and that seems to be a distinction, quality of some developers, you know, it, it's not, not a job, it's a lifestyle.

It's a, yeah. It's really part of their, their psyche, their being. It's really enjoy, enjoy it that much.

Oftentimes those are the folks that are, you know, you can't not work because when you work, you're playing, not working, you're playing with, With the Technologies are different ones. That's where a lot of those innovation and that innovation time comes from, which is great. It's great that people that are so passionate about it.

Yeah. Like, like we were joking earlier on with, you know, let's just, they'll just make a new language if you p**s them off. Uh, the reason why I thought of that was, um, thinking about Mr.

Mr. Pike at Google, who loathed all the semicolons, so much so that he developed a, a programming language wherein the compiler added them in for you later. So you didn't have to type them or look at them.

I can't remember if it was Jeffrey Gently or somebody else. Um, use, use the core Loop, the, the Ralph Loop to telling you to create a new programming language where all the primitives are slang from gen gen, gen ZI guess it's, Oh, dear Lord. Yeah.

Uh, that, that exists. Um, uh, we'll have to look this up. Um, but if anyone that's, that's listening, you know what we're talking about, please put it in the comments, because I, it's called curse.

I curse. That's what it's, that's, yes. Okay.

Yeah. Yeah. I just remembered what it was.

Fantastic. That's innovation. Yes.

Yeah. That's, that's, uh, that's, that's a subscription where you have to pay it for all the tokens, is what that is. Well, unless, unless you just don't care Or you don't even care.

Yeah. You work at, you work at Anthropic. Well, I think we may have done enough d damage on this subject.

Um, let's move on to our closing segment. It time for Yes, the drop. Alright.

Um, I think I went first, last time. You wanna jump in with your, What's on your mind? Sure.

Absolutely. I, you know, because, because I, I think very limited scope of things. Uh, my, the thing that's on my mind is related to today's topic, and that is that, uh, yesterday, uh, today's Tuesday, um, and yesterday, uh, anthropic dropped, um, a new tool, uh, a research tool that's free for use because, you know, they're helping to, to build it out called Cowork.

And Cowork works within their, uh, proprietary application, um, on Mac, I think initially, and eventually on their other platforms they support. But basically what it, what it does is take the note, this notion that many of us have really glommed onto with CLI tools, um, uh, to basically use generative AI as a backend to, you know, do things that we normally would have to do either in the file system or, you know, typing it out in ACL I, um, sorry, in a file manager or on the mm-hmm. And, you know, there's no limit to what you can do with that, honestly, because it is, it is the core of the computing experience.

Is it not, you know, the reason why we're all sitting right now at a computer is that on that computer, we, we have information that we can process, uh, and do things with, and this cowork is, is really built not for Claude code developers, so per se, but, but for business users and, you know, and home users and any user who wants to do things with their computer that go beyond, um, and I'm not pointing fingers here, but there are some implementations wherein if you ask the, the operating system to help you do something, it'll basically say, okay, let's do this together, open up your file manager and click on the third link down from the file pull down menu. That is not automation. That is not what we want.

So anyway, I, I, I, you know, I would imagine that we will see more of this as we move forward and especially, you know, to the topic we were talking about creating these sort of, um, you know, very controlled walled gardens of, of ownership of the user experience itself. And that's what we're seeing Anthropic do here. Well, excellent.

You know, I was excited to hear about that too. It's kinda like skills, you know, another capability that was introduced to, you know, here's a different way of, of kind of feeding content in and maintaining that, that content, uh, across sessions. Um, my drop, my, my focus right now is just getting ready.

It, it won't be, I don't think it's today, but probably next day or two, we're releasing the first half, 20, 26 of the software lifecycle engineering buyer decision maker data. So we completed that survey at the end of the year, and that data's ready. We just got put then polishing final touches on it into the intelligence platform.

So I'll be talking a lot about the data, just to hint, hint, one of the kind of really interesting things is AI shot up and has now surpassed even security in the top of the list of where people are Advancing that Shocking their budgets. Guess what That is shocking. Security always tops.

Yeah, exactly. Yeah. So security followed by cloud, followed by, you know, the, the typical things that we have.

So people are investing in it and, uh, counting on making, counting on it, doing some interesting things for us. So we'll have plenty to talk about, Brad, I'm not worried about Yeah. Just the picking, the choosing is the hard parts, That's for sure.

That's for sure. And, and you know, we're imagining you do the same thing, meeting with, uh, both clients and also people who, uh, that we follow and just kind of for the, here's what were happening for the, the year, here's some of the plans we have that they have, et cetera. So for any of our practices, Brad, mine or the others, uh, anybody listening, reach out to us.

If we haven't talked to you yet, we're glad to sit down and talk about both what's happening in our world and what's happening in your world. So we can do that. The exchange of information is an insight is one of the, the best things that we humans can engage in.

So we welcome that for sure. It's, it's fun. Alrighty, well thank you everybody for listening, watching this episode of Agents of Deb.

We hoped you enjoyed a little, uh, Ralph Wiggins conversation while talking about, uh, core Loops and Ralph Loops and also what's happening in, uh, the competitive world of models and development tools and all of that good stuff. So thank you for listening. Please follow, tell your friends about the, the we're getting stats, people are joining.

It's, uh, it's really good. Thank you all. They're coming on board listening and we appreciate it very much.

And send us, send us an email if you have a question or a suggestion or that was really great, that was really dumb, or why don't you talk about this, or what about that. We'd love to hear from you. You can reach us at agents of dev at futurum group com.

On behalf of Brad and myself, thanks for joining. We we'll see you on the next episode. Stay tuned next week.

Control. This is agent Dev. I'm in position.

Copy that Dev standing by.