Flux Brings Ground-Truth Visibility to AI Coding

Hey guys, thanks for the throw. We're here with Ted Julian as the CEO of Flux, and they just raised $5 million for a software engineering intelligence platform. And we're going to have a chat about, well, where is this entire category going?

Ted, welcome to the show. Hey, Mike, thanks for having me. Pleasure to be here.

Yeah. " But explain to us what we're trying to achieve with software engineering intelligence. And it almost seems like these days we need more of it than ever because, well, everything is starting to happen at machine speed, right?

Exactly. Yeah. The AI transformation that's happening in engineering departments is really chaotic, right?

Because it's impacting every part of the stack, code generation, test, QA, you name it. And while there's a lot of good things that can come out of that in terms of productivity and so forth, well, first of all, it'd be nice to be able to prove that, but it's also going to be important to understand what's happening, uncover roadblocks that present future improvements, potential risk, dependencies, spike in complexity. And so those are the kinds of questions that we're answering for our customers with our product.

And given all the chaos that's happening and the need for proving ROI, it's more important than ever for engineering teams to get some clarity around what's going on as opposed to remaining a black box. As one person I talked to recently put it, "AI is great. " And so it does seem to me, though, that there are existing bottlenecks and also some new ones maybe that are starting to be added to this equation.

So what are you guys seeing out there that people are running into, and how does software engineering intelligence kind of surface that and maybe hopefully help them fix it? You bet. What's novel about our approach to this problem is that it's based on the code.

So we have a code-first approach to what we do, as opposed to derivatives of the work, like ticketing systems, tagging in GitHub, things of this nature, which has been the historical approach. And frankly, as we are in the early innings of the agentic coding era, this approach is really crucial because it's the only way that we can really keep up with everything that's happening. These derivatives approaches just don't even have a chance.

Not to mention it being grounded in the truth of the code, and thus, hopefully more accurate. Definitely, we believe more accurate. We're seeing it be more accurate, as well as defensible and provable.

How does the software engineering intelligence platform actually know what's going on? I don't think somebody's running around with a clipboard collecting data and punching it in somewhere, but- No, we analyze every single PR. So we literally build it from the ground up, pull request by pull request.

That's the basis of our code-based approach, and it's also what allows us to be really dynamic and flexible in our analysis. You want to see things and you're a huge company across the entire software estate, thousands of repos? No problem.

We have all the data built from the ground up. You want to pivot and instead focus on specific applications, specific teams, specific business units? Also, no problem.

It's all built from the ground up. These are things that, with a ticketing-based approach, add a lot of toil to the team every time you want to see data a different way. Whereas for us, we have all the data, so all those different views are trivial for us and unlock a lot of value for our customers.

What do you perceive to be the role of the human in all of this as we go forward? Because theoretically at least, I could take the intelligence that you surface, expose that back out to the AI, and have it kind of fix itself. But at what point does a human want to get involved in that?

Oh, a human has to be involved for a couple different reasons. I'd say the first one is simply that there are people dynamics here as well, right? It's not simply a function of code, and understanding both of those things is crucial to success.

Let me give you an example. Most organizations, as they lean into code generation, have a quick hit and a dopamine rush when they see velocity really improves, and that's wonderful. But what usually happens next is a bottleneck in code review because you've maybe 10X'd the code that you're generating, but you probably haven't 10X the people that need to review that code.

And while AI can help for compliance reasons, security reasons, audit reasons, lots of other reasons, you definitely need to have a human in the loop. And so part of our job is to help inform that human, specifically the leaders, the engineering managers, by pulling together the code insights and the team insights and proactively letting them know about these issues, that you do have a bottleneck right now. Like DORA metrics related to code reviews are really trending in a bad way, and here's exactly where that is happening.

Another quick example would be how dynamic the teams are, that people are moving around to different teams. People are becoming more generalists, right, as they're starting to use this code, and that can make it a lot more tricky to understand who is working with who and on what, and how is that going. And if you don't have a way to keep up with that, then obviously you're not able to identify the bottlenecks that are standing in the way of that next progression or team dynamic issues that are creating frustration, these types of things.

To your earlier point, it's been an article of faith that says that if we just wrote more code faster, it would correlate to increases in productivity, and yet what we're kind of seeing is that there is more code in the pipeline than ever. A lot of it is getting rejected for one reason or another, and we're not even sure what percentage of the AI code is making it into production environments, and maybe we don't even know how much trouble it's causing Yeah. So besides that, everything's great.

But I guess my question to you is, is this going to give maybe the senior leadership some more realistic visibility- Oh ... to what's happening? This is the big opportunity for our customers and for us and our business and behind the fundraise that we announced today, which is, if you think about it, most major business functions have been digitized, and there's a platform that helps provide exactly the kind of executive insight you're talking about, right?

Sales has the CRM, marketing has its marketing automation platform, and the product for engineering is nothing . Right? Here we sit, in this modern-day and age with AI doing fabulous things around code generation, and we frankly haven't really digitized this function to be able to provide that kind of self-serve, easy insight into what's going on, what's going well, what areas are ripe for improvement.

And that's the big opportunity that we're chasing. A lot of developers, though, they get worried because writing code in their minds is somewhere between a craft and an art. And they're worried that suddenly somebody's going to have a bunch of metrics walking around, and they're going to- Yeah ...

turn the whole thing into a factory and start punishing people for not being as productive as they might be, even though all code is not created equal. So what's the smart way to approach this so that everybody doesn't kind of freak out? Yeah.

It's a great point, and there have been solutions in the past that have been a little bit big brother-y, where they've been sort of an organ rejection that's happened for these kinds of reasons. And so we have a very strong philosophy that stands against that kind of sort of surveillance of developers trying to weed out the poor performers because frankly, we think the bigger opportunity is to identify where those roadblocks are, where there's opportunities for growth to really unlock that next level of productivity and improvement, not just in code generation, but also in team performance and in the careers of the people that are on the team. I can give you a quick example.

I alluded a second ago to people are moving around and they're doing different things. You might look at, "Oh, Sue, what's happened? The amount of code that she's contributing has really fallen off recently.

What's going on there? " Or something like that. But when you have a more holistic system that understands both the code and the team dynamics like Flux provides, you'd be able to easily see, well, look, her code review time has spiked enormously, and she's actually helping multiple teams because she's so good at this, and she's such a trusted leader on the team.

This is not a bad thing. This is a fabulous team. Sue is leaning into a strength.

She's making the others around her better at what they do. Oh, and by the way, she's unblocking that blocker that emerged because of all the benefits, the good things associated with the code gen. So we see the opportunity as quite the opposite of that, where if you can both understand the code, its quality, the DORA metrics and trends around that, as well as team dynamics, then you really create a high-performing culture and a high-performing engineering organization that really lifts people up by helping them be better at what they do and understanding how they're really spending their time and having an impact.

To your point, done correctly, it almost seems like we have mechanisms to maybe capture a lot of that tribal knowledge that's kind of just floating around the organization and only gets shared between teams by happenstance, and yet I feel like no two teams are the same, and on each team, they each do something in particular- Yeah ... better than the other. This is another big opportunity for us and anybody chasing this market opportunity is that there's both micro opportunities for that within each of our individual customers, right?

This team looks a whole lot like this other team in terms of the nature of the application that they're building and the composition of the team. And so here's opportunities for improvement that team B can learn from what we see happening with team A, and we can proactively make those recommendations to our customers. But then you look at that as a data set across a whole bunch of different customers, and that's where the system can become super powerful because we're starting to derive these best practices across customers.

" We hear a lot these days about platform engineering, and I can't help but wonder if software engineering intelligence and platform engineering are more or less joined at the hip. There's certainly integrations that are going to happen across all these different systems. I think at the end of the day, there's an ICP, an ideal customer profile, and they have a set of problems that they need to solve.

You can very easily spread yourself too thin and find you don't do anything particularly well. And so, while in the fullness of time, I think some of these platforms probably do sort of maybe collapse into a unified thing. I think for quite some time, given especially the change that's happening in all these different parts of the software development life cycle, there will be standalone things with integrations between them, and the agentic era allows those to really provide a lot of value, maybe approaching that of a fully integrated system.

So how do you think AI will get applied to all this data? Because ultimately, anytime we collect data these days, we find some way to throw an AI model at it. Yeah.

We have a compound approach to our analysis that blends both LLMs, multiple LLMs that are code-specific, because we've learned through experience with our customers there are things that they are good at. I'll give you an example.... characterizing a pull request as to the nature of the work.

Right? So is it a feature? Is it a bug fix?

Is it maintenance? In the past, organizations have tried to do this with tagging or something in the ticketing system. And of course, the problem there is everybody's got to do it exactly the same way, and they don't.

And the bigger the organization, the less likely that is to occur. So garbage in, garbage out. But as I'm suggesting, our approach is based on the code, so you don't have to tag anything.

You don't even have to have a ticketing system or tagging within GitHub. It doesn't matter. Every single PR, we do this automatically based on the LLM.

So that's an example of something that LLMs are really good at, and as you might imagine, that's an important thing to understand. What we see all of our customers doing based on this is using this kind of analysis to trumpet the impact of their code generation. Because they'll roll back their pull requests to, say, six months before deploying Flux to establish a before, features versus bugs versus maintenance, whatever.

And then we track all the PRs that six months later, 12 months later, we can almost always show this huge spike in feature delivery as a function of engineering, and it gets people, you might imagine, very excited. ai that talks about this, where our customer, Cobalt, went from 9% of engineering in the first quarter of last year was doing feature work to 43% engineering was generating features. As you might imagine, they got very excited about this.

It wound up in the Q1 board presentation, and even the board members were saying to themselves, "Why have we never seen this before? " So anyway, that's a long-winded example, but a powerful one of LLMs are really good for certain things. But static analysis to look at code security or code quality or code complexity, those tools exist for a reason, and they're very good at their job.

And so we have multiple of these kinds of tools as well as different LLM approaches that inform our compound approach to how we analyze code that is the bedrock of our code-first approach to what we do. Where does things like vulnerability assessment and analysis fit in? Because everybody, unless you've been living under a rock- Oh, yeah ...

everybody's freaking out about this thing and Mithos and all the stuff that comes with that. But I don't think people have a handle on, well, how many of these things are real? How many of them need to be fixed, and how- Yeah ...

often are they fixed? Yeah, to be clear, there's a whole set of companies that are focused specifically on those types of problems and solving them for the CISO and the people on the CISO's team. But what we're doing is trying to shift that left a bit by doing some of that analysis so it doesn't show up as an unpleasant surprise from the security team, but instead is something that is flagged the minute it winds up in a pull request, so that the head of engineering can have that early heads-up and have a more fully formed view of the quality of the code, broadly defined.

So what I mean here is everything from, here's a new library, and yes, it has a third-party dependency you should know about. Or it has some security vulnerabilities you should know about. But at the same time, when looking at PRs across a span of time, what's going on with code complexity?

Is it going up or is it going down? What's going on with code quality? Is that going up and going down?

To be able to provide this holistic view of the code from a quality perspective and to be able to relate that to all the team metrics we talked about earlier. Because those are two sides of a coin that are crucial to unlocking that app. To truly understand the black box of engineering, you have to have it all to do that.

" I think everything's changing so fast that three months ago, my answer probably would've been all the vibe coding that was going on and the lack of scrutiny over that. But look, the models are as bad now as they will ever be, right? They're only going to get better.

And so I think that to some degree, that problem will take care of itself. I would say the shifts that I see coming where I expect to see, well, there already is a lot of scrutiny, and then this will become more part of day-to-day operations is around costs. So a lot of concern around token maxing.

People created leaderboards that took them away because they were just incenting poor behavior. That is the next frontier for us. We're working on this right now, is to be able to relate that token consumption to all the other fabulous things that we do.

Because it's one thing to understand who's burning tokens and to relate that across teams and individuals or whatever. But that in and of itself is not necessarily that valuable. Being able to relate that to quality, to throughput, to productivity, that's what people really want to understand.

" Right? And then maybe look, and then also be able to inspect, oh, and by the way, quality stayed pretty solid. It really didn't erode.

These are the questions that people really need to ask, and applying that level of insight to token consumption is going to be absolutely critical to be able to prove ROI on these investments that I think we all believe is there, but at the end of the day, needs to be proven, and it should be done based on data, and that data should be based on code analysis as opposed to people's impressions, opinions, other things that are subject to human error. All right, folks. You heard it here.

The time has come to shine that spotlight into the dark corners of software engineering that no one's really looked at for a very long time. But I think we're going to like what we see, but we should at least know what's there. Hey, it's a huge opportunity.

Ted, thanks for being on the show. Great to see you, Mike. Thank you.

All right, and back to you guys in the studio.