Taming the Technical Debt Snowball

Hey guys, thanks for the throw. We're here with Ron Brownie, who's the CEO of Dyna software, and we're having a little chat about technical dent specifically when it comes to ServiceNow, which is probably maybe now one of the most widely used IT service management platforms out there. Ron, welcome to show.

Thanks. Yeah, thanks for having me. Mike.

Technical debt exists I think before and after we adopt ServiceNow. So, uh, walk us through this a little bit. What kind of technical debt are people bringing with them into the environment in the first place that maybe they should just check that baggage at the door?

Um, well, when you think about technical debt, um, a lot of things are occurring in terms of, um, legacy systems, legacy tools, business needs, and oftentimes just to accelerate or even meet business demand, when you're say implementing ServiceNow, really, really either poor configurations or poor thought out, um, designs get ported into ServiceNow or items that from a translation perspective don't make a lot of sense to either rebuild in a custom way as opposed to leveraging outta the box features and other elements. Um, oftentimes what what, uh, we certainly encounter in here is customers having that whole drive for speed and also trying to meet with what's the business trying to do and basically getting pinched with, we've gotta just roll this out and pull it in. And from that point forward, that's where you start to get lots and lots of problems that start to snowball and sort of, sort of go forward from there.

But, um, that, that's probably the, the easiest overview to, uh, to sort of explain the stuff coming in when, when you're just starting off, How much of that is a technical problem versus a cultural problem? 'cause I think what happens is a lot of people just wanna bring their existing workflows and port them on the ServiceNow, when in reality somebody at ServiceNow probably already thought this through and just build a feature into the platform. Right?

I think I would agree with you on, on the most part, there's, there's an element of, well, to be honest, there's sort of that frontend components of governance when you're thinking about standing stuff up. And one of pr one of the key principles to really start to instill in that type of a framework is the decision making around what moves in and, uh, and why is, is really the, the key element of it. So I would certainly agree with you from a, from a a people perspective, process perspective and why they're choosing to actually move stuff through is, is the ultimate culprit in terms of the, the initial onset of technical debt in that, that circumstance.

Now, once I start running ServiceNow, I also seem to wind up generating some technical debt. So what's the source of that and what can I do to kind of minimize that? Well, um, very similar situations where you've got driving business demand, um, you've got short turnaround times that start to create, um, creative approaches in terms of, of developing.

Um, what I've seen in my past specifically is situations where the fastest path was the key path to get something done. Um, ServiceNow is a unique platform where, and oftentimes there's maybe 10 different ways you could do something. And of those in terms of safety, and I'll explain safety in a second here, um, there's maybe only two to three that are really the, the best path to actually execute to do that.

One of those typically being just configure as opposed to script or build or, or develop the safety sort of aspect is really thinking about what else is this related to? So a really good example, early days for me, um, I encountered a, a, a, a customer dealing with an implementation of, uh, HRSD, the human resources, um, um, application suite from ServiceNow. Um, and the issues with that, not being able to move forward and not realizing the actual interconnections to ITSM and the knowledge management component, um, and really not having the ability to recognize and understand there's linkages here as you're actually driving forward.

So when really building out and developing and meeting some of that business demand understanding are you configuring or, or building in the right sort of path, and also what ServiceNow themselves doing that you need to pay attention to that might be related or could be related in the upcoming future. Almost the same kind of question, but as part of the issue is I'm bringing a bias to that platform where maybe I think I need to build everything when I just need to configure it. And if I build it, then I gotta support it and maintain it.

And that's where the technical debt comes from, A hundred percent. And the, the cascading challenges, uh, from there, Mike are now you've got longer durations in in upgrades, you've got more complication in analyzing and understanding what's there when you're either introducing brand new product or features from ServiceNow. Plus when you're also building unique things for your business needs on ServiceNow.

It's almost like a snowball effect. Um, one of the best, uh, sort of analogies for what technical debt turns into, uh, that was ever shared with me was really about a loan where you've got compounding interest as you're going forward. And if you're not actually dealing with that and paying attention to it, it's eventually gonna get extremely costly.

And that's one of the big challenges a lot of customers are facing these days is situations where even ServiceNow themselves are doing analysis to understand how much tech debt is on a platform. And the recommendation's starting to come back. You just need to re-platform, meaning just get rid of the whole thing and start fresh because it's become way too complicated and way too difficult to support and introduce some of those new features.

To that end, you cannot go anywhere near ServiceNow these days without somebody leaping out to tell you about some new AI agent that can do this, that and the other. Will these AI agents help us reduce the technical debt or might they actually increase the technical debt? That is, is, uh, an awesome question, Mike.

So my, I'll call, I'll say my current perspective, it's a double-edged sword. Um, there's a lot of things where it can accelerate stuff like development. Um, you've probably heard the term vibe coding and other stuff, the, these days.

There's a lot of things that can accelerate certain pieces. The big worries I have is of what gets built and pushed out, are the proper mechanisms in place to validate one, is it actually good code two, is it secure? All those different elements.

Missing component always for me is what's this related to what ServiceNow is doing? 'cause what I've found in the past is not so much what I'm doing, but what ServiceNow is doing themselves. And when those come into conflict, I'm guaranteed to have some form of problem or challenge that's gonna limit me in in the future.

On the plus though, you do have a lot of different things that are coming out where it is accelerating either usage, um, in terms of end user and and elements there, certain pieces where if it's done in the right way, um, accelerating different aspects of development. Um, but my future that I hope we really see is something where you've got very, very specific boundaries that some of these AI agents are working within, uh, both in terms of context of the target instance or the target customer and what they've configured uniquely for themselves. And also, um, making sure that you're sort of staying inside the rails of what is sort of safe to actually go and configure.

I can't say that there's a whole ton of stuff out there in that vein yet, but I think that's the direction where stuff is starting to lead to as people are getting more normalized with using it to basically develop and configure and seeing some of those sort of challenges. I wa I was mentioning just a second ago, Are people kind of conscious of technical debt or is it just something that kinda adds up over time and then they wake up and they go, holy moly, this stuff is gonna collapse out of its own weight because I didn't pay attention to all this stuff. So maybe do we need, I don't know, flags in the system that's somewhere that says, you know, here's your technical debt rating.

I, you know, it's a great, it's a great question and, and um, things that I've really seen is, um, it's not so much that people are unaware as it's going forward. Um, it is for sure a snowball effect. Uh, oftentimes the root reason is, you know, here's something that that got developed, that got built, um, maybe something that gets scanned after the fact, meaning the code is already done and uh, they're about to promote it, then they find an issue and then they start to identify, do we need to pull this out or do we push it in and put something basically to fix it after the fact?

And oftentimes the business pressures are, we're just gonna have to push this in, effectively broken, get enough value out of it while we circle back and fix it. And that all starts to eventually build up. And the big challenge ends up being usually platform owners recognize this is becoming unwieldy to try to move forward.

And it's everything from the total duration in terms of upgrades, plus moving through new enhancements or installing new things, um, and the also ever-growing cost of supporting it. And that creates a situation where people are that own the platform are, are reluctant to add more things on. So business value and ROI sometimes gets deflated or stifled.

Um, also then, uh, uh, creating a situation where they're looking at this mountain and to be honest, to go to business to say, I need half a million dollars to remediate this stuff. Very difficult conversation to have to get interest because perspective on that side of the fence usually is one, why does that even exist? This is your problem, but two, how does this even help my business?

And making that connection, that reducing the technical debt actually helps the business oftentimes is very, very difficult to actually achieve. Then you basically got stuck with a, with a project that's hard to actually get off the ground, and then you just see nothing but accumulation, you know, as you go forward. And I think that's part of the reason why ServiceNow starts to identify you're, you're gonna have to replatform and, and, and, uh, you know, start fresh.

Now, does this issue get more challenging when, I mean, historically ServiceNow has been an ITSM platform, but we're seeing it extended into all kinds of different use cases because, well, I can have a single platform and it's more cost effective, but a lot of the people who are running those other workflows outside the IT department don't even know what technical debt is. So, or are we gonna see more of it because we're gonna have more, shall we say, non-professionals building application environments that for different workflows and they won't even think about technical debt till it's way too late. I, I think you're bang on there and, and just to link it back to, to your question about AI and, and where that's going, that's one of those big risks that I see.

You know, the, the ultimate goal from a an enterprise use perspective would be get the ability to create closer to the actual, um, end user or the business side. But everything you just said now becomes the major risk is what is actually being built. Is it considerate, I mean at the root of it with architectural principles and other elements, is it considerate of what's it gonna be interconnected to and what could it affect upstream, downstream that then affects everybody else?

So it definitely is a bigger risk as you go. What I was saying earlier about, um, you know, having some, some aspects of, of uh, um, uh, visibility and understanding and awareness and creating some governance around that, that becomes the key thing to actually start to solve, you know, those types of challenges. Nothing I'm gonna say is a hundred percent ever gonna be bulletproof.

Things change, things happen, but without something like that, it becomes very difficult to manage business expectations and speed while keeping a stable platform. Of course, everywhere you go these days, people are talking about, well, AI and is this gonna eliminate the need for this IT person or that IT person? But, um, much of what you just described seems like it still requires some sort of human in the middle of this thing so our IT professionals listen to this conversation and basically having a good chuckle about, I don't see that happening.

I, I think it's true. And you know, I even in early days when, when, when at my company we were really investigating where all can we, can we leverage ai? What, what does this mean from what our customer's potential is?

The way I'd always look at it is that this is enhancing development, not replacing it. Um, even when you get into things like really complicated architecture integration, more of that systems level of how is this all gonna fit together? And I'm meeting more peripheral to even just ServiceNow itself, it's gonna be very hard to find something that's gonna understand all the bits and pieces.

So you're always gonna need that level of oversight. The other thing that I always worry about too is that to get that skillset and capability in terms of our workforce, what does that mean in terms of the future and the gap that might be created if a lot of those junior roles that we're gaining the experience to get to that level I just described start to disappear or get replaced by ai, it's a, it's an interesting conundrum sort of down that path. But to your point, I don't think this eliminates the need for, you know, really smart and really capable developers.

What's your best advice then to IT leaders about how to have this conversation with the business side? Because to your early point, it's a little difficult and from their perspective, maybe even a little esoteric. So how do I kind of get folks outside of the traditional IT leadership to wrap their heads around this?

Um, uh, the way that I've seen it best done, to be honest, is making those connections to cost and business impact. And there could be situations where the recognition in terms of how quickly or how how slow it takes before they're actually affected might come into play, but really making those connections that if we're not setting things up in a certain way and structuring things to basically protect what you are using, we're gonna end up ever increasing cost and it's gonna translate to your needs being slowed down, um, you know, consistently over time more and more and more. And how fast do you think businesses are moving these days or wanna move on these ServiceNow platforms because, um, just because we can do something doesn't mean we should or do, but, um, it almost seems like, I think the proverb of something is to the effect that, you know, if you want to go fast, go alone, but if you wanna go far, go with the many.

And is this an opportunity to do the right thing with the many? It, it's a good question. Um, you know, and I think over the last say six years or so, I've seen a bit of peak and valley in terms of enterprise use.

And if I go back maybe about four years ago, I saw very substantial increase in terms of organization, shared service use. So I'm thinking things like, obviously IT facilities, hr, like something where your actual, um, stakeholders in terms of employees would be affected and you could create more of a common experience that would be there. I saw some sort of slowdown in terms of that and, and a little bit more shaping towards it and more IT function, security, those types of areas.

But in the last few years, two or so, I've seen a lot more increase where the idea of enterprise usage and more importantly, overall service delivery and actually getting further than just employee and into customer space or citizen space depending on, um, what sector you're sitting in. Um, that's starting to increase. And we're seeing a lot more large scale enterprise organizations really looking at ServiceNow as a foundational component to enable better service delivery and things that are actually affecting customers and actually affecting the way they generate revenue.

Well, let me ask you this, 'cause you mentioned security and we are starting to see IT teams take more responsibility for at least security operations, you know, the management of the firewalls, whatever it may be. Um, but I can't help but wonder if the way IT teams are organized should change in the age of ai because we built a lot of those roles and responsibilities around the various silos that existed and maybe we're taking the silos down, so maybe we should reorganize the teams. What do you think?

Well think, thinking back to some of my operation days, that, that the reorganization probably creates some different turmoils, but I think to your point, the reshaping though, because there is a lot of shared responsibility that that really starts to come around when you're thinking about who's using AI and for what and what it's actually gonna be affecting. Um, there's definitely security components that need to come into play, but I think there's also gonna be, um, a little bit more visibility and a push on ownership in terms of those users and let's just call it business areas that are trying to adopt to make sure that this is actually meeting and, uh, um, security requirements and being able to be more of a responsible and safe usage. I am seeing a lot more organizations standing up responsible AI tied with security where a lot of the questions are diving into not just, you know, what's the technical aspects of the ai, but what's the business use and the why, what's it related to data, all these different types of elements.

So I think we will see something that becomes a shared responsibility. I, I'm reluctant to say, I think it'll really reshape organizations and that's only just my, my own personal opinion on seeing how difficult that sometimes is. But it's a great question.

So What is that one thing you do see IT teams doing in the ServiceNow environments that just makes you shake your head a little bit and go, folks, maybe we wanna be a little bit smarter than that. Whew, that's a great question. Um, you know, I, I, the, the, the way I probably answer that is, is paying more attention to some of the basics.

And oftentimes what I'll see is some, some ServiceNow customers getting themselves into challenges by sort of ignoring some of the basics in terms of code reviews, in terms terms of creating some of those structures of control that help make sure the right things happen. And there's two things that on that note, there's for sure process and sort of individual involvement in terms of being able to have those right checkpoints. There's also leveraging tool sets that can help to automate those to make it more simple.

And my belief is that a lot of times those get ignored just because of how difficult it is to capture everything and make sure you're, you're, you're seeing things. The worst stuff that I've typically seen is environment drift, where it's just the easiest thing to control. ServiceNow is built in mechanisms, but how they shape delivery, they start to just ignore and drift.

That changes in controlled environments, those types of things. So my gut would be right at just ignoring some of the basics as, as, as crazy as it sounds and surprising. All right folks, well, you're heard it here.

Hey, even in the age of ai, there's no substitute for mastering the fundamentals because if you don't, you're gonna pay for it later. Anyway, Ron, thanks for being on the show. Yeah, thanks so much, Mike.

All right. And back to you guys in the studio.