Paying down technical debt

Is all technical debt bad? Can it serve a valuable purpose? We share our opinions along with some tips on how to be strategic when paying down technical debt in your project.

Show Notes

Have a question? Want us to discuss a topic? Do you like sending email to random people on the Internet?
Contact us at [e-mail] for all of the above.

Creators & Guests

Host

Aaron Saray

Host

Joel Clermont

What is No Compromises?

Two seasoned salty programming veterans talk best practices based on years of working with Laravel SaaS teams.

Joel Clermont (00:00):
Welcome to No Compromises, a peek into the mind of two old web devs who have seen some things. This is Joel.

Aaron Saray (00:08):
And this is Aaron.

Joel Clermont (00:16):
Recently I saw a tweet. It was a little tongue-in-cheek but it was the basic idea of there's two types of engineers. One enjoys complexity and views it as job security and the other avoids complexity at all costs. I get where they're coming from but my immediate hot-take reaction to that was, I honestly think most teams that have legacy software that's super complex and lots of technical debt, it's not a willful choice. It's really a lack of action that got them to that place and they would love to get out of it but they just don't know how. I think 80% of developers hate complexity and just don't know how to stop it.

Aaron Saray (01:00):
I think I would agree with that. I also think a tweet like that is just meant to get so much harder raise out of you. There's only two types of developers and you're probably one of them. No, I think you're right. I've run into people where I've really kind of felt like it was unsaid that they kept complexity sort of to secure their job. I was like, "What are you doing? You don't need to do that. You're making everything worse and you're kind of being selfish." Some of those people do exist, but I think you're right that it's probably a lot more in the middle. Especially when it's like, "I don't know how to get out of it." Or more so like, "I have some ideas but how do I make sure we can... How do I make the value? How do I prove the value of getting out of this complexity?"

Joel Clermont (01:48):
Yeah. Sometimes it's a time element and other times it's just like element of fear. Like, "If I start fixing this one thing it's going to lead to me rewriting the whole app or like 20 other things are going to all have to change to support this and I just can't even fathom that." Yeah, the part of the reason too other than that tweet that I was thinking about this is just this idea of when to pay down technical debt is because you and I are dealing with it right now on different projects. But kind of getting into the guts of a fairly thorny complex technical debt situation and just ripping things out and fixing it, but also kind of capping it to not have it turn into that scenario where you're rewriting the app. So maybe we could talk through some specifics of what we're working on and decisions we've made around how to go about doing that.

Aaron Saray (02:42):
Yeah. Well, I think even before we get started into that I want to talk a little bit about technical debt and why it isn't the worst thing in the world though.

Joel Clermont (02:51):
Okay.

Aaron Saray (02:53):
Here's a humble break. When I was doing my work and my business degree, one of the things that we had talked about was how to leverage specific debt in your business and understand that businesses with debt aren't ugly assets, that they actually are positive. The idea was that if you can rent some money, like go into debt and make more money because of the money you've rented, you can then pay that off and have had more money than if you just sat around with your own money. The idea is, if you can get a 3% loan, an API with 3%, but you can make 10%, in the end you're making 7% with someone else's money versus having to put in your own money. The only reason I bring that up is because I think a lot of times we think about technical debt and we say, "Oh, horrible, horrible, horrible." It's like, no, it's just one of the many tools that we have to keep in balance in our projects.
I've worked for companies and clients that start out saying like, "We're never going to have any technical debt." I'm like, "Well, then you're never going to get anywhere." Sometimes you have to do these things that are going to cost you later in order to be able to obtain the things that you need now. And that's hard as developers too for us to think like, "Sometimes I have to maybe cut a corner that I know I'm going to have to come back later and fix, and why don't you just let me do it the right way?" Well, because sometimes it's what you need to do to move forward fast enough to be marketable or hit a need, or whatever. As we talk about paying down these technical debts, I think we all kind of agree that we'd have them but I don't want to make the idea of having some technical debt the super evil thing. It's just that what we want to do is we want to try to make it in balance first.

Joel Clermont (04:43):
Yeah, you don't want to overcorrect too far the other direction. In your analogy there with financial debt to technical debt, who's the banker in the technical debt example? Is it a business owner where they're going to hold you accountable or is it like a team lead, a CTO? Like, who? I'm just curious.

Aaron Saray (05:01):
Well, yeah, the banker is the market or whatever problem you're trying to solve. It's not a one-to-one I guess but you basically are saying, "I need to solve this market." The market is asking for something for my product and I can either miss when it needs it and have it perfect, or I can give it to it now and understand that I'll have to work a little extra later to clean up my mess. The way you get the money is to give it now and you kind of clean up the mess as you go along. The problem is when people just never clean up the mess.

Joel Clermont (05:35):
Right. Well, that's where I was going with that. Is like, if your team is actually making a decision to take on this technical debt it should have a mechanism in place to sort of track it. Like, an internal guideline is like, "Okay. Now, when do we clean this up?" Not just, "All right, we'll deal with that at some undetermined point in the future," and then you never get there.

Aaron Saray (05:55):
Yeah. There's a lot of different mechanisms and I've used in the past where I've labeled things. When I'm working on tickets or a feature, I might end up finding that I make another ticket with a specific label on there called, like internal or technical or something like that, so I don't lose track at the time. Because I can tell you what, anyone who puts a to-do in your code, you're just never going to do that. I mean, there's no visibility there either. There's no visibility for the managers to see it or for the stakeholders. They're not going to look at your code and be like, "Oh, it looks like you have 27 to-dos." But they can see it in a ticket system if you start to kind of put those out there. And the idea about that then is you make these negotiations with your stakeholders and you say things like, "Okay. For the sprint or whatever, we're going to do two features and one technical internal task." They can still see there's making progress and you're still cleaning stuff up.

Joel Clermont (06:47):
Yeah. Those to-do comments resonated with me because the particular project I'm working on has, I don't know if I counted them, but I would easily guess over a hundred and some of them are not even relevant anymore. It just goes to show they kind of rot in the code base and nobody is actually looking at them. In this particular code base, just to maybe to give a concrete example of the sorts of things we're talking about, this was an older Laravel application, actually Laravel 4.2, and it had never been updated. So went through the work of doing that but there's like this question then. Okay, so that in itself is paying down a lot of technical debt but along the way a lot of things have changed. One specific example is, there used to be form and HTML helpers in Laravel 4.0 and those got ripped out of the framework. There's an extra package that you could bring in, that Laravel Collective package that brought that same behavior back.
Now, to make things even worse this particular project extended that from helper and added its own methods on top of it and so it wasn't working, right? After everything had been updated those extensions were no longer working. And I had this thought, "Well, should I just rip all of this out? Do we really need ( 00:08:05) or can I just put a form tag in here?" There's a lot of things that you could mechanically make that change. But I made the decision, "No," that was going too far. Yes, that having a reliance on that package was technical debt but it didn't really give me the same benefit that getting Laravel to version 8.0 did. So I made the decision like, "All right, at some point in the future this will probably have to be dealt with but right now is not that point." Because I might have added a week or two weeks to the project and held up all that other good stuff, good value, you know, fixes, improvements, in the meantime.

Aaron Saray (08:46):
Well, you've made a great point there. Which is, I'm going to talk about that 3% interest rate again or whatever, right?

Joel Clermont (08:52):
Okay.

Aaron Saray (08:53):
Maybe every month you're paying 3%, so $3 on your loan or whatever. Would all of the money and time you spent fixing this gain more than that $3 every... Could you have done something else that'll bring in more money? The debt does cost you, it does slow you down but maybe it's just irritating and doesn't really slow you down that much. Well, then you have to kind of suck it up and say, "Not every job is perfect and sometimes I have to deal with this in this project because the upside of this is I'm just happier." Which okay, cool. I mean, there's other things like security holes and things like that where it's like, well, there's a tremendous amount of risk so it's not so much like missed opportunity cost. It's risk. But there's a lot of times you have to ask yourself too, it's like, "Fixing this particular technical debt going to gain the whole project or is it just for me?"

Joel Clermont (09:52):
Yeah. In this case it was pretty clear cut, it just was annoying me. Like, to have to deal with this. It's like, "Ah." But it's also why it's good to have somebody to talk to. Aaron, you might have talked me down from that legend fact. Like, "Do you really want to do all this work right now?" And so I'm happy with the decision.

Aaron Saray (10:09):
I think before we move on from this, the last kind of little idea I wanted to talk about is when we talk about this technical debt sometimes it's hard to make the case of why it matters. We kind of joked around a little bit saying like, "Oh, it just could bother you," or something. That's not necessarily something to just throw away, it's just something to understand that that's one of the many costs that there is. There's many costs of technical debt. There's missed opportunity of doing stuff because it takes longer to do a thing. There is missed features because those features aren't available in this older package. There is productivity and that could be a productivity of the app itself, it doesn't go fast, it's slower. We can miss sales or we could make our customers upset.
There is developer sentiment and all those different things you have to, as developer, kind of make the case when you talk about technical debt and wanting to solve it is, you have to put it in the measurements of what your stakeholders understand. So sometimes, I'm not saying you need to throw a fit but I have told a stakeholder or two, like, "Hey, we've put this off now for seven or eight months. It's really making me frustrated to work on this project, I really just want to fix this little bit. I understand it's not really going to help a lot, but it'll make me feel better to work on the project if I could just do this. It's only going to take me a week. Can I do that?" A decent person, you know, stakeholder is going to say, "Yeah, fine," but you can't ask them that every week. But they're going to allow you to solve some things like that. That's the thing about this technical debt is to actually kind of tie it back to what the cost is and then also assume that your stakeholders don't fully understand what that term means and what the drawbacks of having it are.

Aaron Saray (12:08):
All right, do you remember CDs?

Joel Clermont (12:11):
Yes, they weren't that long ago.

Aaron Saray (12:14):
Yeah. I mean, they were in the before times, whenever that was. But, yeah, I remember that and I remember that they were like 60 minutes and then there was one that was like 72 minutes. And you're like, "Oh, that's super. That's a lot of music you can listen to." But the one thing that I really remember about CDs, and I think they're on CDs definitely more than on than on tapes, was the secret track or the hidden track. Where it was either at the last song, then there was like nine minutes of silence and then there was another song. I remember there was one where it was like they had 14 songs and then track 35 was another song. You just had to skip through a bunch of empty ones. Do they have that these days? Do we have the joy of a hidden song anymore?

Joel Clermont (13:03):
You know where I encountered it? Is when I ripped one of those CDs and it's in my iTunes library right now and then all of a sudden I'm like, "Hey, why is no music playing?" Or I'll forget about it and then boom, music will come back on and I'll jump out of my chair. But, yeah, Spotifyland, like that's streaming services. Is there a hidden track anymore?

Aaron Saray (13:22):
Yeah. I mean, on movies sometimes they have another scene at the end of the movie. A lot of the Marvel ones did that. The other thing with CDs too is remember like a Diskman or something like that, where you could have the portable CD player. Kids these days will just never understand the decisions we had to make. Which was when you put in the AA batteries, do you want maybe 30 minutes of skip free music? Or do you want to play a CD for six hours but it's going to skip whenever you hit a bump? Because I don't know if you remember that, they had that skip thing where you put on skip protection. I guess I must have just scanned the CD ahead of time but your batteries were dead halfway through the CD but they didn't skip.

Joel Clermont (14:10):
Right. Yeah, that's especially important like riding the school bus or something, it would skip all the time.

Aaron Saray (14:15):
Then you look over, there's your friend sitting on the seat holding his CD player like a platter on his hand and he's trying to absorb the bumps on the route.

Joel Clermont (14:25):
The human anti-skip.

Aaron Saray (14:28):
I remember listening to... I had one of those adapters for the tape player in the car so you could put in this fake plastic tape thing and it would play as if it was a tape, your music on your CD. I'm just explaining for all the kids who are under 40 listening.

Joel Clermont (14:43):
Like, with the cording out of the tape?

Aaron Saray (14:45):
Yeah, there wasn't even Bluetooth. I remember driving down the road I had windows down and after listening to your CDs for a little while, they get skipped and they would scratch and they'd skip anyway. I'm driving down the country road and it's bumping around anyway, and this CD is skipping. I was just so mad, I was like, "This damn CD." I decide that because I was young and an idiot, I was going to throw it out of the window. I just took the CD, threw it out the window but I neglected to take it out of the Diskman.

Joel Clermont (15:20):
Okay.

Aaron Saray (15:21):
So I just threw the whole Diskman out of the window with the CD in. I was like, "Wait, what?"

Joel Clermont (15:27):
Did the tether save you to the cassette?

Aaron Saray (15:30):
No, it just came right off. So I just threw away my Diskman it was like a hundred bucks. I'm like, "Well, how do you explain that when you get home?" "Oh, where's your Diskman?" "It flew out the window. You know how it gets so bumpy."

Joel Clermont (15:42):
One last anecdote related to music and cars. I had an MP3, one of the early MP3 players. It was The Nomad, remember that? The Creative Nomad and-

Aaron Saray (15:53):
Oh, yeah.

Joel Clermont (15:54):
... it looked like a Diskman, it was about that size and shape. But anyways, my dad and I were on a road trip. It was the two of us driving cross country and so we were listening to some audiobooks. One of them we queued up, I think it was Hobbit or Lord of the Rings, one of those books. After three hours, we realized it was on shuffle because the story made no sense. But it would play a chapter and then it would shuffle to another random chapter and I had read the book, so I was kind of following it but I'm, "Something is not right here." Then I realized it was on shuffle.

Aaron Saray (16:29):
Hey, how are we doing? Do you like this podcast? Do you like our other material?

Joel Clermont (16:34):
We'd love to hear from you. If you have a specific question or like a topic to be discussed or just have general feedback, send an email to podcast.nocompromises.io.