My LinkedIn title has been “Tech Debt Loan Shark” for years and it has garnered some of the funniest cold opens from recruiters over the years. I don’t think there’s a real problem with admitting technical debt at healthy engineering orgs.
I call myself a “Loan Shark” because if you aren’t given an option by leadership to take the time to do something right, “admitting” technical debt is less about “taking blame” than it is about covering your ass. When that debt comes to bite the company later, bring your receipts and use them as leverage to do some proper technical investment driven by the engineers on the ground floor.
My own project has a number of TODO: comments in it. Some of them could be accurately described as unfixed bugs. Others are more like 'feature requests'.
When I am in 'bug fix' mode, I will try to locate and fix a few of them. I try to allocate a day now and then to focus on cleaning them up.
When you are in startup mode, trying to find the right product-market fit you cannot afford to make the code perfect. You have to get it to the point where potential customers find real value in what it can currently do. Shortcuts are a byproduct of that process.
Even if you can't get to it right away, a quick note of the issue in the code itself can make it easier to address it later when the problem is not fresh in your mind (or if someone else is tasked with doing it).
We run on a debt-based economy so it should be no surprise that consciously issuing and managing tech debt works for growth in similar ways IMO.
Sure, it's possible to live debt-free and manage finances or architecture frugally - and I consider that a noble thing - but you will have a hard time keeping up with those who keep covering up their debt with more and larger debt.
Yes, and I think that is what makes risk assessment hard.
All debt packages risk. However, the kind of debt/risk perception we are personally conditioned around in daily life does not translate to software. Managing software debt requires investment banker-like thinking, rather than the mindset of a credit card or mortgage holder.
Not sure if the code is the best place to track technical debt. Whenever a compromise made that results in tech debt, a ticket should be created in the issue tracker. That way it's easier to get an overview about the project's health.
in my experience no one doubts that the debt exists, rather people just don't see the same value in addressing it, and aren't easily convinced by arguments they've already been ignoring for years or decades
i'm one of two feature devs on a team of four. over three years the team wrote a few hundred tickets for debt. when the client was acquired and we finally got space to address debt, non-technical stakeholders asked us to nuke the list and write a much shorter one that we can prioritize by complexity and customer impact. the debt is now being addressed in the same way as new build, because that's how we treated it; much like new build, it won't be completed to our satisfaction
in my experience the only thing that works, short of a major incident that devs forecasted and offered to prevent, is buying time--either to just write better code up front, or to use on _undirected_ remediation
People always have stories about technical debt at work and open source, etc. Does anyone have stories of projects with no technical debt? Perfectly maintainable code?
We like to blame bizdev and sales for putting pressure on us, but that feels like a cop-out to me. Maybe I'm just a bad programmer but even if you took out the deadlines and "requirements" I'll probably still ship code with some warts here and there. Complaining about people in other positions is no way to improve.
Sorry, maybe a bit of an extreme take. Am still interested in stories of especially maintainable codebases, as I'm probably over-correcting here.
I do a lot of technical due diligence for companies in a bunch of different contexts (private equity pre-acquisition, post acquisition, management change over, etc). I have found that tech debt is highly correlated to the business goals and tech organization structure, and has to be evaluated in those contexts.
Startups you want to see with fairly high tech debt. Tech debt there is a very often a sign of shipped code that is meeting some customer need. You do need to watch out for insane tech debt that may be hard to undo later on (running everything in an ancient proprietary language for example).
Startups with low tech debt often are devs navel gazing without a lot of revenue coming in. This is surprisingly true almost all the time.
Very large companies will tend to have a mashup of all kinds of technology that is never fully untangled. A very large Corp with little tech debt is exceptionally rare. If it ain’t broke don’t fix it is the rule.
Mid sized corps and late stage startups seem to be the sweet spot. Fairly low tech debt systems that is providing high value is still possible. This is where you have enough maturity to know what “good” looks like, and enough developers, momentum and budget to implement the vision.
One small anecdote of a project at work. The oldest current project. All our projects are fairly simple and similar. They all use libraries and templates. This one is the last of the early projects before those templates and libraries. It's the easiest to manage. Things rarely go wrong. No forced upgrades because of a library vulnerability or template update. Bugs have been always been straightforward to find and fix because everything is procedural and clear. No abstraction 7 layers deep. Just clean, simple code.
Code doesn’t have to be perfectly maintainable and „elegant“. It just has to be useful i.e. good enough. If it solves your user‘s problem, that’s good enough. If your development speed can keep up as you add new requirements, that’s good enough.
Doesn’t mean you should ship untested code because that would decrease both the value to the user and your development speed. There are projects out there that go into the other extreme, entering the realm of premature abstraction and accidental complexity. In my experience that’s even worse because in that case it’s hard to convince stakeholders that there even is a problem.
It‘s a tough pill to swallow for devs with perfectionist inclinations like me, but at the end what matters is providing value to the user. In fact, I think technical debt will always be present as long as requirements change, and as the saying goes, change is the only constant in software engineering. It‘s not something to be completely avoided as that’s impossible. It just needs to be managed in a pragmatic way to keep up velocity.
Every engineering team I've worked with virtually every engineer complains about technical debt - regardless of how much time is allocated to manage it. I'm not suggesting it isn't important to take care of it, just that sometimes it comes across as the boy who cried wolf.
I have micro-services that haven't been touched in years because they do one thing, and the requirements of that one thing never change.
It is possible but only in your own projects as tech debt is the result of:
Programmers who aren't you, and those damn MBAs ruining your flowstate
Technical debt is entered into deliberately, as a trade-off for time that must be paid back, with interest. There's no technical debt that isn't "self-admitted".
Technical debt is entered into deliberately, as a trade-off for time that must be paid back, with interest.
The implication here is that if only you had enough time you wouldn't have any tech debt. I don't think that's true.
Some of the tech debt is due to not having enough knowledge about the future of your system; what it needs to do, how it'll be used, etc. You have to make a decision about the tech now and sometimes that decision turns out to be wrong, or at least incomplete, which leaves you with some debt that will need paid off some time.
I also believe some tech debt is unavoidable due to churn from outside of the system. Best practises for a given part of an app aren't fixed (eg React class components shifting to functional components), so sometimes something that was once good can turn into debt over time with no input from you.
Publicly self-admitted, in this case. Apart from that: if I inherit the code base of a bad programmer on my job, with a lot of technical debt, which part of it is a deliberate trade-off? It's still "time that must be paid back, with interest", but how it came to be is not part of the definition. The term is merely meant to help confer the problem to non-technical people, who _then_ can make a conscious decision to accrue technical debt. If they do so out of incompetence, it's still technical debt.
> but how it came to be is not part of the definition
It's possible the people working at the time were not conscious of the trade-offs they were making. But it still happened and that's how it came to be.
> if I inherit the code base of a bad programmer on my job
It sucks, but it's almost guaranteed to be the case, to some degree. Think from the company perspective. You're thinking too much from your personal perspective, which is kind of irrelevant to the company. You're not using the term just for you.
> Think from the company perspective
I do, rest assured, part of this is my job description. I'm just bumping over the assertion that a deliberate trade-off is a prerequisite for technical debt. It's too specific a reason, and like saying your expenses are higher than your earnings, that's why you are in debt. But in reality, there are many more ways to end up in debt, spending more than you earn is just one possible cause.
If you identify technical debt, either hypothetical or existing, the term is an option to explain this to your superiors in a way they might understand. They'll know that they need to do regular small investments to pay it off, or chose an alternative option to avoid it. But that's all there is to it.
Your bank goes bankrupt and you lose your funds. A good you purchased turns out to be stolen. You lose a lawsuit. You purchased a company in an equity deal and the company turned out to carry liabilities (just a note, I'm not from the US, so details might actually be different there).
Likewise, for technical debt: a dependency you relied upon is not maintained anymore. There is an exploit found, or regulation changes, and you realize you used the forbidden pattern everywhere.
Basically: earnings minus expenses are the operating result. Concious decisions to accrue technical dept would be the equivalent. But there are costs directly affecting equity as well, which can still result in debt.
I think the trade-off is at the company level. They hired a worse programmer to save money/time back then and accelerate development in the short term.
How would they be in a position to judge programmer competence? It might well be the person was hired based on being cheaper and/or a personal relationship (boss' nephew or manager's room-mate) but that is not necessarily an indication of how good they might be.
More often it's just a sign of a new company not knowing what they are doing and making unforced errors as a result, rather than a conscious debt/investment transaction.
Yes, that probably is more likely. I was merely trying to paint the decision makers in a good light to offer an alternative viewpoint. I usually default to the managerial incompetence view, wanted to have a change.
Yes, I do think it's a geniune term. Just like real debt, it's a very useful tool and many things wouldn't be possible without it.
Technical debt isn't inherently bad. Otherwise it would be called "techinical loss".
Agreed, just like real debit it isn't necessarily bad as long as you have a valid plan on how to repay it. And, like real debt, taking on technical debt may allow you to do things that wouldn't be possible otherwise (getting a product to market in time for example), similar to how if you refuse to borrow money to buy a house / car but save up for it instead you'll probably have to wait a long time before having it.
The problem though is that in the financial world there are rules and often laws about how much debt is acceptable and a bank will generally refuse to let you get into dangerous levels of debt whereas there are no such rules concerning technical debt so a company can keep piling it on until it eventually causes a collapse.
Not always. Technologies get old and have to be replaced over time. Tech debt is now mostly treated as a part of a product and developers do not decide what to do about it, but bussines through roles like product owners.
We have technical debt mostly because business guys want it done yesterday with as little people as possible.
I always tell POs that if they want X done until Y day, that is going to result in tech debt.
When they are asking later why Z doesn't work properly or why it takes time, I just remind them of that technical debt they've agreed on.
Does this change anything in the long run?
It certainly does sometimes.
This kind of "debt" is seldom "technical". It is a symptom of all manner of factors. In software organisations the symptoms of trouble become visible in codebases and/or behaviour of systems. Thinking about the artifacts as the problem area tends to cause fixing of symptoms and fringe causes, not systemic causes. As far as I've experienced, this sort of "narrow solving due to narrow framing" is endemic in software. It takes a lot of culture building for things like Gemba walk to work in practice, i.e. sure, root about for bad smells because that's how you discover something's fishy, but then work at a systems level to root out the systemic causes and malincentives.
edit: use better words
Never heard the term Gemba walk before. So I've looked it up:
Gemba means "the actual place" (where value is created).
The core idea behind the Gemba walk is a follows:
> Gemba (aka Walkie Lookie) walks denote the action of going to see the actual process, understand the work, ask questions, and learn. It is also known as one fundamental part of Lean management philosophy.
> Taiichi Ohno, an executive at Toyota, led the development of the concept of the Gemba Walk. The Gemba Walk is an opportunity for staff to stand back from their day-to-day tasks to walk the floor of their workplace to identify wasteful activities. The objective of Gemba Walk is to understand the value stream and its problems rather than review results or make superficial comments.
We stopped some devs in our project from adding too may TODOs and FIXMEs.
For good reason; the more TODOs and FIXMEs a codebase contains, the less impactfull they are. We had so many TODOs no-one every batted an eye when seeing them, and they never got fixed. It just made you feel bad.
Also a lot of TODO's expressed opinions about the "ideal" scenario, but in practice most of those TODOs were for code that was "fine" or "good enough". In those scenarios it's better to omit the TODO/FIXME keyword and just describe the design decision that was made.
Despite their name, i think TODOs should not be treated like tasks to complete, you should not let them make you feel bad. Instead, they are kind of a shorthand documentation of what's going on in an implementors mind, what extensions did they think of? Where did they feel improvements could be made? When changing or rewriting the code, these thoughts can be considered.
Oh, this smells…
If you have mostly "fine" and "good enough" code I fully understand why people want to put TODOs and FIXMEs everywhere; as the code is likely a technical debt catastrophe all over the place. (Because that's the usually meaning behind wording like "good enough".)
If nothing of this got ever fixed this means the devs never got time to do that. Just the next red flag…
If a manager would tell me I shouldn't put TODOs / FIXMEs into the code I would be on my way out pretty quickly. As this means they constantly produce tech dept but actually want to hide this fact.
Disagree, but this maybe stems from different interpretations of those keywords. Too often seen, and similar often in the aftermath realized myself placing TODOs at places that really were just good enough, bug-free, served all business needs, and mostly could have been a MAYBE or NICE2HAVE (if not totally omitted, because just distracting and not relevant).
Often however those ideas are also wrong perfectionalism and can lead to more bug potential or complexity to understand, like trying to create the perfect abstraction while you only have n=1 use cases, especially the younger devs are prone of. (Or one dev believing this part should be written like that and that, while 5 other devs would disagree, that's the best.)
TODOs should be really for stuff that is still missing and needed, because of a requirement or some real ugly workaround taken and tech debt created, and not just because a dev has feelings about some code. Similar FIXME also only for really broken stuff or corner cases one for whatever reason cannot fix right now..
> If a manager would tell me I shouldn't put TODOs / FIXMEs into the code I would be on my way out pretty quickly. As this means they constantly produce tech dept but actually want to hide this fact.
Noone said that (just not too many), it is not black and white. Often what one sees as tech debt is only wrong perfectionalism, that in the end doesn't help. If you need to place TODOs and FIXMEs often, then why not do them right now? If they are really necessary and you need to put them often like once a day, then actually I would question the place.
Placing TODOs and FIXMEs is right where necessary, but please not once a day for something that maybe could just be done better.
I btw like the "No TODO without a ticket" rules, they help with this problem automatically :)
It's just as likely that it's "TODO: add handling for this case which is out of scope during design and which no user is likely to ever request, because the developer argued that it should be there and got shot down"
A TODO means you have identified a way to make the code better. Since not everything needs to be at the highest quality, it is helpful to say what will benefit from being improved for future programmers.
If the TODO is not really "to do" before release, I change the name to: PERF (performance), MAYDO (nice to have) or FUTURE. If it doesn't break the user's story then it's not a real "to do".
I also leave comments for obvious good ideas that don't work, in case someone has the same idea later.
The implication that a TODO is a bad thing is wrong. A TODO is an opportunity for refactor. Yeah the code 'works' today, but it could be improved. A TODO means there was a blocker (in my experience: time).
Someone should write a GPT-3 based technical debt detector.
I also don’t think tech debt is a problem in and of itself. And I’ve worked a number of startups with serious tech debt that was holding back velocity.
I really enjoy the challenge of plotting a course out of a hole like that. However in my experience without management buy-in it just won’t happen. Even if you as an engineer do the planning work to make it as easy as possible to pick off low hanging fruit, you’ll typically be seen as “off task”. You’ll get lip service about how code quality matters, because without that they’ll scare away the other devs. But they’ll be hoping you give up and just focus on delivering what the management team is asking for each quarter.
It takes a ton of patience and management hand-holding, on the scale of 2+ years to shift the culture. And you have to be willing to forgo any chance at personal advancement for that time. Because you’re “off task”.
I think you could take that approach and eventually get an advancement bump around the 3-4 year timeframe but it’s not the fastest way to advance by any means.
And the total failure rate for an endeavor like that is pretty high. Maybe more than half.
Still, it is fascinating work.
That's hilarious! Does that make Management "Technical Debt CDO Traders"? :)
"Loan Shark" nicely anchors the business consequence of piling debt on top of debt. The interest rate ain't gonna be your safe little bond interest rate, it's gonna be whatever the situation says it is.
> has garnered some of the funniest cold opens from recruiters over the years.
Some recent recruiter email subjects:
- Open to connecting about tech debt?
- What's the going rate on a tech debt loan these days?
- Tech Debt Loan Blue Whale Opportunity