Software. Efficiency. Scalability.

Technical debt

You have probably heard the term “Technical Debt”. All-knowing Wikipedia describes it as:

Neologistic metaphors referring to the eventual consequences of slapdash software architecture and hasty software development

Sometimes when you need to ship or make a release you have to cut corners. You know it’s not the best thing to do, but it’s a quick thing thing to do, and you need to ship your product.

There is nothing wrong with cutting corners to make a release. In fact, if you don’t have to cut corners, you are shipping too late. Virtually every single software project that shipped had to cut some corners. Sometimes more, sometimes less. No matter what, there is technical debt associated with each project.

Understanding technical debt

It’s important to understand what debt is. Debt is a claim on future labor. It’s an obligation to either do something later or provide resources (money, for instance) that are equivalent to labor.

When you owe money, there are 3 ways to get rid of debt: pay up, default, or print money (if you are a government). Now, most of us can’t print money, so it boils down to either paying the debt or defaulting on it.

The last important thing to understand about debt is interest. Because you get something right away and promise to pay for it later, you usually have to pay for it in a form of interest. This is why, when you borrow money from a bank to pay for your house, you have to pay interest to the bank. You get get the house right away, and the banks makes money.

Technical debt is not an exemption to any of these rules:

1. It’s a claim on future labor. You promise to go back and fix some of the issues that you didn’t have time to do right in the current release.
2. Down the road you have a choice between working on those items or defaulting on your debt by doing nothing. Just like with normal debt, defaulting on a debt comes with consequences. I will describe them in a moment.
3. Fixing something way down the road tends to be more expensive than fixing it right away. Partly because developers move on and don’t remember all the details, partly because there can be new components that expect you to work in current non-optimal way.  This is the interest you pay.

Why no-one repays technical debt

While there are always intentions to fix something down the road, more often than not it doesn’t happen. New issues arise, new features need to be shipped, priorities change, etc. It is a lot more common for businesses to pay technical interest until it becomes unbearable, default on the debt, and then file for bankruptcy in the form of deciding to rewrite the whole damn thing from the scratch.

Let’s get this straight: rewriting your software from the ground up is an expensive thing to do. Yes, you lose your technical debt, but you also lose a lot of your assets. Just like when you file for bankruptcy, you are likely to lose some of your assets. Usually, a very significant portion of them. The same rule applies to software project: you will lose all the good code that you’ve written, tested, debugged over the years. It represents a big investment. While some of it might be salvaged, a lot of it will be gone. It will have to be re-written, re-tested, re-debugged. New deadlines will have to be met and guess what? New technical debt will be accumulated!

We see this cycle over and over again. Companies abandon old projects and announce moving to the “new and better” projects all the time. Why?

There are several major factors contributing to it and they explain why it happens so often:

1. Business people don’t see technical debt. If you think they understand your blabbering about re-factoring a piece of code to work faster you are wrong. They can humor you by pretending they understand but they don’t. If it works – it’s done. They want new features for new customers. New customers bring new revenues and bigger bonuses. System being sluggish and crashing often is not their problem, it’s the fault of developers and hosting people.
2. Perceived low ROI. This is a direct consequence of #1. When talking to business people you can often hear “It doesn’t generate us new revenue so it’s low ROI”. The problem is that ROI stands for “Return On Investment” and we are not talking about investment! We are talking about paying off debt. Debt is not an investment, it’s something you owe. When you buy a house you borrow money from the bank, creating debt. Then you invest borrowed money into the house. If your house goes up in value you made a successful investment, but it doesn’t change the fact that you still owe the bank. Now, think about it as saying that repaying your mortgage has a low ROI since, you know, you already have the house.
3. Developers don’t like repaying technical debts. Well, let me correct myself. They despise it! Writing new code is more fun then fixing old code. Especially if its not written by you. Developers might not like creating technical debt (gah, have to cut corners again!) but fixing old code is not something developers crave to do either. Let’s be honest about it: developers would rather bitch about how the whole thing stinks than fix it. Oh, it’d smell like roses if only they had a chance to rewrite the whole damn thing.
4. Development processes don’t focus on it. Modern development processes gravitate towards features and bugs and tend to ignore technical debt. There are no user stories for products requiring a lot more servers to scale, for having to struggle with strange joins because database schema is designed badly, for developers spending extra hours working around legacy code quirks, or for having a much steeper learning curve for the new people. The list goes on and on. Unless it gets unbearable and produces acute disruptions to the business process (outages, slow beyond reasonable, etc.), modern development methodologies tend to discourage long running maintenance projects.

All these things contribute to not repaying technical debts. Nobody likes it, but that’s just the way it works.

Conclusion

When working on complex software projects, technical debt is a fact of life. It’s created often and willingly and it’s extremely hard to repay. Once enough debt is accumulated the project is deemed ‘old and crappy’ and a ‘new and shiny’ project emerges. While it’s a terrible business practice and comes with a huge cost in time and money, it’s more of a norm than an exception in the modern software engineering.

In the next post I will show one way to repay your technical debt on time and have a stable technical economy for your project.