Software. Efficiency. Scalability.

In the previous post, I talked about technical debt. It’s an essential part of any software project that has shipped with time line in mind. However, at the same time it’s one of the major reasons why projects have to be rewritten from the scratch so often.

You do want to manage your technical debt and there are ways to manage it.

All you need is time

As I mentioned before, there are only 2 ways to get rid of debt: repay it and default on it. If you are trying to avoid a default, you don’t have a lot of options. You have to repay it.

Debt is a claim on future labor. That labor needs to be provided, which means someone needs to put the hours in. It is possible that you will need to spend more hours down the road than you would have spent initially. That’s fair; it’s the interest you pay.

The problem becomes getting these hours from the management. It needs to be timed right and explained in a way they can understand.

Feature holiday

One of the most straight forward ways to deal with the problem is to set aside a period of time right after the release. No new features, only bug fixes and re-factoring. Depends on your release cycle it can be a couple weeks to a month. Either way it needs to be a significant amount of time so that you can actually go and implement important changes.

The upside of it is you can ask for this time before release, while your management is focused on getting there. It won’t seem like such a big deal to do it right after the release, and it might be an easy sell. The downside of this approach is that you don’t know how much time you need. Sometimes you need more time, but sometimes you need less or not much at all.

Technical Tax

Another way to get the time needed is to tax your business people for hours they spend.

Management generally can comprehend the idea of maintenance and doing some re-factoring. Depending on your organization type, you can explain it using different methods. Say, If you are in the Agile camp, you can say it’s in the Bible Manifesto: “Continuous attention to technical excellence and good design enhances agility.” Just ask them if they want to increase agility, or if they don’t believe in Jesus Martin Fowler.

What it really comes down to is taxing your business people for hours you need to spend on maintenance. They don’t get to have those. Those hours are not theirs. The rate can go up or down, depending on the situation, but the hours need to be taxed until the debt is repaid. Let’s call it Technical Tax.

How to tax hours

Lets assume that as a developer you work 40 hours a week. The actual number is irrelevant, some companies use points, some allow to work 1 day a week on your personal project, etc. But say you work 40 hours a week. You have 3 people on your team, that comes down to 120 hours a week.

Your team worked 240 hours a week for half a year to get the first release shipped. You worked so hard that your girlfriend left, but you only realized it when you accidentally turned on your phone and listened to your voice mail. It was the only message there, as everybody else pretty much gave up on calling you. However, there is a new day and the project is now shipped. You can relax and work your 40 hours a week, like a normal person.

Of course, you cut a lot of corners trying to ship your software. Now you need to go back and fix it, refactor bad code, implement all the little optimizations you had in mind. This is where you find out that your product manager insists on implementing a list of new features that the sales want in order to close a big client. You are told that re-factoring something that works is “bad ROI”.

This is where your team needs to tell management that paying debt is NOT an investment and start taxing the hours that business people get. Figure out what refactoring you need to do, create a plan, go to your business, and tell them you need XX hours a week to fix what needs to be fixed. Just like ‘feature holiday’, the best time to implement something like this is right after a release. It may vary from project to project, but it should be around 10-30% of the total hours. At 15% your team will get 18 hours a week, which means one of you can spend 2 days per week repaying the debt, refactoring code, etc. Business people get to play with the rest of the hours, but they can’t touch these hours under any circumstances.

If you feel it’s not enough — negotiate a rate increase. It’s not going to be easy; nobody likes tax hikes. The trick is, though, once you have the tax system in place, you will be debating how many hours you spend paying off your technical debt, not whether you do it or not!

If you have too much debt, you might need get more people on your team or improve your practices. If you need to release something fast you can announce a tax holiday, but always set the end day for it and always try to get more hours after it’s done.

How to manage technical tax

If taxes go too high, people will try and avoid them. The same is applicable to the technical tax. Keep it as low as possible and people will justify it by having a better product. Less technical debt means you will be implementing features faster, your product will perform faster, it will scale better, it will require less maintenance. Before you know, business people will fight to get the credit for implementing it and boast around how they outsmarted the developers.

There are a couple things you need to watch out for:

1. The tax will only work if you use it to repay the debt. It’s not your ‘do whatever you want’ time. You do what needs to be done and then it will work.
2. If you run out of critical things to do, you should reduce the tax. Always remember that it’s a necessity and that the goal is to minimize the debt, not maximize it.

It’s OK if the tax goes to a very low amount, say 2 hours a week for just reviewing that everything looks good. Just make sure it doesn’t go away so you don’t have to have a debate about implementing it again.

Conclusion

Use technical tax to strengthen your project and to keep it in a state where it’s ready to expand, ready to accept new clients, ready to scale, ready to be deployed on more servers. You’ll never want to rewrite your application from scratch ever again.

Technical debt

You have probably heard the term “Technical Debt”. All-knowing Wikipedia describes it as:

Neologistic metaphors referring to the eventual consequences of slapdash software architecture and hasty software development

Sometimes when you need to ship or make a release you have to cut corners. You know it’s not the best thing to do, but it’s a quick thing thing to do, and you need to ship your product.

There is nothing wrong with cutting corners to make a release. In fact, if you don’t have to cut corners, you are shipping too late. Virtually every single software project that shipped had to cut some corners. Sometimes more, sometimes less. No matter what, there is technical debt associated with each project.

Understanding technical debt

It’s important to understand what debt is. Debt is a claim on future labor. It’s an obligation to either do something later or provide resources (money, for instance) that are equivalent to labor.

When you owe money, there are 3 ways to get rid of debt: pay up, default, or print money (if you are a government). Now, most of us can’t print money, so it boils down to either paying the debt or defaulting on it.

The last important thing to understand about debt is interest. Because you get something right away and promise to pay for it later, you usually have to pay for it in a form of interest. This is why, when you borrow money from a bank to pay for your house, you have to pay interest to the bank. You get get the house right away, and the banks makes money.

Technical debt is not an exemption to any of these rules:

1. It’s a claim on future labor. You promise to go back and fix some of the issues that you didn’t have time to do right in the current release.
2. Down the road you have a choice between working on those items or defaulting on your debt by doing nothing. Just like with normal debt, defaulting on a debt comes with consequences. I will describe them in a moment.
3. Fixing something way down the road tends to be more expensive than fixing it right away. Partly because developers move on and don’t remember all the details, partly because there can be new components that expect you to work in current non-optimal way.  This is the interest you pay.

Why no-one repays technical debt

While there are always intentions to fix something down the road, more often than not it doesn’t happen. New issues arise, new features need to be shipped, priorities change, etc. It is a lot more common for businesses to pay technical interest until it becomes unbearable, default on the debt, and then file for bankruptcy in the form of deciding to rewrite the whole damn thing from the scratch.

Let’s get this straight: rewriting your software from the ground up is an expensive thing to do. Yes, you lose your technical debt, but you also lose a lot of your assets. Just like when you file for bankruptcy, you are likely to lose some of your assets. Usually, a very significant portion of them. The same rule applies to software project: you will lose all the good code that you’ve written, tested, debugged over the years. It represents a big investment. While some of it might be salvaged, a lot of it will be gone. It will have to be re-written, re-tested, re-debugged. New deadlines will have to be met and guess what? New technical debt will be accumulated!

We see this cycle over and over again. Companies abandon old projects and announce moving to the “new and better” projects all the time. Why?

There are several major factors contributing to it and they explain why it happens so often:

1. Business people don’t see technical debt. If you think they understand your blabbering about re-factoring a piece of code to work faster you are wrong. They can humor you by pretending they understand but they don’t. If it works – it’s done. They want new features for new customers. New customers bring new revenues and bigger bonuses. System being sluggish and crashing often is not their problem, it’s the fault of developers and hosting people.
2. Perceived low ROI. This is a direct consequence of #1. When talking to business people you can often hear “It doesn’t generate us new revenue so it’s low ROI”. The problem is that ROI stands for “Return On Investment” and we are not talking about investment! We are talking about paying off debt. Debt is not an investment, it’s something you owe. When you buy a house you borrow money from the bank, creating debt. Then you invest borrowed money into the house. If your house goes up in value you made a successful investment, but it doesn’t change the fact that you still owe the bank. Now, think about it as saying that repaying your mortgage has a low ROI since, you know, you already have the house.
3. Developers don’t like repaying technical debts. Well, let me correct myself. They despise it! Writing new code is more fun then fixing old code. Especially if its not written by you. Developers might not like creating technical debt (gah, have to cut corners again!) but fixing old code is not something developers crave to do either. Let’s be honest about it: developers would rather bitch about how the whole thing stinks than fix it. Oh, it’d smell like roses if only they had a chance to rewrite the whole damn thing.
4. Development processes don’t focus on it. Modern development processes gravitate towards features and bugs and tend to ignore technical debt. There are no user stories for products requiring a lot more servers to scale, for having to struggle with strange joins because database schema is designed badly, for developers spending extra hours working around legacy code quirks, or for having a much steeper learning curve for the new people. The list goes on and on. Unless it gets unbearable and produces acute disruptions to the business process (outages, slow beyond reasonable, etc.), modern development methodologies tend to discourage long running maintenance projects.

All these things contribute to not repaying technical debts. Nobody likes it, but that’s just the way it works.

Conclusion

When working on complex software projects, technical debt is a fact of life. It’s created often and willingly and it’s extremely hard to repay. Once enough debt is accumulated the project is deemed ‘old and crappy’ and a ‘new and shiny’ project emerges. While it’s a terrible business practice and comes with a huge cost in time and money, it’s more of a norm than an exception in the modern software engineering.

In the next post I will show one way to repay your technical debt on time and have a stable technical economy for your project.