The Solution to Technical Debt

From http://blog.crisp.se/2013/07/12/henrikkniberg/the-solution-to-technical-debt

By Henrik Kniberg

(related article: Good and Bad Technical Debt – and how TDD helps)
(Translations: Russian)

Are you in a software development team, trying to be agile? Next time the team gets together, ask:

How do we feel about the quality of our code?

Everyone rates it on a scale of 1-5, where 5 means “It’s great, I’m proud of it!” and 1 means “Total crap”. Compare. If you see mostly 4s and 5s, and nothing under 3, then never mind the rest of this article.

If you see great variation, like some 5s and some 1s, then you need to explore this. Are the different ratings for different parts of the code? If so, why is the quality so different? Are the different ratings for the same code? If so, what do the different individuals actually mean by quality?

Most likely, however, you will see a bunch of 2s or worse, and very few 4s and 5s. The term for this isTechnical Debt, although for the purpose of this article I’ll simply call it Crappy Code.

Congratulations, you have just revealed a serious problem! You have even quantified it. And it took you only a minute. Anyone can do this, you don’t have to be Agile Coach or Scrum Master. Go ahead, make it crystal clear – graph the results on a whiteboard, put it up on the wall. Visualizing a problem is a big step towards solving it!

Don’t worry, you aren’t alone, this is a very common problem. <rant>However, it’s also a very stupid, unnecessary problem so I’m baffled by why it is so common.</rant>

Now you need to ask yourselves some tough questions.

Do we want to have it this way?

If not, what do we want the code quality to be? Most developers want a quality level of 4 or 5. Yes, the scale is arbitrary and subjective but it’s still useful.

If opinions vary strongly then you need to have a discussion about what you mean by quality, and where you want to be as a team. You can use Kent Beck’s 4 rules of Simple Code as a reference point. It’s hard to fix the problem if you can’t agree on where you want to be as a team.

What is the cause of this problem?

This is just a rhetorical question, because the answer is clear.

“Things are the way they are because they got that way!” -Jerry Weinberg

Crap gets into the code because programmers put it in! Let me make that crystal clear: Crappy Code is created by programmers. The programmer uses the actual keyboard to punch the actual code into the actual computer. Irregardless of other circumstances, it is the actions of the programmer that determine the quality of the code

The first step in solving technical debt, is to admit and accept this fact.

“but wait, we inherited a bunch of crappy legacy code. We did NOT write it!”

OK, fair enough. The relevant question in that case is: “is code quality improving or getting worse?” Rate that on a 5 point scale (where 1 is “getting worse fast”, 5 is “getting better fast”). Then reapply this article based on that question instead.

Why are we producing crappy code?

The answer will vary. However, I’ve asked it many times and I see some very strong trends.

It’s probably not because you WANT to write crappy code. I’ve never met a developer who likes writing crappy code.

It’s probably not because you don’t know HOW to write clean(ish) code. The skills will vary, but it’s enough that you have a few people in the team with good skills in writing clean code, and a willingness from everyone else to learn. Combine that with a habit of code review or pair programming, and most teams are perfectly capable of writing code that they would rate a 4 or a 5, if they take the time they need.

It may be because of broken window syndrome. Crappy code invites more crappy code, because people tend to adapt their new code to the quality of what’s already there (“when in Rome…”). Once you realize this, you can decide to simply Stop It, and introduce code review or pair programming to police yourselves.

However, the most probable reason for why you are writing crappy code is: Pressure.

I often hear comments like “we don’t have time to write clean code” (<rant>that statement is at worst a lie, at best a lame excuse</rant>). The truth is, you have 24 hours per day just like everyone else, and it’s up to you what you do with it. Face it. You do have time to write clean code, but you decided not to. Now let’s keep examining this concept of Pressure.

Where does pressure come from?

You might want to do a cause-effect analysis of this. Is the product owner pressuring you? Why? Who is pressuring the product owner? Who is pressuring the person who is pressuring the product owner? Draw the chain of pressure.

Then ask yourself. Is this pressure real? Do these people really want us to write crappy code? Do they know the consequence of crappy code (I bet you can list many), and do they really think it is worth it? Probably not. Go ahead, get on your feet and go ask.

Sometimes the cause of the pressure is the programmers themselves. Developing a feature almost always take longer than we think, and we really want to be a Good Programmer and make those stakeholders happy, so the pressure builds up from inside.

NOTE: sometimes there is business sense in writing crappy code.  Sometimes technical debt is Good. We may have a critically important short-term goal that we need to reach at all costs, or we may be building a throw-away prototype to quickly test the market. But that should be the exception, not the norm. If you clean up the mess you made as soon as the short-term goal is reached, or you actually throw away the throw-away prototype, then you won’t end up with a chronic case of Code Crapiness.

Shall we decide to stop this nonsense now?

This it the most important question.

If you are a programmer, the fact that you (as programmer) are responsible for the problem is actually Good News. Because that means you are perfectly capable of Solving the problem. And the solution is simple:

Stop Writing Crappy Code

(Just what the world needs – a new acronym: SWCC™)

“but but but but but…. we don’t have time… the PO bla bla bla, our release date bla bla”

No, spare me the excuses. Just Stop It.

Well OK, you might actually decide to continue writing crappy code. You might decide that this battle is not worth fighting. That’s your decision. If so, at least don’t call yourself an agile development team, and do challenge anyone else who thinks you are agile. One of the fundamental principles of Agile software development is Sustainable Pace. If you are consistently creating Crappy Code, development is going to get slower and slower over time. There is no business sense in this, and it is certainly not agile.

But assuming that you do want to stop, let’s explore what happens.

As a team of programmers, you can take a stand: “We Will Stop Writing Crappy Code”! Write it up on the wall. Shake hands on it. Add “no added technical debt” to your Definition of Done.

Tell the world, and the people who you believe are pressuring you into writing code: “We have been writing crappy code. Sorry about that. We’ll stop now.” Trying saying it loud. Feels good!

Stop Writing Crappy Code

Look at these two curves.

Yes, this is a simplification, but the difference is real. If you Keep Writing Crappy Code, you get slower and slower over time (as you spend more and more of your time wrestling the code). If you Stop Writing Crappy Code, you get a more sustainable pace. But there is velocity cost – you will slow down in the short term.

As team, you decide how much work to pull in – that pull-scheduling principle is fundamental to both Agile and Lean. It’s built into the agile methods. For example, in Scrum Sprint Planning the team chooses how many backlog items to pull into a sprint, same in XP Planning Game. In Kanban, the team has a work-in-progress limit and only pulls in the next item when the current one is Done. Basically, the team has full power and responsibility over quality. Use the power!

In concrete terms: If you are doing Scrum, and you’ve been delivering about 8-10 features per sprint, try reducing that. Only pull in 6 stories next sprint, despite any perceived pressure. Ask yourself at each sprint retrospective. “What is the quality of the code that we produced this sprint (scale 1-5)”. If it is less than 4-5, then pull in fewer stories next sprint. Keep doing that until you find your sustainable pace.

This has business implications of course. The product owner (or whatever you call the person who makes business priorities) will have to prioritize harder. She’s used to seeing 8-10 stories come out of each sprint. Now she will only see 6-7, so she needs to decide which stories NOT to build.

Yes, this will lead to arguments and tough discussions. The real source of pressure (if there was any) will reveal itself. Quality is invisible in the short term, and that needs to be explained. Take the battle! Stand by your decision. If programmers don’t take responsibility for the quality of their code, who will?

Code quality is not Product quality

Code isn’t everything. There’s more people involved in product development than just programmers. There’s business analysts, testers, managers, sysadmins, designers, operations, HR, janitors, and more.

Everyone involved is collectively responsible for the quality of the product being built. That includes not only the code, graphical design, database structure, and static artifacts like that. It includes the whole user experience as well as the business result of the product.

Code quality is a subset of product quality.  You can have great code, but still end up with a product that nobody wants to use because it solves the wrong problem.

What about vice versa – can you have a great product, but crappy code? I hate to admit it but, Yes, technically you can build a great product with crappy code. Somehow teams seem to get away with this sometimes. However, improving and maintaining the product is slow, costly, and painful, because the product is essentially rotten on the inside. It’s a lose-lose proposition and over time the best developers will leave.

What about the Old Crap (a.k.a Legacy Code)?

OK, so you’ve stopped writing crappy code. Congratulations! You’ve stopped accumulating technical debt. You’re still paying interest on your existing debt, but at least the debt has stopped growing.

Next step is to decide – can you live with the existing technical debt, or do you want to do something about it?  If you decide to reduce technical debt, the consequence is that you will slow down even further in the short term, but speed up over the long term. Like this:

Sometimes it’s worth it, sometimes not. The answer is not obvious, it’s a business decision, so make sure you involve the people who are paying for it.

If you decide to reduce your current technical debt, make that a clear decision: “We will Stop Writing Crappy Code, and Gradually Clean Up The Old Code”.

Once you agree on this (and that’s the hard part), there are plenty of techniques for how to do it. Here are two techniques that I’ve seen work particularly well:

1. Add to your Definition of Done: “Technical debt reduced”. That means whenever you build a feature or touch the code, you leave the code in a better shape than you found it.  Maybe rename a method to make it more clear, or extract some duplicate code to a shared method. Small steps. But if the whole team (or even better, all teams) does this consistently, code quality will noticeably improve within a few months.
2. For the larger cleanup areas, create a “tech backlog” & reserve time for it. For example, list the top 10 areas of improvement and commit to fixing one every week or sprint, before building any new features.

Just keep in mind that, however you do it, repaying technical debt means Fewer Features in the short term. Adapt your velocity forecasts and release plans accordingly. Just like any investment, there is a short-term cost. Make sure everyone is clear on that.

You need to Slow Down in order to Speed Up.

Final words

Bottom line: code quality is the responsibility of the people who actually write the code (otherwise known as Programmers).

As programmer, you own the problem, and you own the solution. There’s no need to fret or be ashamed of the past. Instead, stand proud and use your power to do something about it – make a loud decision toStop Writing Crappy Code. That will start a chain of events and Good Stuff will likely follow in the long term. It’s hard and it takes courage, but I don’t know any other way to solve technical debt.

Good luck!

/Henrik

FAQ

Is technical debt always bad?

No. Having some debt can be fine. See my other article Good and Bad Technical Debt (and how TDD helps).

The problem I’m talking about here is chronic, out-of-control, continuously growing debt. I’ve seen so many companies drowning in technical debt, moving painfully slow, losing key developers, and regretting bitterly that they didn’t take code quality seriously from the beginning because it’s so much more expensive to fix it afterwards. Cutting quality gives short term benefits, but the long term is longer than the short term, and most companies do want to survive in the long term.

What is technical debt anyway?

Anything about your code & development environment that slows you down. For example:

• Unclear, unreadable code.
• Lack of test automation, build automation, deployment automation, and anything else that could be automated that you do manually today.
• Duplicate code.
• Tangled architecture & unnecessarily complex dependencies.
• Slow, ineffective tools.
• Uncommitted code & long-lived branches (hides problems that will slow you down later).
• Important technical documentation that is missing or out-of-date.
• Unnecessary technical documentation that is being maintained and kept up-to-date.
• Lack of test environments.
• Long build-test cycle & lack of continuous integration.