Technical Debt - Die quickly or a slow painful death

Originally published on LinkedIn here
"Photograph by Rama, Wikimedia Commons, Cc-by-sa-2.0-fr"

"Photograph by Rama, Wikimedia Commons, Cc-by-sa-2.0-fr"

All software products face this choice. Martin Fowler explains it as follows

"You have a piece of functionality that you need to add to your system. You see two ways to do it, one is quick to do but is messy - you are sure that it will make further changes harder in the future. The other results in a cleaner design, but will take longer to put in place. Technical Debt is a wonderful metaphor developed by Ward Cunningham to help us think about this problem. In this metaphor, doing things the quick and dirty way sets us up with a technical debt, which is similar to a financial debt." - Martin Fowler (see here for the full article)

This is a decision that all product development companies live through everyday. This simplistic look at technical debt frames it exclusively as a strategic decision.

  1. Slow Good Design and the product gets released late and dies a premature death.
  2. Quick Bad Design and the product gets released on time, but it becomes costlier and longer to make changes and fix issues as time goes on and your customers curse you, your support staff curse you and finally the cost is so high the product dies.

As a side note Martin offers a mitigation to this scenario with Refactoring

A more nuanced discussion about the technical debt metaphor is by Steve McConnell.

  • "The first kind of technical debt is the kind that is incurred unintentionally. For example, a design approach just turns out to be error-prone or a junior programmer just writes bad code. This technical debt is the non-strategic result of doing a poor job."
  • "The second kind of technical debt is the kind that is incurred intentionally. This commonly occurs when an organization makes a conscious decision to optimize for the present rather than for the future."

Intentional Technical Debt is a strategic choice between Slow Good Design and Quick Bad Design. However, Unintentional Technical Debt is poor execution (Bad Design without the intent of gaining speed) and should be eliminated.

How to eliminate Unintentional Technical Debt

Eliminating bad design and code by junior programmers is a "no-brainer" and you would assume every organizations would have a system in place. I ask around and the only answer I get is "Code Review".

In my opinion Code Review comes with some problems.

  • It ties up expensive "experienced" resources.
  • You lose time because already written code has to be re-written.

My friend Charles and I started down a different path. We came up with a very light-weight design method that you can easily teach junior programmers called "Software Design Sutra".

If junior programmers use this, they will not introduce Unintentional Technical Debt which saves you time and money. Sometime back, I took over a product which was very buggy and not able to respond to customer needs. Hence customers were leaving. I got the team to use this technique. Over the next few months the product stablized and new versions addressing customer needs started getting released.

I recently gave a detailed introduction of the technique to a group of architects. They responded very positively to it. Luckily they recorded it and allowed me to put it up on YouTube.

The presentation is available here if you have trouble reading it in the video

Let me know your thoughts.

Avoiding the Drunkards Walk

Ask any programmer whether he likes to write new code or maintain existing code. You can be sure he will choose - write new code. Ask him the next question, which of these is harder? The answer will be - maintain existing code.

Ask any software company what they want and most will answer Growth. Ask them how they will grow and they will say either "new projects", "new products" or "new features".

This alignment of organization interests and programmers interests results in the more experienced and best programmers working on writing new code. The easier task. The rest will be asked to maintain existing code. This means the least experienced programmers are working on the harder task. 

This results in the quality of code deteriorating fairly rapidly and however good the initial architecture it very quickly becomes a big ball of mud. What are the results of this big ball of mud architecture? Bug fixes and minor modifications become harder and take longer to do and gradually existing customers find their complaints and suggestions are being ignored.

This slippery slope will be familiar to many software organizations. The older the product the harder it is to keep existing customers happy.

Is there any way out of this? The rest of the blog will look at this question in more detail and make some suggestions. 

Let us examine the deterioration of the quality of code. Why exactly does this happen?

First we need to understand the drunkards walk algorithm. When a drunkard leaves a bar if at every intersection he goes in a random direction, including the one he came from, will he ever reach his house? The answer is he will, but its very hard to predict when that will happen. (In the name of political correctness it has been renamed Random Walk.) Here is a visualization of a drunkards walk.

Developers often use a similar strategy when they maintaining code in complex systems with which they are not familiar.

They squint at the code and say "I think this is the problem" and make a change. They hopefully throw it across to the testers. The test team re-opens the issue. Often without rolling back the previous change, the developer will squint and make one more change. At some point the problem goes away.

Then one more defect comes in and another fix is found with this drunkards walk. Slowly but surely the code is becoming fragile.

The problem of deteriorating code is invisible. It feels good to be solving point problems quickly. People appreciate you for fixing the defects. You may actually get a few employee of the month awards for "caring for the customer".

Like this mom who has solved a point problem. She is proud of it.

Till one day she drive over a bump too fast and the inevitable happens, the chair tumbles out. It is a good solution but unfortunately it is fragile.

This is the concept of Jugaad that we Indian's are so proud of. These are low-cost "point" solutions that will not scale and are unsafe.

In software these Jugaad solutions slowly but surely increase technical debt. One day when the architecture reaches the inevitable big ball of mud, developers will throw up their hands and say, until we do a complete re-write we can't add more features or fix some of the bugs.

When I talk about this to developers, they kind of shrug their shoulders "What other way is there." They know no better approach. Before we go further, think about it, is there a better way to maintain code? If you do know another way, are you using it? Do you make sure every programmer in the organization is using it?

So here is my suggestion. You need to follow the Boy Scout Rule. "Always check a module in cleaner than when you checked it out." This sounds like a good thing to do. But the Boy Scout Rule neatly skips over the hard question. How do I do this? How do I make sure it is better?

The answer to this hard question is simple. Whenever you make changes to an existing code base you have to first understand how the relevant portion of the existing system works. Then make the changes in such a way that they extend and improve the design.  (How do you understand an existing system? How do you identify what is relevant? - more blog posts, coming shortly :-) ). Once you feel you have understood the system, the critical step is verifying that you actually have understood the system. To do this you first make a prediction. What will be result of making some change in the system.

Then make the change. If your prediction was perfect, you have understood the system and can make a "clean change". If it is wrong, roll back the change. Spend more time understanding the system, repeat till your predictions are perfect. Then make the required changes keeping the Boy Scout Rule in mind.

So the suggestion to avoid the slippery slope of constantly deteriorating code and customers getting disillusioned with you, teach all your programmer how to avoid the drunkard's walk.

The "Know" techniques in PowerBoost teach you in detail how to avoid the drunkards walk.

Techniques like these are part of the PowerBoost Framework.



People Friendly Processes

Here is a simple puzzle.

There are four cards on a table. The cards have numbers on one side and are either colored red or green on the other. What cards would you turn over to verify the rule – If a card has an even number, then the other side is green.

What was your answer? Did you find it hard? You will find the right answer a little further down. Try the next puzzle before you check the answer.

There are four cards on a table. Each card has the age of the person on one side and what he is drinking on the other. Which cards would you turn over to verify the rule - A person has to be over 18 to be drinking alcohol.

What was your answer?

The right answer for the first puzzle is 8 and Red.  Did you get it right?

The right answer for the second puzzle is 16 and Beer. Did you get it right?

These two puzzles are equivalent but less than 10% of people get the first “abstract” puzzle right. While almost everybody gets the second “people friendly” puzzle right. See the details here.

I first read of this in a book called "How the Mind Works". It had another wonderful example of how converting an abstract statement to a people friendly statement is more likely to lead to the right result. I have put that at the end of this blog post.

This concept had a huge influence on me. I started phrasing problems and processes in a people friendly manner and found people getting the right answer far more frequently. If you teach developers abstract processes, most are unlikely to get it right. On the other hand, if you teach developers, people friendly processes most are likely to get it right.

Let’s look at an example

Imagine your grandmother asks you what you are doing. You want her to be proud of you. Your grandmother is very proud of you when you help other people. She always says "Shabash Beta" if she finds you helping others.  “Shabash Beta” translates roughly to "I am proud of you kid"

Developers are told before coding, their first step should be - “Articulate the business benefits, avoiding technical jargon"

Consider an equivalent people friendly process. Developers are told before coding – “First come up with a statement that will make your grandmother say ‘Shabash Beta’ ”

Making People Friendly Processes is a proven method to help people get things right. The entire PowerBoost program is designed around People friendly processes. This is what makes it unique and effective. The key to a successful product that people actually use is user friendliness. The key to a successful process that people actually follow is people friendly processes.

I will soon write about a few more people friendly processes from the PowerBoost Framework.

If you liked this article why don't you subscribe to the blog. And if you have questions or opinions please do share it in the comments.

Want to find out more about the PowerBoost Framework?


The other example of people oriented reasoning that I found in “How the Mind Works” book goes like this.

Sarita says she is itchy. There is a test for Allergy to Cats, but this test is not always right:

For people that really do have the allergy, the test says "Yes" 80% of the time

For people that do not have the allergy, the test says "Yes" 10% of the time

Question: If 1% of the population have the allergy, and Sarita's test says "Yes", what are the chances that Sarita really has the allergy?

This question was asked to some doctors and many said 70%

What is your answer?

 Instead of working in abstractions let us convert it to people statements.

Out of 1000 people only 10 have the allergy (1%)

If you test the 10 who have the allergy the test will say "Yes" for 8 of them (80%)

If you test 990 people who don’t have the allergy it will say "Yes" for 99 of them (10%)

So the test will say "Yes" to 107 people (8+99) but only 8 of these actually have the allergy. So even though the test said "Yes" the chances of Sarita having the allergy is 8/107 that is roughly 7.5%

You should read “How the Mind Works”. It is a fascinating book. Start describing problems and creating processes that are people friendly and you will see how effective it is.



Micro Rapid Iterations


I have been involved in software development as a hobby / profession for the last 30 years. The most significant change that I have seen is the acceptance of rapid iterations as the key to delivering high quality software quickly. The faster the iterations the better it is. Continuous delivery is the new buzz-word.

I am constantly looking for ways to improve developers capabilities. I found that giving my team members feedback on the work they have done the previous day was very useful in getting things finished faster. (Micro Rapid Iterations).

The developers were definitely seeing the benefits of the daily feedback. The problem was  

  1. Each feedback session went on for a long time.
  2. Doing this everyday with every developer was a huge time commitment from my side.

I wont tell you the long story of the many things I tried, to get daily feedback sessions working in a sustainable manner. I will only tell you what I discovered and how I finally implemented daily feedback.

What I found

  1. Feedback from peers - other developers, testers and product managers was often as useful as feedback from the technical lead / manager.
  2. Feedback from a different person every day was better than showing it to the same person every day.
  3. The feedback session was taking so long because it was becoming a discussion / argument.

Finally I came up with a set of rules that made "feedback sessions everyday for every developer" sustainable.

  1. You have to get feedback every day
  2. You can take feedback from anybody
  3. You cannot go to the same person for feedback without giving two days of cooldown in-between.
  4. You should try to get at least one feedback sessions every week with an expert / more experienced developer
  5. The feedback session will start against a 10 min timer and you have to end the feedback session when the timer goes off. Treat the timer going off like a Bomb going off.
  6. You can only take notes during the feedback. Giving reasons and justifications is forbidden. The person giving feedback MUST abandon the feedback session if this happens.

This has got incorporated into the PowerBoost framework as the XPO ritual. You have product XPO's, Code XPO's, Doc XPO's. There is more to the XPO ritual then just these rules. The importance of observation, psychological tricks to prevent you from rejecting valid feedback, all this gets covered.

Techniques like these are part of the PowerBoost Framework.

If you liked this article why don't you subscribe to the blog. And if you have questions or opinions please do share it in the comments.