Kanban - The Next Step in the Agile Evolution?

November 6, 2009 | Author: PM Hut | Filed under: Agile Project Management, SCRUM

Kanban - The Next Step in the Agile Evolution?
By Ketil Jensen

The software industry has embraced agile methods. A rising number of teams is now trying to deliver value incrementally in small iterations. This is of course a good thing. Customers get new functionality more frequently out into the market, an increasingly important factor in a competitive global economy.

The innovators and early adopters of agile methodologies were able to deliver great results using agile methodologies like Scrum and XP. But these early adopters were software craftsmen, proud developers of well-crafted software with lots of enthusiasm and good communication skills in addition. These early agile teams would probably succeed using less agile methods than Scrum and XP. Now the agile train has really started rolling, the majority has “seen the light” and even the laggards are starting to think that this might be worth looking into. Should we be happy? Or is it time to get just a little bit worried?

The challenge the software industry is facing now is that instead of getting low quality software into production once every year, we get low quality software delivered 4-12 times a year (depending on iteration lengths and a couple of other factors). But wait, this is a good thing isn’t it? We can clean up the software, stop projects that don’t deliver value and so on. In theory yes, and I guess some organizations are able to do this. Many organizations however, do not have the required discipline.

Teams struggle to deliver software on time with expected quality

The following is not an unlikely scenario in a typical Scrum based project:

The team starts the sprint with sprint planning where they select the top prioritized stories from the product backlog. This is followed by detailed planning and estimation of tasks. The task board is set up with stories and tasks and everybody is happy. The team starts on the new sprint but after a few days it becomes clear that the team has under-estimated the complexity of the stories. New tasks are identified and added to the scope. From here, the team has quite a few choices, two of them are:

  1. They can start to scope out stories, or even parts of a story (typical if the team has not been able to split up the stories)
  2. Compromise on quality and allow bugs into production.

I’ve seen both, and in some cases both combined. The text book states that one should never compromise on quality. However, when the organization is measured on the amount of new functionality delivered, it is easy to understand that new functionality is prioritized over bugs and technical debt. Software degrades and changing it will become causing even more bugs and technical debt. This is a negative feedback loop and breaking it can be extremely difficult.

Improving Reliability with Kanban

Kanban can help teams build quality software that the customer wants within a predictable time frame.

Queue and WIP limits

Limits are an extremely important element in Kanban. There are two types of limits: Queue limits and WIP limits.

Basically, queue limits will prevent you from doing too much up front planning.

The ideal work planning process should always provide the development team with best thing to work on next, no more and no less.

WIP limits on the other hand are there to limit the amount of work in progress at any given time. The morale is (quoting Corey Ladas):

“Don’t write more specs than you can code, don’t code more than you can test and don’t test more than you can deploy.”

The Kanban board below shows the value of limits. The “pri” column holds the 3 top prioritized “tasks” (bug, feature, etc..) This is enough to keep the team busy and make it easy for team members to pick new tasks. New items are prioritized into the queue before the queue is empty, usually when there is less than 2 items left. These tasks are pulled in from the “Stories in progress” station.

Kanban Board

Kanban Board

The development station can hold a maximum of 5 Kanban cards. Setting this limit can be a challenge; one that worked for us was to allow every developer to hold 2 cards (to compensate for blocking), but then extract one Kanban card to avoid the anti-pattern of all developers holding 2 cards at all times.

System test is the next station in our process and a very important one. It can hold 3 Kanban cards. Having this as an explicit part of our process has several positive effects:

  • Every feature and bug is properly tested in a full system test environment.
  • We keep the system under relatively good control since we don’t overload the system with too many new changes before we have verified those already in the process pipeline.

Here lies the beauty and the power of pull systems and limiting work in progress. If a bottleneck occurs downstream in system test, upstream station development will eventually be blocked (see figure below). This prevents us from overloading the system with new features or bug fixes before we have verified the current wip in system test. This means that team members need to help out in resolving the bottleneck. It might not be necessary, or even possible, for all team members to help out. Instead, these resources can spend time improving the overall quality of the software being built (more tests, refactoring etc…).

Kanban Board - Bottleneck Identified

Kanban Board - Bottleneck Identified

It’s quite interesting that since the system will force us to go slower for a short period of time, this gives us the potential to go faster in the future. Limiting wip may be hard for some teams in the beginning.

It can be especially hard to see why you should not take on more development work when the wip limit is reached. After all, we’ve been taught that productivity is proportional with the amount of work we deliver. It takes a little while to get used to the fact that in order to go fast, it is sometimes necessary to wait.

The importance of defining testable features

An interesting learning point we made not long after starting our Kanban system was that if we wanted systemtest to be part of the process, then all our feature tasks had to be truly testable. Before, our tasks had been very focused around technology and technical details. Now we are concentrating on defining tasks that actually provide value to the customer. This also helps us focus on delivering what the customer actually wants. If the team is practicing techniques such as ATDD and/or BDD and the alike, we are improving the odds even further.

Handling rework

No matter how great your process or your team is: You will find bugs. So handling them in an efficient manner is important for building a quality product. Basically, bugs must treated as first class tasks just like features. If not, bugs will accumulate and the software will degrade at a much greater pace than necessary. So handling bugs sooner instead of later is almost always preferable. There are many ways to treat bugs in a Kanban system, a detailed overview is beyond the scope of this post but a good subject for a later post.

Lead time

Skeptics claim that since there are no iterations or estimation in Kanban, you don’t know when you are finished. We have already seen how WIP helps you get your features through development and test and potentially shippable. Another important concept in Kanban is lead time. This is the time it takes for a feature to get through the process once it gets in. After a while you can start making predictions like: “We can deliver user story X within 7 days”. In my opinion this is far more predictable than saying: “We think we can deliver stories X, Y, Z within 4 weeks”.

Start with something small and experiment

Kanban is a lightweight process, unlike Scrum it does not prescribe much. This makes it pretty easy to get started. Start with applying WIP to your task board. Keep it simple and observe and reflect. When the system has stabilized, do another change and observe the result. But remember to keep the changes small. Rocking the boat too much can make it roll over.

Conclusion

It is my opinion that timeboxed iterations are causing more harm than good. What we see now as Agile has become mainstream, is that teams struggle to deliver within these timeboxed iterations. The consequence is less value for the customer and poor quality. I think Kanban can improve on this situations, if we can:

  • limit work in progress,
  • focus on delivering testable features,
  • and include testing (automatic test, system test, exploratory testing etc….) as part of the process pipeline,

then we are in a better position to deliver high quality software aligned with what the customer wants within a predictable time frame.

Ketil Jensen is currently a senior consultant at Miles AS, a Norwegian based consultant company. He has a strong interest in Agile Software Development, Test Driven Development, and Domain Driven Design and testing. Ketil maintains an interesting blog on Agile: Walk the Walk.

Share this article:
  • Digg
  • del.icio.us
  • Facebook
  • Google
  • blogmarks
  • LinkedIn
  • Reddit
  • StumbleUpon
  • TwitThis
  • Yahoo! Buzz

Related Articles

14 people have left comments

I had not heard of this application of Lean / Kanban / TPS manufacturing techniques to SW development before. I like it!

I can see how it gates the flow of work through the development phases (stations) and puts priority on the completeness of the work product vs how quickly it gets done. Like lean manufacturing systems, the overall speed (ie end-end lead time) can be increased later by appropriate throughput balancing of each station.

I’m going to keep my eye open for more articles on this technique and monitor its adoption.

JohnD wrote on November 6, 2009 - 5:44 pm | Visit Link

Hi JohnD

You can find out more about Kanban Systems for Software Engineering at the Limited WIP Society

Karl

Karl Scotland wrote on November 7, 2009 - 5:43 am | Visit Link

I had heard of Kanban in the early days of ‘97 as part of the JIT management but this aspect in software seems new and to that extent a little more time consuming to me.
remember, agile was always about delivering quality code in small manageable chunks but that in itself has become an issue to deal with since everyone is taking the principals of agile methodology and twisting them to their own convenience, calling for higher number of working hours and more expectations from the teams..
which in fact is NOT what should have happened.

now (Just In Time) Kanban approach in software development/maintenance would mean the client who asks for the services needs to be more aware of the limitations of what (s)he wants and when.

In the real world, the development team following agile methodology is never given the authority to decide what they can deliver and what they can’t. it is the client who still calls the shots…

so implementation of Kanban to that extent would have to be again considered with the client approach in mind…

but let’s hope this is a better way.. only implementation will tell

Shrikant wrote on November 9, 2009 - 7:56 am | Visit Link

disclaimer: I have no experience with Agile or Scrum beyond a cursory reading of the techniques

Maybe I’ve got this wrong, but it sounds basically like you’re removing the ‘management’ part of Project Management and simply reacting to emergencies.

Your Kanban appears to be a less a new method and more a reaction to poor planning/estimating techniques -

“The team starts on the new sprint but after a few days it becomes clear that the team has under-estimated the complexity of the stories.”

“After a while you can start making predictions like: “We can deliver user story X within 7 days”. In my opinion this is far more predictable than saying: “We think we can deliver stories X, Y, Z within 4 weeks”.

Yes, it’s more predicable. It’s even more predicable to say “here’s what we’ll have done tomorrow and what we won’t.”

That’s not management, that’s status update.

Trevor K. Nelson wrote on November 10, 2009 - 2:34 pm | Visit Link

Trevor,
You raise a common concern with Kanban.

My response is, would you rather a team try to improve their software estimation skills/techniques, or would you rather they try to improve their software building skills/techniques?

Kanban frees teams to concentrate on the latter.

Also, generally, the less project management work needed is a positive sign for the team and project. If you find yourself immersed in PM work, then your already in trouble, you just may not realize it yet.

Troy Tuttle wrote on November 10, 2009 - 5:30 pm | Visit Link

Trevor,

The problem we are trying to solve is in my opinion:

How can we deliver high quality software aligned with what the customer wants within a predictable time frame.

These days people more or less agree on that splitting the delivery of a product or projects into smaller deliverables is preferable over delivering the entire product at the end. In order to accomplish this there will of course be a need for planning. This is something that I don’t get into details about in the article, but you always need to plan your work. People claiming that agile equals no planning has got it all wrong. In Kanban there is in fact continuous planning. When the ready queue (holding the work that has been planned) is starting to get empty, the team (together with stakeholders of course) need to do a planning session to carve out new features/bugs to fix etc… which will then end up in the ready queue. How much to plan in each session is something that needs to be experienced with.

The clue is, stated so elegantly by Corey Ladas:

“Don’t write more specs than you can code, don’t code more than you can test and don’t test more than you can deploy”

The world changes and if we do to much up-front planning we run the risk of basing our plans on the wrong assumptions. Experience shows that delaying decisions to the last responsible moment is almost always feasible. The reason is of course that you increase your knowledge as you go. This does not mean that you can’t make predictions about the cost of a project at a high level, you should. I also think that t-shirt sizing features (that are testable valuable units) can have some value. But I consider doing detailed estimations of each task in a project to be waste. I think Troy describes it nicely:

“Would you rather a team try to improve their software estimation skills/techniques, or would you rather they try to improve their software building skills/techniques?

I know what I will choose.

Ketil Jensen wrote on November 11, 2009 - 6:25 am | Visit Link

Ketil & Tory,
I don’t doubt that Agile and Kanban include (some) planning. What I question is the accuracy of it.

You’re asking a question that avoids the issue (from a Project Management standpoint) -

Would I rather have a team improve their estimation techniques or their software-building techniques?
I want both. The two are not mutually exclusive. I want a team that knows what they’re doing and knows how long it will take.

“Don’t write more specs than you can code, don’t code more than you can test and don’t test more than you can deploy”

That’s good advice. It’s also what I would call estimating. The problem is that, in your example, you don’t know what that amount is until AFTER you’ve started.

Trevor K. Nelson wrote on November 11, 2009 - 1:24 pm | Visit Link

If we take your situation above of a bottleneck in the System Test environment, you say:

“It might not be necessary, or even possible, for all team members to help out. Instead, these resources can spend time improving the overall quality of the software being built (more tests, refactoring etc…).”

What if members of the team are non-developers, i.e. testers, UX, UI people. If they don’t have the skill sets necessary to help remove the bottleneck and we continue to enforce the queue and WIP limits do they run the risk running out of work and/or being under utilized?

Phil Murphy wrote on November 15, 2009 - 10:12 am | Visit Link

Phil,

You say:

“What if members of the team are non-developers, i.e. testers, UX, UI people.”

I assume therefore that the block occurs in development and not in test? If you don’t have any resources that can help out to solve the block, then some of your resources will be idle. In such a situation these idle resources can pair up with developers. This may be fruitful both for developers and non-developers as they can learn from each other.

When the bottleneck is solved the team should try to find the root cause of the problem and take appropriate action based on the findings. If its a recurring problem, then one possible solution can be to reduce the WIP limit for development station. This will decrease the lead time and help the team push features faster through the system. Setting the correct WIP and queue limits is something that a team should experiment with If you have 3 developers on your team an approriate limit could be 3-5. Start with something simple and adjust as you go, but don’t change values too often. When a system change, it needs some time to stabilize.

Hope this answers your question.

Ketil Jensen wrote on November 16, 2009 - 3:27 pm | Visit Link

[...] teams deliver too much code too fast, leading to low quality code with high defect rates (see Kanban – the next step in the agile evolution?). Finding defects late is extremely costly…. Furthermore, delivering low quality software will [...]

From Scrum to Kanban - PM Hut wrote on February 24, 2010 - 3:13 pm | Visit Link

Wow, why is it always Scrum vs. Kanban?

We use both methodologies, each where they shine most:
Scrum for development of new software; Kanban for recurring activities (e.g. maintanance).

Neither of these methodologies is superior in general. It all depends on the problem you want to solve.

For details see here: http://www.infoq.com/minibooks/kanban-scrum-minibook

Urs Enzler wrote on March 8, 2010 - 8:04 am | Visit Link

I see this is discussion thread on Kanban and Agile as 2 methodologies that claim to produce high quality software and in a reasonable time frame.

Neither methodology it would seem has all the answers to this aspect of software development, with Kanban trying to correct for where Agile would seem to have failed.

I’d just like to add the point though, that in bringing on new methodologies, we lose sight of the fact that it is not the methodology , or the client , or even the technology that causes low quality software or project overruns, it is P O L I T I C S.

With this in mind, consider the case where a PM or scrum master choses to laydown the law and prescribe a 1 week sprint to cover say 7 tasks out of 3 stories. Whether or not the tasks were fully scoped, The fact of the matter is , that POLITICS rears up and forces the PM to cut quotes by 25% for no methodical reason beyond pressure from above. When pressed on this decision at a post mortem, the PM may say that Agile demands short iterations, yet the PM may have failed to acknowledge that Agile also demands consultation with the system developers , not just ‘make my date’ or else.

Kanban or any other new methodology cannot account for or combat these failures to adhere to development guidelines. Its the Politics that makes the methodology look faulty.

Rodney wrote on April 12, 2010 - 11:48 pm | Visit Link

Kanban is a valid method for project management used for years, though mostly with whiteboards and yellow stickers. Lately software development with agile started to also use it extensively… The number of Kanban plugins out there shows it.

We are a web development company and for years had clients asking for solutions around Kanban board, but for totally different businesses. They just didn’t know it’s called Kanban…

So we developed smartQ - (http://www.getsmartQ.com).It’s basically Kanban for the masses - makes it easy to adjust it to any process and business type.

With distributed teams the whiteboards and yellow stickers will move online, so tools like that should help more teams get to know Kanban or Kanban-type methods.

R. Jared wrote on November 2, 2010 - 3:26 pm | Visit Link

I have been using Scrum for years with great success. I’m new to Kanban. I’m looking forward to trying it out. The thing I have been working on is how Scrum aligns with the Federal Aquisition Regulations. We have developed a Scrum based process that aligns with the JCIDS process. Although it is not how a commercial software development company would choose to implement Scrum, Federal government programs have to comply with a myriad of regulations, we have figured out how to do that using Scrum. Drop me a note if your interested in learning more.

Roy Maines wrote on January 15, 2012 - 11:34 am | Visit Link

feel free to leave a comment

Comment Guidelines: Basic XHTML is allowed (a href, strong, em, code). All line breaks and paragraphs are automatically generated. Off-topic or inappropriate comments will be edited or deleted. Email addresses will never be published. Keep it PG-13 people!

XHTML: You can use these tags: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

All fields marked with " * " are required.

Project Management Categories