Posts Tagged ‘software’

It ain’t Kanban if you don’t use WIP Limits

Wednesday, April 26th, 2017

In my last post I discussed How WIP Limits work to help you get more stuff done.

Some folks like to compare Kanban and Scrum, but this is not really an apples to apples comparison. Scrum is a framework with defined roles, activities and artifacts – It gives you a prescriptive formula for how to run a Scrum Team. Kanban on the other hand is a set of Principles and Practices that you apply to whatever process you have (even Scrum) to improve it. These Principles and Practices get somewhat different wording depending on your source, but here are the ones I tend to reference:

Kanban Principles: Keep these in mind whenever making decisions about your process

  • Start with what you do now
  • Agree to pursue incremental, evolutionary change
  • Initially, respect existing roles, responsibilities & job titles
  • Every team member is a leader

Kanban Practices: Apply these to you processes

  • Visualization
  • Make Policies Explicit
  • Limit WIP (Work in Process)
  • Manage Flow
  • Implement Feedback Loops
  • Improve Continuously and Collaboratively

So while Scrum is rather prescriptive, Kanban is general non-prescriptive. There are many approaches you can take that are consistent with the Principles. And there are many ways you can apply most of the practices…. well except for one, you either limit WIP or you don’t. But that alone should not convince of the veracity of this blog post’s title. To do that let me take you to Kanban’s roots… in the factory.

 

The Kanban process started with the Toyota Production System for manufacturing cars in Japan. The term Kanban in Japanese is as follows

看板

..and it means “card” or “sign”. What we call Kanban is actually the Kanban process – a process that uses these Kanban or cards. Here is what a Kanban card in a physical factory might look like (ref)

It specifies a part to be produced and a quantity which to product.

Let’s see how this process works in the factory. In the below diagram (ref) the Green A is the Kanban card. This is one machine in an assembly line, there is another machine downstream (to the right) consuming the widgets made by the machine below.

The Kanban card limits the total number of widgets. The completed widgets sitting there waiting to be consumed by the next machine plus the widgets in process of being manufactured cannot exceed the quantity on the Kanban card. There can be multiple Kanban cards used in which case it can never exceed the total quantity on all the Kanban cards – this is the Work in Process (WIP) limit. Without the Kanban card (and associated WIP limits) this machine would just run and produce widgets even if the next machine cannot keep up. In that case they would just pile up on the floor in front of the machine building up inventory which takes up resources and delivers no value (value is only delivered at the end of the assembly line after all machines have processed the widgets). The Kanban card optimizes flow through the system, and creates a pull system by which this machine only created widgets when the downstream machine requests (or pulls) them.

So now let’s come back to software. Here is what a Kanban board may look like for a software project. The numbers in brackets represent the WIP (Work in Process) limits

Since we know Kanban (看板) means “card”, I will ask, where on this board are the Kanban cards? Are the sticky notes the Kanban cards?

The sticky notes correspond to the work being done. They are the “parts” in the factory being produced. The columns on our Kanban board are the “machines” in the factory. Remember, the Kanban card limits WIP of any given machine.

àThe WIP limit is the Kanban card ß


If we wanted to, instead of writing “[2]” we could put two markers in our Implement column and state we can never have stickies (work items) in that column without a marker. These markers would be Kanban cards, limiting the work in process (WIP) in the Implement column.

It just easier to write the WIP limit as a number than to create all the Kanban cards.

So, without WIP limits you are not using Kanban cards. And if you are not using Kanban cards then you are not doing Kanban.

You might have lovely continuous flow process. It might have a glorious board, and your team might methodically update it and move their stickies from start to completion. This might work for you and I would not tell you that you need to change it. But It ain’t Kanban if you don’t use WIP Limits.

How WIP Limits work to help you get more stuff done

Wednesday, July 20th, 2016

Two key practices in Kanban are

  • Limit WIP
  • Manage Flow


(image from Lean Kanban, St Louis)

“Limit WIP” means limit the work in process — for any given step in your workflow you limit how many items can be in that state. So a WIP Limit of 2 on your “Develop” step means once 2 items are at that step, the team cannot start to develop any more items until at least one of the current items moves to the next step.


This diagram comes from One day in Kanban land by the always awesome Henrik Kniberg – you should bookmark it and check it out: http://blog.crisp.se/2009/06/26/henrikkniberg/1246053060000

Recently a developer on one of my teams which is using Kanban complained to me that the WIP limit seemed artificial, and that when he was ready to start new work it seemed silly to restrict him from doing so. That got me thinking about how to explain why do we limit WIP and how does limiting WIP help the team to get more stuff done. In this blog I will explain how

  1. Limiting WIP improves flow, and helps us produce more quickly and reliably
  2. Limiting WIP drives collaboration, and gets the team working together
  3. Limiting WIP aids our commitment to completing valuable features

Limiting WIP improves flow

You have a workflow where items come in, are developed, tested, deployed, and other steps towards Done (or “Live” as above). You want your workflow to… well…. flow. That is items come in, do not spend too much time in any given workflow step, and continuously and quickly emerge out the other end into the Done state.

Kanban Aside:

  • Items continuously reaching done is measured by “throughput”, the number of items done per time.
  • Items quickly reaching done is measured by “cycle time”, the amount of time it takes from starting an item coming into the workflow, until it reaches done.

Here is an example of a workflow without WIP limits


The team is busily working on Development (say coding) of many items. Everyone is busy and working hard. But look at the flow. Are items getting done continuously or are they just accumulating under Develop? Are items getting done quickly, or do they linger in the Develop state? It reminds me of something like this


What then if we applied a WIP limit to the Develop state? It might look like this


You can imagine that the WIP limit of 4 on the Develop state “squeezed” all the work downstream to more evenly distribute it along the work states and drive more of it to done. This “squeeze” visualization is OK as a thought exercise, but in reality what has occurred is one of the following:

  1. Developers unable to take on new work into the already full Develop column either collaborated with other developers working on items already being Develop (see the next section)
  2. Or they moved their focus downstream to work on code review or validation. In this latter case this moves those items downstream creating “open slots” in Code Review and Validate.

Number 1 does not necessarily help in this case as there is nowhere for an item to move to once Develop is complete, since Code Review is also full. So developers instead focus on action number 2 and move items downstream towards the Done state. This creates space for items to move on from Develop and therefore creates space for items to move from Backlog to Develop. This is the “pull model” of Kanban in action. So perhaps non-intuitively, it is applying WIP limits that opens up that valve in the bulging water line picture above, focusing work downstream rather than continuously stuffing it into “the bulge”.

Kanban Aside:

  • There are also other actions besides 1 and 2 identified. Another one is called “slack” where developers take on tasks outside of the main workflow to improve things such as efficiency, operations, or addressing technical debt. Developers might even just read a book or take training, this is called “sharpening the saw”. Slack in excess is a sign of a problem, but a little slack is a good thing.

Limiting WIP drives collaboration

As mentioned earlier, then a developer cannot pull in a new item into Develop due to a WIP limit, one way she can continue to be productive is by collaborating with other developers on an item already being Developed. Or perhaps she looks downstream and sees QA needs help, she can pitch in there and help the QA Engineer out (perhaps helping to build out some automation). The power of collaboration is getting multiple talents and multiple perspectives so that we arrive at the best outcome (pair programming also leverages this advantage of collaboration). But another advantage of collaboration is that stuff gets done faster, and getting stuff done faster was one of our goals (and a sign of good flow).

To illustrate how collaboration (driven by lower WIP limits) gets things done faster see the following workflows. One has a WIP limit of 1 and one has a WIP limit of 2. Assume for simplicity that each item takes two developer-days to complete (and that all developer are equivalent). Note that the overall amount of work done (Throughput) is the same — one item per day. But look at an individual item like Item 2. With a WIP Limit of 2 it takes two days of work to complete (this is the Cycle Time). But with developers collaborating under a WIP limit of 1, it only takes 1 day to complete. Completing items faster is preferred because

  • The code doesn’t rot as team members check in new code
  • The developer doesn’t lose the context he or she gained in writing the original code
  • And we get feedback on the item faster, speeding up our ability to inspect and adapt


Kanban Aside:

  • There is mathematical rigor behind the assertion that lower WIP (driving collaboration) gets stuff done faster (reduces cycle time). It is called Little’s Law and can be stated as


Limiting WIP aids our commitment to completing valuable features

In Scrum we plan a sprint, and as a team commit to completing the stories in that sprint, thereby delivering value with every sprint completion. In Kanban we usually do not have sprints, instead using a continuous flow model. WIP limits are how we provide this same level of commitment that Scrum provides.

In Scrum we plan a sprint based on past velocity, and then as a team commit that we will complete the selected stories for that sprint by sprint end. This commitment is an important part of Scrum, however I have observed that it is a very common problem among teams world-wide that they fail to complete all the planned stories for a given sprint. “Punting” unfinished stories from one sprint to the next is supposed to be a practice to be avoided, however it is in actuality common among even the best intentioned scrum teams.

In Kanban, we often have a continuous flow model and not a sprint one, so the Scrum method of committing to a set of sprint stories will not work. We take a different approach to team commitment. By limiting WIP we commit to ensure that the most important things (the things the top of the backlog) get the teams’ attention, and that often multiple members of the team will collaborate to deliver those items.

So in Scrum we might recognize half-way through the sprint that stories will not be finished, and then swarm on a select few stories in order to finish those and meet at least part of our commitment. In Kanban we limit WIP and we ALWAYS swarm to ensure we Stop Starting and Start Finishing.


Why inspect and adapt?

Tuesday, November 10th, 2015

In a recent talk I gave on Scrum I highlighted the power of inspect and adapt cycles.  Or as the the Agile Principle puts it:

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.

First I talked about a non-agile project that did NOT inspect and adapt.   On this project every month (or week, or day) we add some good stuff (valuable features and functionality) and we add some bad stuff (bugs, worthless features, poorly implemented functionality).  Conceptually it looks like this:

image

But when we inspect at the end of every cycle (month, week, day) and adapt based on that inspection then we find what works and double-down on it and find what does not work and eliminate or fix it.  Conceptually then it looks like this:

image

And we would rather release something that looks like the second graph and not the first.  That is the power of inspect and adapt.

Dog bone approach to testing

Friday, May 22nd, 2015

We testers like our testing “shapes”.  For example consider the Testing Pyramid

test-pyramid

Well then, here is my contribution.  This is the dog bone approach to testing.

image

It is shaped like a dog bone because it is fat (more effort and resources) on both ends and skinny (less effort and resources) in the middle.  The three boxes represent:

Developer Testing

Unit Testing, Test-Driven Development, Behavior-Driven Development, Component, Inter-component

BUFT: Big Up-Front Test

End to end functional testing, Non-functional testing (see more here)

TiP: Test in Production

  • Using real (organic) and synthetic transactions and data in production
  • This is about Discovery – what we cannot know before contact with production.
  • It includes customer experience as part of the quality assessment
  • (yeah, I went back to using TiP instead of DDQ…)

Note that whenever we find a problem downstream that we could have found upstream, we roll this information back upstream – for example if we find a bug in production that could have been found with a unit test, we add that unit test and then assess why we missed that unit test in the first place.

In summary we need strong up-front testing to prevent where it is cheap and high value (the developer testing), but as test costs increase due to environment and complexity (BUFT) we only want to do as much testing as needed to gain confidence so we can dip our toes into production.  It is in production that we learn things we could not learn pre-production.

More on the old versus new Microsoft

Thursday, March 19th, 2015

I found these quotes which tie-in well with my previous juxtaposition of Sinofsky versus Nadella

Windows 8 is a disaster. Period.

Paul Thurrott, SuperSite for Windows (pro-Microsoft author and blogger) [ref]

 

Windows 10 Undoes the Disaster of Windows 8 (Mostly)

David Pogue, Tech columnist, Yahoo (formerly with the New York Times) [ref]

Win10 Windows 10 preview.  Return of the start menu for keyboard and mouse users. 

Although I think that Thurott still misses the point, saying

The reason this happened is that while Sinofsky had the maniacal power and force of will of a Steve Jobs, he lacked Jobs’ best gift: An innate understanding of good design. Windows 8 is not well-designed. It’s a mess. But Windows 8 is a bigger problem than that. Windows 8 is a disaster in every sense of the word.

I have made the point before (Those pesky customers) that unless your name is Jobs, you would be best served by using real data to understand your customer and their needs.  That “innate understanding” will help you to avoid delivering a faster horse, however your design still needs to be informed by genuine understanding of your customers.

Why Windows 10 will succeed where Windows 8 failed –Data Driven Quality

Sunday, February 15th, 2015

image

You can’t sort of A/B test your way before the product launches, because you don’t have it in users’ hands yet. You need to use your product intuition to make the right choices. You make these choices and people are paying you to make them.

Steven Sinofsky, former Microsoft President Windows

D11 Conference, May 2013

 

image

It starts with everyone in an organization having the curiosity, having the questions, trying to test out hypotheses, trying to gain insights, and then taking actions. You need to have a data culture inside of your organization.  …this is perhaps the most paramount thing inside of Microsoft. The thing we need to do better on is to be able learn from our customers and the data they exhaust … and continuously improve our products and services, that is job number one by far

Satya Nadella, Microsoft CEO

Executive Keynotes from SQL Server 2014 Launch, April 2014

(I cannot find the quote recorded anywhere, but I personally wrote it down while viewing his keynote for the SQL Server Launch. Some oblique references here, here and here)

Those pesky customers

Sunday, February 8th, 2015

steve_jobs3It is funny that whenever I teach (or lecture, or cajole) software engineering teams about the necessity for customer centricity, I almost inevitably get challenged with the example of Steve Jobs’ Apple.   Jobs did not care about focus groups, Jobs did not task his teams to deeply mine customer data… he just told them what was right and they did it.  Sometimes they got an Apple Lisa out of it, but it was from Jobs’ mind that the iPod, iPhone, and Mac Air sprang forth.

The exception that does not really prove the rule

Well I do not know precisely what to do with Mr. Jobs’ legacy in this context.  I could try to argue how much of his innovation came on the shoulders of others.  The original Mac UI and mouse paradigm having started at Xerox Parc and such… but that is not really relevant.  There is no denying that Jobs’ had the ability to tap into latent need of his customers.

thMAJ42OAXLatent need?  Yup, that is when you give customers what they desperately want, but they do not even know they want it.  For example, Henry Ford is famous for saying

If I had asked my customers what they wanted they would have said a faster horse.

Nobody got rich delivering faster horses… or at least not as rich as Mr. Ford got delivering automobiles to the middle and working classes.

I am the decider

So maybe Jobs and Ford are exceptional.   That does not change the need for understanding your customers (with hard data).   Just recently I was talking to a software entrepreneur who had just launched a new app targeted at fitness trainers and the gyms they work at.   He told me he was dismayed to find all these features that were missing that trainers and gym managers wanted and needed to make the app useful to them.   I started to suggest that this was good news and this limited release gave him precious data about his customers.  His response surprised me.  He said that all these ex-Microsoft guys that work for him suggested a beta release to collect data but he HATED that idea.  He told me this was his vision, and it was that vision he wanted to see realized, resulting in great success for the app.

Hmmmm…

I am thrilled to see ex *Microsoft* guys associated with customer centricity and data-driven quality.  This is a stark change from just a short while ago, and one that I can humbly claim to have been some part of.   And I guess I should not be shocked that every entrepreneur fancies himself the next Jobs with a Jobsian vision, but I also hope that such entrepreneurs can learn that perhaps they are not, and that data about customers, their problems, and what thrilling those customers with the perfect solution looks like, is perhaps the most powerful tool for success.

Culture of data and experimentation

amazon-instant-video-06-535x535In closing consider the case of my current boss, Jeff Bezos.  Bezos certainly had (and has) a vision for Amazon, but he said early on (and later codified) that Amazon will focus what is right for the customer.  Certainly there are many “Jeff projects” that go to production without a focus group and based purely on Jeff’s vision, but there is always metrics that everyone knows up front by which we will assess the idea, use to change it, or scrap it altogether.  I remember in the early days of Amazon Instant Video when we first launched streaming.  Jeff was sure that the free preview should play the first n minutes of the movie, so that is what we implemented.  The data quickly showed that this is not what people wanted, and they wanted the traditional preview/trailer.  Sure we all knew that would be the case… or did we?  Without first trying this idea we truly did not know… perhaps this is what people had been waiting for and it would be Amazon’s advantage?  It wasn’t, and that is fine… this is the culture of experimentation driven by real users and real data that a company needs to embrace to succeed.

Bing’s fault or Bartell Drug’s fault?

Sunday, February 1st, 2015

Not quite Testing in Production…..but amusing nonetheless

 

bartells2

Before it was called “DevOps”, it was just “magic”

Monday, January 26th, 2015

One thing I had forgotten in my time away from Amazon was that their code management, packaging, and deployment systems are magic. Seriously impressive internally built tools to manage a large, but usually apolllodecoupled codebase across thousands of servers in data centers around the world. Magic indeed… sometime black magic, but magic nonetheless.

There are pretty strict limits on what I can say about internal systems, but on the deployment piece at least the cat is out of the bag. Werner Vogels, Amazon CTO talks about Apollo, the Amazon deployment system here.

By comparison, the other massively scalable deployment system I know about is Microsoft’s Autopilo, which you can read about here,.  Autopilot is good for what it is, but Apollo has the advantage in many areas including interface (usability), configurability, packaging, and versioning, which is not surprising given its 10 year head start.

And those systems are just deployment.  My ability to search code, have code dependencies just handled for me (at development, build, and debug times), and to build code in a consistent trackable way is simply impressive at Amazon, reflecting Amazon’s long history in the software services space.

Finally my ability to request new servers at Amazon (generally EC2 virtual servers) is as easy going to a website and filling out a form.  I remember in my early days at Microsoft having to select SKUs and meet with a vendor to place the order. Then a week or so later I got a phone call telling me my servers were on a pallet on the data center loading dock…. what did I want done with them :-)

Amazon has done DevOps long before that word existed.  Like Microsoft, any company making the move into software services learns quickly how important that is.

How to communicate: Tools of the trade

Monday, January 19th, 2015

One of the biggest issues I see on struggling software teams (although this is not limited to software) is problems with communication.  Modern software is complex, and therefore our software teams can be complex.  There are many groups within and external to the team that require information for successful delivery

  • Developers
  • Testers
  • Engineering managers
  • Product managers
  • Business stakeholders
  • Customers
  • Upper level management

Here are my quick thoughts on HOW we choose to communicate.

This list is in ascending order – bias to use the ones at the top over the ones at the bottom

Discussion (meetings, phone, IM)

Best for complex updates and impactful events. Maximizes understanding and reduces conflict via interactive communication.

Have a whiteboard (or virtual equivalent) to ensure understanding around complex technical or business logic

Issue management / ticketing systems

Best for long term item tracking, history, and status snapshot (NOT good for conveying major changes unless accompanied by discussion)

Wiki / OneNote / shared collaboration documents

Best for capturing discussion outcomes and project status. May not need if tracking system offers rich enough experience to accommodate these.

Email

Best for "simple" updates.

Email can point to Issues/Reports or Wiki.

OK to start discussions which require documents or spec review (providing pointers to those) and then complete discussion outside of email

I am obviously biased towards agile. The following Agile Manifesto values are represented here

Individuals and interactions over processes and tools
Customer collaboration over contract negotiation

Note on meetings

Meetings when done wrong go to the BOTTOM of the list. Specifically having too many people present who do not have a clear contribution to the meeting and/or meetings without clear goals. Bias for short meetings with small groups — a quick ad-hoc 2-person face to face (or virtual equivalent) often suffices. Then follow up with one of the other three communication tools to verify all are aligned on what was agreed, and to provide a history.