You are currently browsing the tag archive for the ‘Lean’ tag.
I presented the attached PowerPoint at the last Lean & Kanban Socal meetup describing our success using Kanban for both our release process and for software engineering. Unfortunately images have been removed since I do not own rights to them, but there’s still some good information there along with a timeline describing our rollout.
We’ve had pretty amazing success with Kanban so far so this is more a success story than anything, but we still had some interesting lessons learned that were worth sharing.
I’ll be presenting a case study on my experience rolling out Kanban at CityGrid Media (both for release management and for software development) on the 14th. If anyone is able to make it please come by and say hello!
More details at http://bit.ly/cb9go0.
My engineering teams recently switched over to Kanban, and we’ve been having great success ever since. One of the things I never fully understood until today, though, was how you tune your WIP limit. My team had been using a WIP limit factor of 1.5, meaning that for every developer there were 1.5 stories in progress.
We felt this was a reasonable approach; that keeping a high WIP limit would ensure that there was always something for everyone to do and people wouldn’t end up idle. We were using three service classes to organize the work – production support, product development, and maintenance class. Things seemed to be going pretty well, and they largely were, but this approach carried significant risk that we ended up paying for today.
I track cycle time as well as a total completed story points for every release, and I noticed that with today’s release we had a significantly higher cycle time and finished about half as many points as we usually do. Shocked, I took a look at our work in progress on the board and noted that we were at our standard WIP limit of eight. What was different this time, though, was that almost every story on the board was large: 13 points or more.
For six developers, we had 66 points in-flight. Digging deeper, our cycle time on a per-story basis had gone through the roof. Because the stories were so large, and there were so many of them, we ended up leaving a lot of work on the floor when the release, which occurs on a regular two week cycle, arrived to pick up whatever we had finished by then.
As students of Lean, I’m sure many of you have seen this type of diagram before. It illustrates that by completing work serially rather than in parallel, you can deliver value to your customers faster. Well, today really brought this point home and in doing so I learned a lot about WIP limits in Kanban.
Part of the benefit of keeping as low as possible WIP limits is that you allow your developers to swarm around as few large in-flight items as possible, ensuring you:
- Get more eyes on a single feature to ensure quality and shared understanding.
- Reduce cycle time on individual stories (maximize developer participation), both reducing the average amount of time spent on stories and ensuring that as many items are deliverable to customers on a given day as possible.
Because we transitioned from Scrum, our product team was used to seeing a large body of work in progress that had been committed to at an earlier date that would end up getting delivered as a release; to them that’s what a team being successful at delivery looks like. In an effort to ease the transition into Kanban, and also because I didn’t see the risk, I went along with higher work in progress limits so that things generally looked similarly to what they saw as efficiency. Now I realize this is a dangerous approach that should be avoided.
Instead of using the classes I mentioned above (production support, product development, and maintenance) we are now organizing the WIP rules around the size of the stories (we’re using fibonacci numbers, and this is roughly what breaks down to large and small for us).
- Large Story – Any story that is 8 points or larger can be worked on with a WIP limit factor of 1/3, rounded down.
- Small Story – Any story that is 5 points or smaller can be worked on with a WIP limit factor of 1/3, rounded up.
Our WIP limit factor is now significantly smaller – instead of 1.5 developers per story, we now have .66. Any excess capacity on a day-to-day basis is soaked up on testing automation, but under this new approach I expect to see cycle time return to normal levels and a much reduced risk of incomplete work at code freeze. Additionally, we’re balancing our work around the way we actually do work – we want more smaller stories in progress since typically a single developer can work on them, but at the same time we want to see developers swarming around the larger, more involved stories.
I learned two things here. First, that Kanban really does either require stories to be a consistent size or, if you do want to use size variants on a single board, that you account for this in your rules. The second thing I learned is that when you keep artificially high WIP limits, even with good intention, you are not likely doing anyone any favors.
Registration has opened for the 2010 Lean & Kanban conference in Belgium this summer, and I’m lucky enough to be attending this one. The lineup looks great, with David Anderson, Mary Poppendieck, Alan Shalloway, and many other great Lean thinkers giving presentations over a two day course on September 23rd and 24th. Check out the program, and definitely let me know if you’ll be there so we can arrange a meetup!
The first Lean Software & Systems Meetup (SoCal) will be on July 12th at the CityGrid Media offices where I work. There should be some great discussion on Kanban and other engineering/technical process. Hope to see you there!
First of all, I wanted to announce that the scope of this blog might be growing a bit; I was recently promoted from the Release Manager at Citysearch to Engineering Manager for several teams working on our advertiser platform. A lot of the reason I was promoted was because of my experience in process development, so there will still be a lot of correlation between the original content and where we go in the future, but I’ll definitely be growing and talking about new subjects with relationship to the broader organization.
In that light, I wanted to talk about my experience moving my software teams from Scrum to an iteration-less form of Kanban. My CTO, Christophe Louvion, provided a nice high level summary at http://runningagile.com/2010/05/31/a-few-weeks-of-kanba/ based on feedback in a presentation one of my teams gave, and I’m going to expand on the list:
Stand ups differ – going story by story, not based on individuals
The story-based focus, where we start at the right of the board and focus on how to move stories into the “complete” column, has encouraged the team to be more collaborative in analyzing, executing, and testing their story implementations. Previously, the teams were going down the list on the first day of the sprint and assigning items out one at a time in an effort to essentially maximize their work in progress. Unfortunately, this was pathological in that it ensured that only one person knew the implementation for that story (and that you were done if that person ended up getting sick) and that it discouraged the team from swarming around one or two items. Now, everyone on the team is aware of what the entire team is working on and can share the load as needed.
Less Rules: No roles prescribed, More Dynamic
This speaks to the core Kanban idea that the process itself is something that is being engineered and tailored to the team; the team feels that they don’t need to have roles (such as the Scrum Master) that the team has in many ways grown out of the need to have. If they are mature enough to self-organize, they are more than welcome to do so – and they are.
Backlog more Flexible
If you’ve ever worked in advertising you know there are a lot of stakeholders out there and when deals get done they need to be implemented irrespective of any two or four week sprint cycle. The flexibility of the “to-do” list in Kanban allows our product team to reprioritize any work that the team has not already started development on, which in turn gives the product team the ability to ensure we’re always working on the latest requirements from the business. Everyone knows they are working on the latest and most important features and that means real value when we deliver.
Pros: Better Team Focus, Less Meetings More Time to Deliver, More Flexible Work Process
Again, the second two points here speak to the ability of Kanban teams to tailor the process to themselves. The first point, however, is really worth digging in to. In my opinion, the increased team focus comes from the participation of the team in engineering their work process. Prescriptive methodologies such as Scrum, although good general rules for many teams, do not necessarily require the team to understand the why behind many of the reasons that the process is the way it is. In fact, they are designed so that even teams who do not understand the whys can be effective. In contrast, because Kanban drives change from within the team, the team is encouraged to experiment with the process (even making mistakes, as long as there are lessons learned), which results in a team that understands at a very deep level what works for them, and why. On my team Kaizen events have jumped significantly now that the team understands the process and knows that they are empowered to make changes in a safe environment.
Cons: Less Structure Could Cause issues with Some People
This is one item that I’m still learning a lot about. Kanban does seem like, much more than Scrum, it requires leaders at all levels in the organization to drive the positive process change. I’ve been on a couple different teams that were learning Scrum as they went – I’m not sure that Kanban is capable of working for teams at that level. It simply provides too much flexibility to the team and much of the guidance asks you to analyze your specific situation rather than reading out of a playbook. That said, I haven’t actually tried Kanban yet with a less mature team so my data is incomplete here.
How Its Working Out: Feels Good, Increased Our Collaboration, Communication Has Increased, Forces Follow Up to Eliminate Blockers
“Feels good” is another important component. If you saw the last video post I made, you know that two of the major elements that motivate people are autonomy and mastery. Kanban provides these in that it trusts the team to engineer their process in a way that works for them – the methodology is not from “on high” and passed down to them to execute; instead they are an integral part of the process. We hire smart people who are good at engineering software systems, it turns out those kind of people have an intuitive sense for engineering process as well. The Kanban board and metrics provide a visual way for the team to see that things are getting better and to know that they are leading themselves to an increased level of mastery.
All in all, moving to Kanban has so far been great for the team. We’ve only been experimenting with the new process for a few weeks now, but already it seems like things are on an exciting new trajectory. I’ll be updating this blog with more details on how things are going as we continue to learn and grow.
Ash Maurya posted a very in-depth case study over at Eric Ries’ blog a few days ago, and I highly recommend reading it. Interestingly, this case study is from a company that has existed for seven years (not a startup) so the lessons here apply to a broader audience.
Continuous Deployment is Continuous Flow applied to software. The goal of both is to eliminate waste. The biggest waste in manufacturing is created from having to transport products from one place to another. The biggest waste in software is created from waiting for software as it moves from one state to another: Waiting to code, waiting to test, waiting to deploy. Reducing or eliminating these waits leads to faster iterations which is the key to success.
You can find more case studies at the Lean Startup Wiki over at http://leanstartup.pbworks.com/Case-Studies.
I came across a great article over at the ACM blog regarding how Agile Facebook is and the kind of value they are seeing from using these kinds of processes even though they are well out of startup mode. A lot of people think that Lean-derived techniques such as continuous deployment, Scrum, and Agile are really only for small, startup-type companies, but just as Lean manufacturing isn’t just for small workshops, Lean software development can deliver astounding results to any company – especially large ones that have become complacent and allowed a lot of non-value producing process to creep into their methodology.
Excerpt:
Perhaps the most interesting and revealing aspect of Robert’s talk was the discussion of Facebook’s somewhat unique development process. At the surface it appears to have the contradictory goals of: minimizing down time, scaling fast, and extremely quick code updates. First, Facebook developers are encouraged to push code often and quickly. Pushes are never delayed and applied directly to parts of the infrastructure. The idea is to quickly find issues and their impacts on the rest of system and surely fixing any bugs that would result from these frequent small changes.
Second, there is limited QA (quality assurance) teams at Facebook but lots of peer review of code. Since the Facebook engineering team is relatively small, all team members are in frequent communications. The team uses various staging and deployment tools as well as strategies such as A/B testing, and gradual targeted geographic launches. This has resulted in a site that has experienced, according to Robert, less than 3 hours of down time in the past three years.
In Eric Ries’ latest post over at Startup Lessons Learned, he addresses many of the most common objections he’s heard regarding continuous deployment. According to Eric, most of the objections boil down to the following two points:
1. That mission critical customers won’t accept new releases on a continuous basis.
2. That continuous deployment leads to lower quality software than software built in large batches.
I think he addresses both of these points extremely well and goes further in illustrating the improvements that continuous deployment brings to an organization:
- Faster (and better) feedback.
- More automation.
- Monitoring of real-world metrics.
- Better handling of intermittent bugs.
- Smaller batches
Be sure not to miss out on the great discussions in the comments thread as well. One comment Eric made that really stood out to me was his mention that the dichotomy between “fewer, more stable releases” and “faster, less stable releases” is a false one and that what continuous deployment truly brings to an organization is faster and more stable releases.
This reminded me of a lot of objections I’ve heard regarding Lean in that doing the right thing up front (building infrastructure, identifying and planning around your constraints, etc.) will give you better quality but slower development; in fact the truth (as borne out both in manufacturing and software development) that doing the right thing up front may cost more initially but you’ll end up letting people work faster while also increasing quality and customer value over the life of the project.
Craig Tadlock (over at the Tadlock Enterprises Blog) made a couple good posts recently about Team Foundation Server and Team Build 2010:
Work Item Classifications
Work Item Relationships
He also came across this great blog that is focused on doing automated testing:
http://blogs.msdn.com/vstsqualitytools
as well as this link discussing custom workflow items:
I’m setting up a local TFS test environment for myself now and hopefully soon should have some interesting information on setting up Lean build systems using this great platform. I was lucky enough to work at a company that used the previous incarnation of Team Foundation Server and I have to admit I’m pretty excited to see what goodies Microsoft is giving us on this go-around.
Lean Builds 