Kanban Is For Attacking Flow Problems, Not For Dropping Iterations

Posted on January 10, 2012 by azheglov

I’d like to chime in on the discussion started by Abby Fichtner, a.k.a. HackerChick in Kanban is the New Scrum and continued by many people in comments and blog posts (such as this one).

I don’t find anything to object to in the article, but I feel that a stronger argument can be made for Kanban than the supposed freedom from iterations.

Time-boxes Aren’t Scrum’s Impediments

The freedom-from-iterations argument sounds like treating a symptom. The iteration length should not be an obstacle to more frequent releases if those are important to users or customers. Demos at the end of a time-box (say, on Business Day 10 of a two-week sprint) are not the right time (too late) for a product owner to become aware of a completed feature and they are not for sort of acceptance tests to decide what’s shippable and what’s not. Suppose a feature is ready by Day 6, the product owner is aware and approves it. It’s the right feature, built right and it’s shippable. If it’s really important to ship it to the users the same day, the ops people can do it. They are part of the team, sit nearby and not in a different building, right? When Scrum is without “buts” (“we do Scrumbut, our PO is never around”), time-boxing shouldn’t hold it back. Scrum has a built-in retrospective process to get rid of “buts” holding teams back.

“We are constrained by Scrum’s time-boxed iterations, so we need to try the iteration-less Kanban process” is a weak argument. Kanban is a meta-method for improving other processes and its main strength is that it can bring many teams closer to their problems and understanding the science of their operations.

Flow Problems

This is a class of problems faced by many software organizations. Take a feature that your paying customer wants, don’t take your eyes off it, and follow it from concept to cash (or from hypothesis to happiness, if you will). Now consider all your features. That’s your flow. That’s how software work behaves. (Some people don’t see it that way. They prefer to look at the org chart. But that doesn’t change the reality.)

Today’s reality for many in the software industry is that our work doesn’t flow very well. There are pools, back currents and even dams (often built by us because of our lack of understanding of flow). What we need to do is to understand how work flows in our context, study the flow and make improvements. This is exactly where the Kanban method comes in with its “improve collaboratively using models and the scientific method.”

What models and methods, you may ask. A short, incomplete answer is: Goldratt (qualitative), Reinertsen (quantitative) and Mary and Tom Poppendiecks’ lean principles as super-useful approximations. Consider the Theory of Constraints. When Goldratt’s Jonah asks Alex about quality control in his plant, he thinks about exploiting a bottleneck by inspecting parts before the NCX-10 gets them. That’s “building quality in.” Specification by example is an attempt to apply similar logic to software development. Check out Donald Reinertsen’s Principles of Product Development Flow. You can find many things in this book, among them charts quantifying what we approximate by using the principle of “deferring commitment.”

Distance Between Teams and Their Flow Problems

Consider one team. They didn’t do XP 10 years ago. They’re mainstream, maybe late adopters. You may have met a team like this one recently. They seem to be doing this:

If you were an XP pioneer or an early Agile adopter, there is something important in this picture that is not actually shown in it: the customer, standing pretty close to it on both sides. Your XP or Scrum framework may not say much about the economics of your product development flow, but the way you work and retrospect are aligned with the desired economic results reasonably well. This is not the case for the mainstream or late-adopting team in our example. But let’s zoom out and see what they’re actually doing:

This is a fairly typical case: a two-week iteration cycle, a quarterly release cadence, and the total cycle time through the whole value stream of more than one year. What is needed here is a dose of lean thinking. Kanban can deliver it and help this team get underway on their lean journey, understanding and studying their flow and finding solutions to their problems. And they may even decide to keep their iterations, or as we call them in Kanban, input, release, and retrospective cadences.

I totally agree that we shouldn’t throw the Scrum baby out with the bathwater. Dropping iterations is not a good reason to do so.

Posted in hands-on | 7 Comments

Notes from Agile Coach Camp: Explain This Picture

Posted on November 21, 2011 by azheglov

This one continues the series of delayed posts about the sessions that I led or contributed to at the Agile Coach Camp Montreal that took place several months ago. (Earlier in this series: the Marshall Model and the Lean Startup). I’m writing this down to rethink and remember what we talked about.

The picture to be explained was the one on the cover of David J. Anderson’s Kanban book (to the right). The session was an invitation to all Kanban enthusiasts to compare their notes from reading this book and their experiences applying Kanban. People unfamiliar with Kanban had a chance to explore it.

The first set of questions we got out of the way was: who is stuck, who is too busy, who is idle, why, and what is their role on the team. The book had been in print for more than a year, so everyone knew the answers. If you want to know, read the book and/or talk to someone in your local agile community, they’ll explain.

How to Kill Kaizen

More interesting were the questions about trying to improve the situation. One option we didn’t consider was to simply increase work-in-progress limits. That way we can give someone who is busy with three tasks a fourth task. And someone who has started more tasks that can fit on the board can start even more. This option means doing things the same way that got the team in this difficult situation and avoiding the conversation about improving the process. It ensures that the team will not benefit from kaizen, the culture of continuous improvement.

Limit WIP Upstream

The first option suggested by the group was to reduce WIP upstream. Indeed, if we reduced it from 20 to 10, this would underutilize the analyst, but would shorten his feedback cycle from 23 cards to 13, probably improving quality of whatever he is doing. The team’s throughput may about the same, but the quality will likely go up. Interesting that such result may be achieved by working less.

Hire More Developers

This option is often considered and taken by managers of real-world teams. It obviously elevates the bottleneck, doubling its capacity. But it does nothing to address the flow problems affecting the team. The prolific analyst can still overwhelm the developers and, as the total WIP goes up, the feedback cycle will lengthen, putting quality at risk.

Five Focusing Steps

The Five Focusing Steps method, introduced by Eli Goldratt in The Goal states that before elevating the bottleneck, we should exploit it. The exploitation action is often away from the bottleneck, so that makes it very clear that hiring another developer right away is not the solution.

One source of inspiration for such exploitation solutions is the Goal episode when Jonah (the wise consultant) visits Alex’s plant. Shortly before leaving he asks: “One more thing. Show me where you do quality inspection on bottleneck parts.” Interesting that Jonah asks where, not how. All he cares about is if the quality control is ahead of or behind the bottleneck.

What follows from here is not about the team on the book cover – we don’t know enough about them to say this will solve their problem – but it’s a real solution for real-world teams as long as it fits their context. (Everybody who attended nodded their heads here.)

Example

Let’s move the idle tester over to collaborate with the analyst and, assuming the tester has some automation skills, get specification by example going. Automated acceptance tests make the developer more effective and reduce the amount of defects that escape into the testing phases. The tester presumably will still have some time left for exploratory testing.

Together, the attendees redrew the book cover picture on a flipchart to look like this:

We’re not stuck, I’ve got slack, I’m kinda busy.

Posted in conferences | 1 Comment

A Goldratt Reader’s View of “No Child Left Behind”

Posted on November 19, 2011 by azheglov

I read a book earlier this year that I should have read much earlier: Eliyahu Goldratt’s The Goal: A Process of Ongoing Improvement.

The book is not new, written in the 1980s, when many American manufacturers struggled to compete with the Japanese. The book introduces the so-called Theory of Constraints, which the basis of lean manufacturing. The value of this book to today’s knowledge workers is that it the TOC is still relevant and understanding it puts one’s understanding of lean software development principles and Kanban on a more solid foundation.

The Goal is very different from most textbooks; it is a novel, set in a fictional small Midwestern factory town of Bearington. The main character, Alex, is a plant manager, whose plant is struggling. It has the latest and greatest machines, its machines and workers work hard, but if you look at the assembled goods leaving the plant’s shipping dock, there is little throughput. Orders are constantly late, shipping dates are unpredictable, customers are frustrated. The company is losing business to foreign competitors, who can ship their product faster and make it cheaper and with higher quality. Alex’s superiors give him three months to turn the plant around or else the have to close it.

Boyscouts on a Trail

Alex turns his plant around in the end, but he has to learn the Theory of Constraints to do it. He learns it using Socratic method, taught by a wise consultant named Jonah. One of Alex’s discoveries happen during a weekend when he leads a troop of boyscouts (his son’s classmates) on an overnight hike.

The scouts must hike 10 miles of narrow trail to reach their campsite. The strongest kid, Ron, is in front and sets a reasonable pace, two miles per hour. But Alex discovers at the midpoint that the column is moving at only one mile per hour pace and there is no way they can reach the campsite before sunset at this pace! It turns out that scouts’ speeds vary randomly. The positive deviations don’t accumulate, because you bump into someone. The negative deviations do accumulate, slowing down the column. The slowest kid, Herbie, is way behind, sometimes walking, sometimes stopping, sometimes running, and very tired. The only kid able to sustain two miles per hour is Ron who is comfortably in front.

The fluctuations and dependent tasks remind Alex of something that happens inside his plant. There are statistical variations of tasks performed by workers and machines and there are dependencies: just as Ron must walk over a point on the trail before Dave can walk over it, some machine must process a part before it goes to another machine. (In software development, variability and dependencies between tasks and the economic effects of variability and dependence are even more severe!)

What All This Means to Education

The scouts are our children. The trail is what they need to learn.

For public education to fulfill its mission in the XXI century, it must figure out how to move the whole column at two miles per hour. We live in the era of knowledge economy and globalization — sorry for the cliché — and countries that figure it out will have advantage.

As we know from reading Goldratt, letting the kids walk at their own paces gets us one mile per hour, so that is not an answer. Putting the slowest kid in front and holding back everyone else is not an answer. Measuring kids’ speeds individually on a straight, flat part of the trail (if you get my drift) is not an answer.

What do you think? Makes sense?

Posted in books | 2 Comments

Building Domain Model Checklist

Posted on November 10, 2011 by azheglov

In our last book club meeting, we revisited Chapter 7 of Eric Evans’ Domain-Driven Design and tried to create a checklist for turning a domain-model into an object-oriented design. This is what we came up with.

0. The model is given, not done from scratch. This is the starting point of our checklist. The analysis, the conversations with domain experts leading to this model have already been done at the earlier design stages (and in the earlier chapters of the book).
1. Prune class relationships. Get rid of unnecessary relationships, especially many-to-many.
2. Identify applications — users of the domain model. In a shipping domain model, examples of such applications are a booking application or a tracking query for cargo. Remember that the application and the domain model live in different layers of the layered architecture.
3. Distinguish between entities and value objects. Enough said.
4. Design associationsbetween entities and value objects. Figure out directionality. For example, value objects shouldn’t reference their owners.
5. Figure out aggregate boundaries. Think not what to include in an aggregate, instead, look for reasons to exclude it. If someone wants to look up Y without X that owns it, then Y doesn’t belong in the aggregate where X is root.
6. Select repositories. Limit consideration to the roots of aggregates. If you are implementing an association between two classes as a collection, you don’t need a repository; if you are implementing it as a query, then you need a repository.
7. Walk through scenarios. Remember there is no penalty for throwing away a value object, even a complicated one. Think about associations and what happens to them when your model objects change, especially in aggregates.
8. Design object creation. Decide where to use constructors, what to pass to those constructors, where to use design patterns such as Abstract Factory, Factory Method and Prototype. Make sure you create valid objects that fulfill their invariants and create associations when they’re required.
9. Refactor. Note that we’re talking about refactoring the design, not the code that implements it. There is no general rule on how to do it. The only example in the book shows a refactoring move driven by a performance requirement. An important takeaway is that you need excellent model documentation in addition to its diagram! Otherwise, it may be impossible to tell why you did a particular refactoring move.
10. Think about modularity. The model may be large enough to divide it into several modules. In the shipping model example, such modules could be shipping, billing, and customer contact.

Posted in hands-on | Leave a comment

Notes from Agile Coach Camp: The Lean Startup

Posted on November 6, 2011 by azheglov

This post continues a series of delayed posts about Agile Coach Camp 2011 in Montreal – cross-posted here and on the camp Wiki. Dave Rooney lead it with his notes and impressions from SFAgile 2011.

Very quickly, here are some of the key points.

The five principles of Lean Startup are:

  1. Entrepreneurship is management
  2. Entrepreneurs are everywhere
  3. Validated learning
  4. Innovation accounting
  5. Build-measure-learn

“Entrepreneurs are everywhere” means they can be two founders in a garage or they can be in multinational, century-old, global company. The “startup” part of “lean startup” does not refer to an organization’s age or headcount, only to their mission: discover a new scalable business model in the conditions of uncertainty.

Validated learning is the only unit of progress.

The build-measure-learn loop is the central concept. The result of “build” is a product, which you use to measure; the result of that is data, which you use to learn, which results in ideas that you use to build the next version of the product. The goal is to get through this loop as fast as possible.

In the months since the coach camp, Eric Ries’s book The Lean Startup was published and this much became crystal clear: build-measure-learn is essentially the lean startup’s value stream. At the end, you have produced a unit of validated learning. At the start, you have an unvalidated hypothesis. Part of this long value stream is “build”, which resembles the traditional SDLC value stream in the sense that you start with a request to build a product feature and end with a built feature and its release to users.

At the end of the session, I got up to talk about Steve Blank’s customer development model. (Steve Blank is a venture capitalist, one of the pioneers of the lean startup movement, and the author of The Four Steps to the Epiphany.) I also mentioned a bunch of important points from my book club’s conversation with Steve Blank. There is no reason to repeat them here, but I do recommend that you follow this link.

I am downplaying the customer development model here, because – and this is one of my surprises from reading Eris Ries’ book – Eric downplayed it in his book, giving much more space to lean things, like Kanban and Five Whys. (And you should know that he gave a glowing recommendation to Donald Reinertsen’s book, “Principles of Product Development Flow” — that should give you an idea where I’m going.) Steve Blank’s main message (“there are no facts inside the building, so get the hell out”) is very important to forming a lean startup leader’s mindset. But to really operate this type of organization, you need to go way beyond that and understand Eric’s book. Lean startup in a nutshell is good old lean applied to a new value stream pattern.

Posted in conferences | 2 Comments

Build-Measure-Learn Is a Kanban Optimized for Response Time

Posted on October 4, 2011 by azheglov

In 2002, I had the only layoff of my career, from a company that had spent some time building a product (and they didn’t come) and landed right away in a DevOpsy lean-startapish Internet ad firm. Of course, words such as DevOps and lean startup didn’t exist back then, I’m just using them to describe what it was like.

We built, measured, and learned.

After building a feature, a new version of our software was ready for release and the measuring began. We ran series of experiments on progressively increasing segments of our user base. We would start with a small percentage, say 0.1%, and run a statistical hypothesis test on it. Then we would roll the software out to a greater number of users, run another experiment and so on. The nature of our experiments varied. The first one usually established that the new feature didn’t do any harm and all preexisting functionality worked as before and produced the same revenue. Additional experiments tested hypotheses that the new feature lead to improvement, such as increased revenue per user. The last test would be, can the company use the improved product to attract more paying customers. I worked with Mark, our VP of Advertising Operations, and his staff to run many such experiments.

I was talking with Dave Rooney at lunchtime at the Agile Coach Camp in Montreal three months ago after his Lean Startup session and reflecting on this experience when I realized: the measure-learn part of build-measure-learn is a kanban. We have a value stream here, with several intermediate value-adding states. There are policies for pulling work from one state to the next and, obviously, tight WIP limits. The boundaries of this kanban are “ready for release” (upstream) and “validated” (downstream).

To be sure, there was no Kanban method back then; I’m just using a modern apparatus to think about a past situation. If we knew back then what we know now, our Measure-Learn kanban board might look like this at some point (I dropped the WIP limits from it, because they would evolve anyway and the precise limits are not the point here).

“Ready for release” is of course the downstream state of our Build kanban. Again, the exact software development process used to build the feature is not important here, but we can use the modern Kanban method to model it. The board visualizing it could look like this at one point:

If we join the two kanbans together, we get the joint Build-Measure-Learn kanban. This is the system and our job is to optimize it for response time.

Now, if you have Eric Ries’ book, The Lean Startup and can open it on page 139, he shows a kanban diagram there: backlog-in process-built-validated. If you drilled into “in process”, you’d get more detail. If you drilled into “built”, you’d get more detail. My last diagram gives you one example, from one particular firm, what one more level of such detail looks like.

Optimizing for Response Time

Why response time? Very simple. What are we producing here? Validated learning. Who is the customer of the validated learning? The founders (and/or VCs). They want to discover their next pivot point as soon as possible. They don’t care about throughput (the rate of accumulation of validated learning), but want the key bits of it that lead them to the next pivot point. Therefore, the response time is king.

Observations and Corollaries

  • Optimizing such systems will be hard. A lot of today’s Agile and Lean implementations concern the Build part. And even that is difficult. Lots of people in the software industry can’t quite wrap their minds around the core concepts of Scrum and Kanban. And this Build-Measure-Learn system is more complex.
  • Optimizing for response time may conflict with optimizing for throughput. Optimization for response time is what startup founders want, but not every human activity has the same optimization goal. If you need to optimize for something else, that probably marks the boundary where you should apply the Build-Measure-Learn idea.
  • Optimizing for response time is likely to involve a lot of slack in the system. Imagine startup programmers rapidly learning the newest technologies and creating many open-source products that make other startups even leaner. Wait, this is already happening.
  • If you optimized your software development process, the Build part, your whole Build-Measure-Learn system is probably sub-optimal. But don’t sweat it. The majority of your colleagues doesn’t yet understand that if their software development process comprises, say, analysis, coding, and testing, then maximizing, say, the testing efficiency doesn’t optimize the process. 😉

What do you think?

Posted in hands-on | 1 Comment

Lean Coffee with Steve Blank

Posted on July 6, 2011 by azheglov

I wrote a summary of our book club meeting with the author Steve Blank, discussing the main points of his book, The Four Steps to the Epiphany. Now I want to write a quick note reflecting on the mechanics of facilitating this conversation.

The club had used the lean coffee technique to organize our meetings and been getting better at it with time. The following screen shot was taken at the end of our conversation with Steve and it shows how much we improved after several months of practice.

The conversation lasted only about 25 minutes, but you can see in the board’s rightmost column that we achieved a pretty good rate of throughput. You can also calculate the cycle time; it was about 3 minutes. And we got substantive discussions and expanded answers out of each topic as you could see from the summary.

That is the magic of Personal Kanban.

Posted in facilitation | 1 Comment

Observations at the Passport Office

Posted on July 6, 2011 by azheglov

I spent 34 minutes today at a local Passport Canada office applying for my passport. It is a pretty lean operation. It was actually a very busy day in their office, in the middle of the summer, lots of people wanting to travel. I also had two applications in hand, one for myself and one for one of my children, on two different forms. All things considered, 34 minutes is a very good time. (I’ve been there before, on a much less busy winter day, was out in a fraction of this time, so I kind of know the source of variation.)

I had some time to observe and sketch the workflow. Here it is:

The efficiency ratio in the case of my ticket was 23.5% (value-added time, 8 minutes, divided by 34), which is pretty good.

The check-in station was clearly building quality in. The parts of application, photos, and attachments were checked and arranged in a particular, standardized order (seiketsu – one of the five S’s). If you forgot to fill in some item on the form, the defect was fixed on the spot instead of sending it downstream.

The efficiency, the quality of work done by this office, and its obsessive thinking about the process clearly put to shame most software development organizations.

Posted in life | Leave a comment

Book Club Meeting With Steve Blank

Posted on June 12, 2011 by azheglov

Our Agile/Lean book club is wrapping up The Four Steps to the Epiphany and the author Steve Blank joined us for the last weekly meeting. Here is my transcript of the most important points.

The main takeaways from the book. “There are no facts inside the building, so get the hell out.” The author said startups are not rational projects, if you understood the odds, you’d never do it. Founders have visions of future products and they see things no one else does, but unrestrained passion can lead to two mistakes early on. The first mistake is confusing your vision with facts and the second one is equating understanding of the customer’s problem with understanding the feature set you need to build. Also, almost all investors and venture capitalists come from business schools, but business schools mostly teach to administer and execute business models, while startups are completely different: they’re temporary organizations that search for scalable and repeatable business models.

You should have a co-founder in a technology startup, but you don’t necessarily need an MBA. You have to acknowledge that your technology idea is not a company. You (as a technologist) can teach yourself all the business stuff, but that will take a big chunk out of your “coding.” Out of two or more technical co-founders, someone is naturally the most comfortable outside the building (e.g. Gates, Ellison, etc.).

If you are an employee of a startup and you’ve seen the light of customer development, but your CEO hasn’t, you have two options: shut up or leave. The odds of your being able to educate someone in time are very, very low. As a variation of the same point, if you’re the founders of a startup whose investors unfamiliar with the customer development process and you decided to do it after you’ve taken their money, chances are you’ll be ex-founders pretty soon.

Customer development is not a better version of something that preceded it. It’s very radical to those haven’t exposed to it. The worst way to talk about customer development is to evangelize about it.

Interesting parallels between customer development and agile/lean. You need to know the key concepts and wrap your head around them and it’s OK if you lapse on some corner cases. If you started hiring sales people before you have any proof, you just lapsed out of customer development. If you did a big-bang launch, if you spec’d and started building every possible feature, if your investors tell you “it’s Month 7 and the revenue plan says X”, you’ve dropped it. This is very similar to our Agile redneck jokes.

Customer development and agile/lean engineering really go together. Steve acknowledged that they didn’t go well together in the book, but they really do so in the life. He was blunt about waterfall: if your engineering team is doing waterfall, put down this book and step away from it!

What should engineering team do during the customer discovery phase? Assume they are building the product. Execute the original vision, but as soon as you have feedback from the outside, re-plan and change the direction. Customer discovery is not a focus group!

It is intimidating to start customer development without having a viable product. Steve agreed with it recounted an episode from many years ago when he was an engineer and a founder and being so frightened that he “stared at a telephone for three days hoping it would dial itself.” But it was helpful to remember that the purpose was not to sell anything, but to listen.

Posted in books | 3 Comments

Learning at the Coding Dojo

Posted on June 4, 2011 by azheglov

Last weekend, I went to the Communitech Coding Dojo in St. Jacobs, not far from Waterloo.

Coding dojo is different from a code retreat. In a code retreat, you have the same problem all day and use each iteration to try different approaches it. The problem is usually too large to be solved in one iteration. In a coding dojo, you work on a series of challenges, called katas, each of them small enough to fit into its time segment.

We did two coding katas in the morning and one legacy code refactoring kata in the afternoon.

Rather than boring my reader with recounting every hour of the day, I want to just mention a few important from this event.

I liked the two-phase structure of the refactoring kata. Alistair and Jason designed it so that the first phase was done on paper. We read the code (working in pairs!) and marked the printouts, identifying various code smells. In the second phase, we made the actual changes to the code, covering it with tests first, them making changes while keeping the tests green.

Here is what the result of the first phase looked like:

And this is a picture of me during the second phase. This legacy code was painful to look at!

Introductions at the start of such events lead to safety. Declan pointed that out in the final retrospective. People who are new to coding dojos and code retreats may be concerned about what to expect, whether their skills are good enough, whether their questions may sound stupid, etc. They need to feel safe asking their questions and making their contributions. Think of it as “hire for attitude, train for skills”, except instead of getting hired people just show up. A circle of introductions can ensure everybody appreciates their attitudes.

Several other important points were mentioned in conversations and I can capture them here as bullet points:

  • It is more difficult to identify and name code smells than to simply point to the place in the code that you want to change by refactoring.
  • With more TDD experience, programmers tend to use fewer mocks.
  • Eliminate duplication ruthlessly in production code, but it is much more tolerable in test code.
  • Stubs and builders are a powerful combination. (The builder pattern here refers not to the GOF pattern, but to test data builders described in Growing Object-Oriented Software, Guided by Tests.) Occasional breaking changes to builders are not a real pain point.
  • Don’t skip the dinner with other attendees after the coding is done!

I posted the picturestaken at this event on Flickr.

Posted in hands-on | 2 Comments