Software Delivery in Small Batches

Adam explains how Deming's "Red Beads" experiment demonstrates the system of profound knowledge and the problems with management by objective.

Show Notes

Free Resources
Links
★ Support this podcast on Patreon ★

Creators & Guests

Host
Adam Hawkins
Software Delivery Coach

What is Software Delivery in Small Batches?

Adam Hawkins presents the theory and practices behind software delivery excellence. Topics include DevOps, lean, software architecture, continuous delivery, and interviews with industry leaders.

Hello and welcome to Small Batches. I’m your host Adam Hawkins. In each episode I share a small batch of software delivery education. Topics include DevOps, lean, continuous delivery, and conversations with industry leaders. Now, let’s begin today’s episode.

Dr. Deming’s played a crucial role in shaping lean thinking, which flows directly to modern software engineering. His 1994 book "New Economics" explained his system of profound knowledge in four parts.
First, appreciation for system. Second, knowledge of variation. Third, theory of knowledge. Fourth, understanding of psychology.

The book "New Economics" also describes an experiment known as the "red beads". The experiment demonstrate the system of profound knowledge and the problems with management by objective. The experiment has become well known and practiced around world. It’s simple and the truths it reveals cannot be ignored.

Here’s how the experiment works.

The goal is to separate red beads from a pile of red and white beads. Willing workers are given a paddle. The paddle has depressions in it to pick up a fixed amount of beads. The willing workers must dip the paddle into the beads, then show the results to inspectors who record how many red and white beads are picked up by each worker.

Management knows how the process should work in detail. Willing workers are given detailed instructions on how far to lower the paddle into the beads, how far to lift it up afterward, even down to the angle to tilt the paddle so beads fall off properly.

Management explains to the willing workers that their jobs depend on their performance. They are not to ask questions, just do their jobs. Management assigns a performance target. They also want to encourage the willing workers so they promote slogans like "do it right the first time", "take pride in your work", or "quality first".

The experiment kicks off with willing workers following the process, recording the results, and being judged accordingly.

Before I say anything more, I’d like you to pause for a moment and predict the outcome of the experiment. What do you think will happen? What will you learn from the result?

The tl;dr of the experiment is that the results are entirely driven by system. The individual people have no control over the outcome. However, management will still judge, rank, promote, and fire the willing workers based on performance. Those decisions do not reflect on the individual but on the process. That has a demoralizing effect on the willing workers, who are of course, performing to the best of their abilities.
Now, let’s examine this using the system of profound knowledge.

The first part of the system of profound knowledge is appreciation for a system. Every system has an aim. The system in this experiment aims to sort red and white beads. Every system must be managed to achieve the aim. We must see the system as the sum total of its components and process.

In this experiment, workers are given a paddle that can pick up a fixed number of beads with specific instructions that no deviations from the process are allowed. Thus, there is no opportunity for the willing workers or management to improve the system. So the outcome is predetermined. Management will—at best unknowingly, and at worst knowingly—rank workers based on results. This is the system they operate in: achieve targets, promote high performers, and fire low performers. Now, we begin to see how incentives influence the system.

The second part of the system of profound knowledge is knowledge of variation. Life is variation. All systems have variation. Management’s aim is reduce variation which results in more stable and predictable systems. In the red beads, management thinks that ranking willing worker is based on individual performance. This is not true, they’re ranking willing workers based on the variation intrinsic in the mechanical sampling process.

Each worker is given the same paddle. What beads they pick up with each sift is statistically predictable. It follows a distribution. Think about it.

There will always be variation in where the paddle is placed, where the beads are at that moment in the pile, and what beads roll off when the paddle is tilted. There is no way to hit a target for each sift, let alone for multiple sifts. This variation is intrinsic to the system. Remember that willing workers are given explicit instructions that no deviation from the process is allowed, so it’s not possible to reduce the variation. Thus the results are the same each time.

The next part of the system of profound knowledge is the theory of knowledge. This aspect focuses on data based predictions backed by working theory of the system. In the red bead experiment there’s a mismatch between working theory and the system. Management follows MBO (or Management by Objective).

They think they’re doing the right thing by giving willing workers performance targets then ranking them accordingly. But management has no rationale for why some willing workers deliver more beads than others. They’re tracking everything, thus gaining information about the system. However Deming tells us: "information is not knowledge".

The fourth and final part of the system of profound knowledge is an understanding of psychology. This speaks to the human factor. Humans are not entirely rational. We are emotional also.

Consider the willing workers in the red bead experiment. They see themselves ranked, promoted, and fired but with no control over their own performance. Consider the managers: frustrated with the willing workers and dealing with the stress of firing them.

One participant in Deming’s red bead experiments wrote him a letter. I think it gets to core of why it’s crucial to understand psychology. Here’s what she wrote to to Deming.

When I was a willing worker on the red beads, I learned more than statistical theory. I know that the system would not allow me to meet the goal, but I still felt I could. I wished to. I tried so hard I felt responsibility: others dependent on me. My logic and emotions conflicted, and I was frustrated. Logic said that there was no way to succeed. Emotion said that I could keep trying.

After it was over, I thought about my own work situation. How often are people in a situation that they cannot govern, but wish to do their best? And people do their best. And after a while, what happens to their drive, their care, their desire? For some, they become turned off, tuned out. Fortunately there are many that only need the opportunity and methods to contribute with.

You may perceive the red beads in your own company and in your own work.

Alright, that’s all for this batch. I hope you enjoyed the episode as I’ve enjoyed sharing the red beads with you. The red beads are a wonderful doorway into Deming’s deep and wide ranging work. I have two things to help you learn from this master thinker.

The first is the Small Batches slack app. I’ve loaded the app with my favorite quotes from Deming’s books, along with tips, and examples of the system of profound knowledge applied to software delivery. The app is currently free in beta for a limited time, so sign up today.

The second is previous episodes of this podcast. There are episodes on variation and interviews with John Willis. John Willis is one of the co-authors of The DevOps Handbook. He’s also host of the Profound podcast which explores Deming’s influence on software engineering.

Find links to the small batches slack app and all the Deming podcast episodes at SmallBatches.fm/67.

Alright, I hope to have you back again for the next episode. Until then remember that life is variation. Happy shipping.