How should we test in the Red Beads Experiment?

The Red Beads Experiment was a part of Dr. W. Edwards Deming’s four-day seminars for senior management in which he asked members of the audience to play roles in a factory that made white beads.

The experiment looks very simple but is constructed to highlight common problems in the industry. It is also useful for testers to look at the experiment to see what contribution the testers make in the experiment and whether they could have a more positive effect.

W. Edwards Deming is best known for his work in rebuilding the Japanese economy after World War Two. His ideas influenced Toyota, and are part of lean and agile. In later life, he worked with US companies such as Ford and Proctor and Gamble. He is also referred to as “the grandfather of quality”.

The experiment starts by asking six members of the audience to be Willing Workers, two members of the audience to be Inspectors, one member of the audience to be a Chief Inspector, and one member of the audience to be a Secretary. Dr Deming acted as the supervisor. The volunteers are all to work for the White Beads Company, which makes white beads.

The Inspectors can be seen as being testers, and the Chief Inspector can be seen as being the Test Lead or Test Manager.

The supervisor explains the roles to the volunteers. The willing workers are given precise instructions as to how to put a paddle, that contains 50 holes for beads, into a vessel that contains white and red beads. They are also told at which angle to bring the paddle out of the vessel. When the paddle comes out of the vessel it contains both red and white beads. Each action with the paddle represents a day’s work for a worker. The workers take the paddle to the inspectors who count how many red beads are in the paddle. The red beads are defects as the willing worker’s role is to create white beads.The chief inspector will compare the count of the two Inspectors and announce the correct count. The secretary will then record the number of red beads against the name of the willing worker.

On Day One the supervisor announces that there is a merit system. After each willing worker has used the paddle to extract beds from the vessel, the supervisor comments on whether the willing worker deserves a merit increase or will go on probation. The supervisor is disappointed with the results of the first day as there are too many red beads and announces that management has declared a goal of no more than three red beads in a day’s work.

The supervisor is also disappointed with the results of Day Two. He tells the willing workers that their jobs are dependent on their performance and that costs are greater than revenue.

Day Three starts with announcements that it is “Zero Defects Day” and posters are put up saying “Take pride in your work”. The supervisor comments on the number of red beads in the day’s work done by each willing worker telling them whether they get a merit or are on probation and reminds the workers that management is watching costs.

At the beginning of Day Four, the supervisor informs the willing workers that “quality is up to you” and says that management has a plan. The plan is to keep the plant open by only employing the best workers. At the end of the day, the three willing workers whose paddles contained the most red beads are laid off and the other three willing workers are asked to work double shifts to keep the plant open.

On Day Five the three remaining workers work double shifts. The supervisor is disappointed as the results have not improved and announces that management has decided to close the plant.

In his book “The New Economics” Dr. Deming gives fourteen lessons from the Red Beads Experiment. These lessons are mainly for management. He says that the variation in the number of red beads in each day’s work came entirely from the system and that there was no evidence that any one worker was better than any other.

Dr. Deming wrote that the willing workers were victims of the process as they had to follow the supervisor’s rules and had no chance to improve the process. The inspectors are victims too as they also lose their jobs.

The inspectors and chief inspectors are the test team in the plant. How do they contribute to the work of the plant? They independently report the number of red beads in each worker’s work to the chief inspector. There is a system of inspection as the two inspectors are independent of each other but do they add value to the production process?

It is useful to consider what lessons there are for testers in The Red Beads Experiment. The inspection/testing function in the plant is carried out by the inspectors and chief inspector. The inspectors record the number of red beads, defects, in the work done by the willing workers and the chief inspector reports the results.

Dr. Deming says, in The New Economics, that we may perceive Red Beads in our own company and in our own work. The only role of an inspector is to find red beads, that is to find defects. They only look for defects once the work has been done, when the defects already exist. This is not unlike testers who work in “waterfall” teams who only test work that is “finished”. The work of the inspectors does not improve the quality of the plant’s output. The chief inspector’s only role is to report the number of red beads found, that is to report the number of defects. This also does not improve the quality of the plant’s output.

Management should support the inspectors working to break down the barriers that exist between the inspectors and the willing workers and between The White Beads Company and its suppliers. Breaking down these barriers will enable collaboration which can improve the quality of incoming material, of production methods and so improve the output of the plant.

The inspectors would add more value to the plant if they worked to prevent defects rather than working to find defects. The quality of the product is poor by the time the inspectors inspect it, therefore the inspection is too late to improve quality. They should stop inspecting the output of the willing workers and instead use their skills in testing, their knowledge of quality, and production to help to build quality into the plant’s processes. There are several ways that they can do this.

The inspectors and chief inspector are a different team from the willing workers. The willing workers endeavor to produce white beads, and the inspectors count the number of defects. The two teams have different goals: one to produce white beads and the other to count red beads. However, neither of the teams are autonomous, so neither of the teams is a closed system. If management viewed the two teams as part of a system that produces white beads, the two teams could work together to improve quality. An example of how the inspectors could work together with the workers to improve quality would be that the inspectors could use their knowledge of inspection and production to experiment with new methods of production to see if they could find a more efficient method of producing white beads. They can make improvements using Deming’s Plan-do-study-act cycle by planning the trial of a new production method, doing the trial, studying the result of the trial, and acting on what they have learned from the study.

Working with suppliers to improve quality would benefit the White Beads Company. The company would benefit from viewing itself and its suppliers as a system that produces white beads. One of the fourteen lessons that Dr. Deming draws from the Red Beads Experiment is that management could have worked with the suppliers of beads to improve the quality of incoming material by reducing the number of red beads in the supplied material. The inspectors and chief inspector have experience of the issues with quality in the plant and can use this to work with the suppliers to create operational definitions at each stage of production to reduce the number of red beads in the incoming material. In “Out of the Crisis” operational definitions are defined as putting “communicable meaning into a concept.

Adjectives like good, reliable… have no meaning until they are expressed in operation terms of sampling, test and criterion”. If the inspectors and chief inspector help the supplier create operational definitions at each stage of production of the raw material they can help the supplier reduce the number of red beads in the material, which will enable the willing workers to produce more white beads. The inspectors testing skills will help them define operational definitions as they will understand how to create criteria that are easy to test. By working with suppliers to improve the quality of raw materials the inspectors are helping to ‘build quality in’ rather than trying to inspect quality in after the work has been completed.

The chief inspector has a senior position and can use their position to investigate how the team of inspectors can improve quality. Dr. Deming’s System of Profound Knowledge provides a system for management that can be used to improve quality. There are four parts of the system.

The first part is the appreciation of a system. The chief inspector should look at the White Beads Company using systems thinking in order to see the whole system that produces white beads and the interaction of components within the system.

The second part is a knowledge of variation. The volume and quality of production will vary. The chief inspector should use their knowledge of statistics to understand the causes of this variation.

The third part is psychology. People are not cogs in a machine and the chief inspector should understand people and how people interact, as people are fundamental to a company’s success.

The fourth part is the theory of knowledge. A theory is a prediction of the future based on our understanding of the past. The chief inspector should develop theories on how to improve the quality of the output of the White Beads Company and then use the plan-do-study-act cycle to test the theories.

The chief inspector should be supported by management in using Dr Deming’s System of Profound Knowledge to continually improve the company’s processes.

In the Red Beads Experiment, the inspection process in the White Beads Company is only designed to find red beads and this is very much like testers who only test “finished “ code. The inspectors can help the White Bead Company to survive and grow if management supports them in viewing the whole system of producing white beads and in using their skills to ‘build quality in’ at each stage of the process. The chief inspector can also continually improve the company’s processes if they are supported by management to use Dr. Deming’s System of Profound Knowledge.

Further Reading:
Out of the Crisis by W. Edwards Deming
The New Economics by W. Edwards Deming
The Fifth Discipline by Peter Senge
DemingNext from the Deming Institute