Your product is amazing and you want to make it even better! (Or it currently doesn’t even work but you know it can be great so you’re trying desperately to improve it…) Either way, you know that experiments can help you determine which things in your manufacturing process actually have an effect on the final product and which don’t.
But did you also know that there are two basic ways you can structure your set of experiments? One you will have learned about already: make a list of all the factors you want to test and then start testing them one at a time. The advantage of this is that it gives you very clean data on the impact of each factor.
The disadvantage?
It might not lead to useful results at all! What if you spend months optimizing everything, and in the end, your product still doesn’t work as it’s supposed to, to the point that you can’t even sell it???
This is Rhonda. If you’re a regular PCat reader, you know that Rhonda is a retired nurse who now runs a small business making wooden items to sell in her garage workshop. Her latest product is a wooden beer mug in the shape of a garden gnome.
Anyway, just like you, Rhonda has been working to improve her product using experiments! Specifically, she’s been trying to make it more watertight. And her friend Process Cat has been helping her.
So, Rhonda asks, how can I avoid spending months and months only to find out that there’s NO WAY to get my gnome mugs watertight enough for my customers???
Enter Proof-of-Concept Experiments
Proof-of-Concept Experiments aim to test whether it is even possible to improve something. This is done by doing all the things that you think might make a difference in one group, vs. nothing to a control group.
A famous example is the first part of the Framingham Heart Study In that study, participants did everything that the researchers thought might help prevent heart disease. They exercised, changed their diets, and a bunch of other things. And the researchers found that it was possible to reverse cardiovascular damage, which previously hadn’t been proven!
Now, it’s important to note that this study came under heavy criticism when it came out, and you can probably guess why: there was no information on which of the factors had contributed to the reversal of cardiovascular damage.
And so it was with Rhonda’s experiment. She used the most watertight wood and her new CNC carver.
And she found that….
The Gnome was now watertight!
So that was a great finding. But she’s not done. See, some of the changes to her protocol are expensive and others are time-consuming. So she needs to go back now and do single-factor experiments to determine which changes are actually necessary and which aren’t.
Weekly Challenge:
Think through any improvements you want to make to your own products. Are there any that would respond well to a Proof-of-Concept experiment? If so, set one up and run it!!!
