Adopting experimentation as a key activity for your organizations can be difficult, even for startups. Some try it and quickly abandon the process because the experiment didn’t work out the way they anticipated. That is exactly the point.
Let’s recap. Experiments have unknown outcomes. For organizations, this can become a barrier to adoption. When the results are confirming of assumptions the results are easy to accept. When the experiment’s results challenge or even run contrary to an organization’s assumption, the impulse may be to discredit or rationalize them away. And when the contrarian results come as part of your first experiment, it can drain the energy for additional experiments.
Experiments are best committed to in bunches, not singularly. This reduces the possibility that a lone experiment can sap the enthusiasm for adopting an experimental mindset. Confirming results builds confidence by proving that a project or initiative is on the right path. Contrarian results may reduce that confidence. It is important to see that both are positive results. The conversion of a core assumption into an evidence-based fact is the goal.
If an organization collects nothing but confirming results, they are likely testing the wrong things. If you already know the answer, don’t waste precious resources asking the question.