One other thing that should be discussed in the post that I forgot about is the amount it costs to properly test a paid acquisition channel. Many underestimate this cost. But getting statistically significant results on just a few variations or targeting segments can be.
> "The rise in A/B testing and other analytics tools have created fairy tale stories of changing a button color, or moving the CTA from the left to the right and suddenly you have game changing improvements. Once again, these stories are the exception, not the rule. It typically takes 10 A/B tests to find one that produces any improvement at all. And when you do have a positive improvement, it is typically incremental instead of game changing."<p>Amen!