Whenever I talk to people about downsides of A/B testing, I always try to talk to them about this slide:<p>< <a href="https://www.chrisstucchio.com/pubs/slides/gilt_bayesian_ab_2015/slides.html#18" rel="nofollow">https://www.chrisstucchio.com/pubs/slides/gilt_bayesian_ab_2...</a> ><p>This one visual sums up the most ubiquitous naive use of A/B testing, which I've seen practiced everywhere I've worked.<p>The whole slide deck is very worthwhile.