I would say that comedians have always been among the top professional A/B testers (long before the buzz around A/B testing). Their entire professional success or failure depends on constant small and large adjustments to their routines based on testing and feedback. And when it comes to being funny and staying funny, comedy is brutal and completely unforgiving.<p>Here's a great 50 minute segment, with Jerry Seinfeld, Chris Rock, Ricky Gervais and Louis CK, in which they discuss various aspects of their material and how they work with it (adding / discarding / making adjustments and so on):<p><a href="https://www.youtube.com/watch?v=sAxoRh06XM4" rel="nofollow">https://www.youtube.com/watch?v=sAxoRh06XM4</a>
I have no idea if this is at all related to the topic (and I realize this reads like typical HN smarter/hipper-than-thou criticism, so I apologize in advance) but while I find Aziz Ansari to be hilarious in roles (Parks & Rec, his small role in Flight of the Conchords, etc), I've seen a few of his standup shows and didn't find his own material to be particularly funny at all.
There's a ton more to being a good comedian than just A/B testing a few jokes. (Good jokes are required but not sufficient?)<p>A good stand-up (like a good DJ) "feels the energy" in a room and subtly alters his delivery as well as the content to work the crowd.
Steve Martin talked about his process making his comedy routine in his autobiography. His process was essentially A/B testing, although not as precise as this approach can be today.
Getting statistically useful data out of this will be difficult. A/B tests can tell you which of a set if options performs better for a measurable metric, but not <i>why</i>, and without no visibility into 'metrics' you <i>can't</i> easily measure -- of which there are a lot.