<i>About halfway through the campaign we figured out that of all variables that affect user behavior (design, usability, imagery, etc.), copy has the highest ROI</i><p>I thought I'd highlight this because it is true, apparently not obvious since everyone discovers it for themselves, and <i>wildly actionable</i> by many people on HN.
<i>> Overall we executed about 500 a/b tests on our web pages in a 20 month period which increased donation conversions by 49% and sign up conversions by 161%.</i><p>How did they measure this? Did they isolate the effects of the a/b testing? Donation and sign up conversion rates will increase anyways as the day of the election approaches.
HN-ers, I'm new at this whole A/B Testing, Copywriting, "testing your hypothesis", and the whole "trying to sell something" + "MVP" (or the Lean Startup).<p>I recalled yesterday I saw a link to Intuit methodology that also do a similar A/B Testing to decide what to build: <a href="http://www.forbes.com/sites/bruceupbin/2012/09/04/intuit-the-30-year-old-startup/" rel="nofollow">http://www.forbes.com/sites/bruceupbin/2012/09/04/intuit-the...</a><p>Could someone connect the dot to me or point to me to somewhere that I can make sense of the whole thing?<p>How can one come up with many pages so quickly to test? and how one could decide what the content of the page to test is?
Great write up. For those interested, I've found similar results in five years of A/B testing on a non-profit fundraising site (www.globalgiving.org).<p>Pro tip: Don't just copy what Obama did. Your mileage will vary based on your user base and context. Take their learnings and A/B test it on your own site to see if it works for you or not. Not all optimizations are universal.
The biggest surprise was the 5% increase for breaking the steps - seems little - when compared to the copy and imagery change (massive 19% and 21%).<p>That's kinda insane! Love the report, thanks.
I have mixed feeling about "Now save your payment information". "Save your payment info for next time" makes it clear that it is optional whereas the latter makes it look like part of the process. It might increase conversion(and did per the post), but I get the feeling that is somehow conning the user into signing up.
Why is it that there's no Romney campaign write-ups? Are they that less open of a tech community? Yes, I know they lost, but technically, they beat out many other competitors in the Republican primaries and so must have exercised some kind of technical prowess?<p>(and yes, I'm aware of their fail-whale Orca system....that would also make for a fun writeup)