How We Increased Conversion Actions 218% And Increased Sales 0%

Point is make sure you're optimizing around a QUALIFIED action.<p>EG - in the Skirtter conversion funnel - you're not disclosing it's a paid service until checkout.<p>That's doing two things:<p>a) Creating a jarring experience for users that are genuinely interested and didn't see any $$$ requirement before investing in watching a tutorial.<p>b) Sending lots of non-qualified people through the funnel.<p>You’ve got some great content here: <a href="http://www.skritter.com/pricing" rel="nofollow">http://www.skritter.com/pricing</a> - maybe incorporate a little of that?<p>Alternatively use a testimonial to disclose $$$ like: "I learned more during the past 5 months than the past 3 years – best $9.95 I ever spent!"

Thank you for this. I especially appreciated that you have brought the concept of "publication bias" (though not by name) into the discussion of AB testing. In medicine there is a tendency to publish positive results and not negative results, leading to a bias in favor of positive results. Most likely, this is an equally large or larger problem for bloggers.

The greatest issue, as you acknowledge in your post, was that you weren't testing the point closest to the sale. The first thing to test is the final registration page. I often spend a fair amount of money to drive traffic that I can optimize against to single page landing pages. If you can't make enough money through Google Ads or Facebook ads (e.g. your cost per sale &#62; revenue per sale), then you either have a bad business or a shitty landing page.<p>Rather than just optimizing existing traffic, buy traffic and find out how far off you are. The best part of these ad systems is that they are relatively efficient. That means you should be able to quickly figure out how far you are off.<p>While I'm sure "learn Japanese" has already been taken, there are probably plenty of long-tail keywords as well as people on Facebook who haven't been targeted. Your key is going to be to find out who those people are and drive them to your site.<p>I tend to be pretty straight forward with my own investments in projects: if there's a huge gap between cost of acquisition and revenue per customer (or present value of the LTV of the customer), you probably need to build something better or change your product all together. Your own internal cost per user will end up being just as high as google ads (if not higher), except google provides an instant vehicle for testing. In other words, it will help you fail quicker ... something that's extremely valuable in the world of startups.

I think that many people that post about AB tests fail to mention that one's expectations of AB testing have to align with the types of results one can get from AB testing.<p>AB testing isn't meant to used through the entire conversion funnel. That is, you can't measure an optimization on your homepage by sales. Sales is an indication of your entire conversion funnel. Each page of every site should have a single goal and you should AB test to that goal. In your case I would say that you're AB test was successful; however, you didn't go far enough. Once you were getting increased traffic to your signup page, it's time to AB test the signup page and improve that conversion.<p>The statistics behind AB testing are often lost by the hype. The tests used to calculated statistical significance require a random sampling and isolated variables. The moment you try to attribute success or failure of your AB test beyond the primary measurable goal, you're introducing outside variables that will skew your results.<p>All of that being said, I would argue that true AB testing is not cost effective for most startups. It takes a lot of timer and energy to test every part of your conversion funnel.

In this situation I would recommend A/B testing to signup. Because that is the first hard decision that people have to make.<p>If you're A/B testing to actual sale, I have two points to make. The first is that you should be aware that it is OK to run multiple A/B tests at the same time. The second is that you should pay close attention to whether there is a possibility that your change will compress the sales cycle. If it does, an apparently positive result can be meaningless.

This is a great article to demonstrate that testing automatically doesn't guarantee boost in conversions.<p>On the homepage, you optimized for conversions to signup page, so you got that! If on signup page, you optimize for signups, you will get that (hopefully). The only thing testing does is to substitute blind flying with a set of gauges. The act of coming up with test objectives and good variations is much more vital for increasing sales and conversions.