Hi HN,<p>I run a review website (similar to Yelp) for a niche business market. Currently, business owners can manage their listings for free, but I would like to make some money now by offering additional features. I don't know if I should offer one premium tier for say $29 a month with all the features, or multiple tiers ranging from $19 to $39 a month depending on what features they want (think: premium, platinum, ultimate).<p>Before I create any of these features, I'm going to see if my current customers are even interested in paying for them. So I was thinking of doing A/B testing, where half of my customers would be informed about a one tier structure, and the other half about the multi tier structure. But here's my problem: I only have 170 people who are currently signed up to manage their business listing. Is it worth doing A/B testing with such a small population? Or should I just pick one option and go with it, hoping for the best?<p>TL;DR: Is having 170 people too small of a population to do effective A/B testing?<p>Thanks!<p>EDIT: changed the wording to use correct statistics terminology.
i would say, it depends on the results, if from the 170 you get a contundent answer then I would say its a go, if results are not contundent enough you can keep your a/b running until it does
Well if all you have is 170 people then you're not sampling but testing the population.<p>I'd go with the A/B testing option just to get more experience with it. Having data at a low confidence interval is better than having no data at all. Maybe it's a wash maybe it's actually very important. Worst case is there is no overwhelming winner and you're back where you started.<p>Use the following link to determine the sample size you need.
<a href="http://www.surveysystem.com/sscalc.htm" rel="nofollow">http://www.surveysystem.com/sscalc.htm</a>