Is it just me, or is this bad statistics?<p>With a small sample size and large numbers of personas / categories you would expect to see a positive bump, even if there was no statistical relationship between the persona and the preference. Since you are only eliminating categories that don't happen to be represented in the subset you are testing, you can only ever actually go up.<p>For demonstration I rolled 20 dice randomly for 6 personas and 3 categories of preference:<p>1, 5, 4, 4, 4, 2, 6, 1, 4, 2, 1, 2, 5, 6, 3, 6, 3, 2, 1, 3<p>A, A, A, A, B, C, B, A, C, A, C, A, B, C, A, A, B, A, B, B<p>A = 10, B = 6, C = 4; Which gives me 20% for C<p>I restrict myself to just the numbers that voted for C (2, 4, 1, 6) removing all 3 and 5s<p>I now am left with:<p>1, 4, 4, 4, 2, 6, 1, 4, 2, 1, 2, 6, 6, 2, 1<p>A, A, A, B, C, B, A, C, A, C, A, C, A, A, B<p>This now gives me A = 8, B = 3, C = 4<p>And now I get 27%, a nice 35% boost! Even better than Superhuman's 10% boost. But this is all an illusion, there was absolutely no dependency between the persona and preference here, which you would only see with a large enough sample size.