<i>After a while, I ran the same analysis I did before. The shopping cart abandonment rate was reduced by double digits: A difference worth more than $1 million dollars per month.</i><p>This is not a statistically valid way to test the effectiveness of your change.<p>For example, if you sell toys, and you run an analysis on the week before Christmas, then you make your code change, and then check again on the week after Christmas, you are going to have differences in customer behavior that have nothing to do with your change.