The money quote at the end: Randomization doesn’t make a study worse. What it can do is give researchers and consumers of researchers an inappropriately warm and cozy feeling, leading them to not look at serious problems of interpretation of the results of the study, for example, extracting large and unreproducible results from small noisy samples and then using inappropriately applied statistical models to label such findings as “statistically significant.”<p>Spot on, unfortunately.
Kinda related, but the easiest way to limit bad randomizations is to stratified by pre-experiment data. But then that would make it harder to p-hack so its understandable why more people dont do that ;)
He's complaining about the statistics version of a con artist putting on a nice suit, or making a glossy website for your crap company. Or putting out a "white paper".<p>Adding the easy trappings or dishonest indicators of "professionalism" to your junk product is bad for society.<p>It's not a novel insight, and the discussion is rather cluttered with irrelevant detail, so I not sure it's worth sharing. (Of course it's fine for Gelman to write his meandering thoughts on his blog.)