I've been involved in two types of usability studies. The first are academic, rigorous, well-designed experiments testing readability, visual search, or other issues in web site design. These were designed to provide statistically significant results. The second are small focus groups used to test, predominantly, GUI design elements.<p>This article claims the second kind of test is worthless and misleading, but I think that's ignoring obvious benefits. No, you don't get statistically significant results, but in most cases you're not aiming for some result like "85% of users prefer the blue button". More often, you want to watch a handful of people use your interface and a result like "2 out of 5 were really confused by the menu structure, so we should rethink that". That is, your useful data comes from watching the process, not recording the result.<p>Rigorously-designed experiments which provide the potential for statistically significant results are great and they definitely have a place. But they're not the only tool in the box.