Using bigger supercomputers to get marginal improvements probably involves overcoming whatever problems exist in training ever larger neural networks, and big nets are inevitably needed for classification of more than just 1000 classes. But the original breakthrough result by the SuperVision team from Toronto (15% error, not completely comparable) trained their net on just 2 GPUs for a week. It would be a pity if it's no long possible to beat the state of the art with your home PC and good ideas.
We knew this in January when Andrew Ng mentioned both the depth and size of the computational layer. Again, the scale that Baidu brings to machine intelligence is remarkable. But I wouldn't put Google or MS out of the picture just yet. There is much not disclosed yet.