Google breaks the trillion-parameter ceiling with the Switch Transformer

It’s an interesting thought experiment to consider models that have more parameters than there are data points being analyzed.<p>What does that mean?

This is impressive but also requires a lot of power to train.<p>If this trend continues ML will soon surpass bitcoin as the worst polluter.