Google's Pathways Language Model and Chain-of-Thought

Correction!! The model costed around 10M not 10B! Thanks for raising that. Mistake during copying from the second slide :(

The article quotes the cost as roughly 10B$ in the first paragraph. Likely a typo? They quote 10M$ in a later paragraph.

The amount of capital needed to train these high-quality models is eye watering (not to mention the costs needed to acquire the data). Does anyone know of any well capitalized startups exploring this space?

I've talked about structural deficiencies in earlier language models, this one seems to be doing something about them.

$10M for a bag of numbers (i.e the learned weights of the model matrices)