Yandex, Hugging Face, HSE, Moscow Institute of Physics and Technology, University of Toronto and the Vector Institute propose a method to share compute power and ran a decentralized collaborative training of an Albert-like model in Bengali.<p>https://arxiv.org/abs/2106.10207<p>https://github.com/yandex-research/DeDLOC<p>https://twitter.com/m_ryabinin/status/1406988626374762503