Hey there!<p>I've been working withe a friend on a pytorch implementation of DownpourSGD, a distributed optimization algorithm. I ended up writing a blog post about our implementation and some problems we ran into along the way. It was a pretty interesting project to work on, would appreciate if y'all checked it out!<p>Post: https://jcaip.github.io/Distbelief/<p>Github: https://github.com/ucla-labx/distbelief