The RPC framework for model parallel training is super cool: <a href="https://pytorch.org/tutorials/intermediate/rpc_tutorial.html" rel="nofollow">https://pytorch.org/tutorials/intermediate/rpc_tutorial.html</a>