https://github.com/kakaobrain/torchgpipe<p>Kakao Brain announces torchgpipe, an implementation of GPipe in PyTorch as a handy library.<p>GPipe is a scalable pipeline parallelism library published by Google Brain. It leverages the training of a giant model which requires much memory. For instance, Google trained AmoebaNet-B with 557M parameters over GPipe.