TechEcho

> DualPipe was created and developed by Jiashi Li and Chengqi Deng and Wenfeng Liang.<p>A CEO who codes.

I attached all 3 algorithms 1F1B (1 forward 1 backward), ZB1P (zero bubble pipeline parallelism) and DualPipe as a picture here: <a href="https://x.com/danielhanchen/status/1894937006352031832" rel="nofollow">https://x.com/danielhanchen/status/1894937006352031832</a> for those interested :)

Sorry for us utter simpletons can someone explain what it do?

I hope all the open sources Deepseek is doing encourages American labs to do more of the same. Surely they'll realize their momentum is more of a moat than their tech at any one point in time.

Does this remind anyone else of the Pied Piper compression algorithm?

Hmm weren’t there also supposed to be the SM re-allocation, doesn’t look like it was included; I may have been mis-remembering the explanation.

> DualPipe was created and developed by Jiashi Li and Chengqi Deng and Wenfeng Liang.<p>A CEO who codes.

Sorry for us utter simpletons can someone explain what it do?

I hope all the open sources Deepseek is doing encourages American labs to do more of the same. Surely they'll realize their momentum is more of a moat than their tech at any one point in time.

Does this remind anyone else of the Pied Piper compression algorithm?

Hmm weren’t there also supposed to be the SM re-allocation, doesn’t look like it was included; I may have been mis-remembering the explanation.

DualPipe: Bidirectional pipeline parallelism algorithm

6 comments

DualPipe: Bidirectional pipeline parallelism algorithm

6 comments