An interesting playground to understand the scheduler is here <a href="https://huggingface.co/spaces/sail/zero-bubble-pipeline-parallellism" rel="nofollow">https://huggingface.co/spaces/sail/zero-bubble-pipeline-para...</a>