Jagged Flash Attention Optimization

Flash attention natively supports packing multiple variable length sequences into a single call, what is the advantage of jagged flash attention?

See also<p><a href="https:&#x2F;&#x2F;github.com&#x2F;thu-ml&#x2F;SageAttention" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;thu-ml&#x2F;SageAttention</a> and <a href="https:&#x2F;&#x2F;github.com&#x2F;thu-ml&#x2F;SpargeAttn" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;thu-ml&#x2F;SpargeAttn</a>