ThunderKittens: Simple, fast, and adorable AI kernels

This is super cool! Especially matrix mult getting similar or better perf than cuBLAS! If anyone is interested on other kernels like swiglu, geglu, RMS layernorm, I coded some at <a href="https:&#x2F;&#x2F;github.com&#x2F;unslothai&#x2F;unsloth&#x2F;tree&#x2F;main&#x2F;unsloth&#x2F;kernels">https:&#x2F;&#x2F;github.com&#x2F;unslothai&#x2F;unsloth&#x2F;tree&#x2F;main&#x2F;unsloth&#x2F;kerne...</a>

CUDA + ThunderKittens 4.5 hour tutorial<p><a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=xcpEl0cGCC4" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=xcpEl0cGCC4</a>

How easy is it to run on older GPUs (think 1080Tis)? The reason I ask this is because torch.compile refuses to support that, and that alone makes things much slower.

I’m working on an inference platform that allows for tokens to be appended to the context after some tokens have been generated. If there’s other sequences in the batch, it means they’ll have to be padded. Currently this means I can’t use FlashAttention because it doesn’t support arbitrary masks&#x2F;padding masks… can ThunderKittens help me?

so, these are hand optimized primitives for specific model of nvidia gpus? do you still have to make launch&#x2F;scheduling decisions to maximize occupancy? how does this approach scale to other target devices with specialized instruction sets and different architecture?

&quot;Coming soon -- ThunderKittens on AMD hardware!&quot;<p>Any update on this?

hi! We&#x27;re the devs - we&#x27;re planning the livestream for 1pm and we&#x27;ll post the link here, twitter, and in the discord tonight

I hate to be that guy, but Metal support?

I dont want to use the Platform Formerly Known as Twitter, but does anyone have a way to get the link to their livestream tomorrow?