AMD AI Software Solved – MI300X Pricing, Perf, PyTorch, FlashAttention, Triton

Honestly, the likes of OpenAI and Mosaic need to consider Nvidia a huge threat long term.<p>Nvidia has shown time and time again, they will royally fuck over anyone they have to in order to drive profit.<p>Then, if they dare talk negatively about them, they will just discontinue their access to hardware.<p>Not saying AMD is a savior, but having only ONE option will lead to long term issues.

Somewhat related: starting a new Ubuntu-pytorch-cuda project? That’ll be 10-15 gigabytes, please. Is there some way to strip down the individual deps? I imagine it’s the way that it is because the drivers and the pytorch source are probably machine generated before compilation, to some extent. Is there any hope for triton becoming a lightweight interstitial frame in the codebase stack that can just allow me to do the hw codegen <i>after</i> pulling, without having to rebuild completely everything, and also simultaneously allowing more accelerators?

Is ROCm actually usable in this years machine learning ecosystem? Can I just drop in any PyTorch model that was developed on CUDA and expect it to work?

A recent latent.space podcast with geohot discusses a lot of related information; his approach to using AMD and the challenges of getting ML to work on anything other than NVidia.<p><a href="https:&#x2F;&#x2F;www.latent.space&#x2F;p&#x2F;geohot#details" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.latent.space&#x2F;p&#x2F;geohot#details</a>

I&#x27;ve been waiting years and will wait more for PyTorch etc to work under Mesa rather than install RocM

Is this real? is ROCm not horrible to use now? Anybody using it rn, if so what card and how is it working?