Nvidia DGX GH200 Whitepaper

Why is this called a whitepaper, as this is more of a documentation and architecture overview of the cluster? Wow a CLOS topology for networking, very innovative.<p>Details on NVLink would be great. For example, the needs and problems solved by their custom cables seemingly required by NVLink would be worth a whitepaper.<p>Don&#x27;t get me wrong, this is still great the general public can get a glimpse into Grace Hopper. And they do a good job of simplifying while throwing around mind-boggling numbers (the NVLink bandwidth is insane, though no words on latency, crucial for remote memory access).

What&#x27;s funny is that even though the DGX GH200 is some of the most powerful hardware available, there&#x27;s such a voracious demand that it&#x27;s not gonna be enough to quench it. In fact, this is one of those cases where I think the demand will always outpace supply. Exciting stuff ahead.<p>I heard Elon say something interesting during the discussion&#x2F;launch of xAI: &quot;My prediction is that we will go from an extreme silicon shortage today, to probably a voltage-transformer shortage in about year, and then an electricity shortage in about a year, two years.&quot;<p>I&#x27;m not sure about the timeline, but it&#x27;s an intriguing idea that soon the rate limiting resource will be electricity. I wonder how true that is and if we&#x27;re prepared for that.

The memory and bandwidth numbers are mind blowing. Going to be very hard to catch Nvidia. It’s as if competitors are going through the motions for participation prizes.

I wonder how much this thing will cost, best I&#x27;ve been able to find so far is a &#x27;low 8 digits&#x27; estimate in Anandtech article but nothing more specific than that.<p><a href="https:&#x2F;&#x2F;www.anandtech.com&#x2F;show&#x2F;18877&#x2F;nvidia-grace-hopper-has-entered-full-production-announcing-dgx-gh200-ai-supercomputer" rel="nofollow noreferrer">https:&#x2F;&#x2F;www.anandtech.com&#x2F;show&#x2F;18877&#x2F;nvidia-grace-hopper-has...</a>

As context: 1x dgx gh200 has 256x gh200s which each have 1x h100 and 1x grace cpu

I would be interesting to know what kind of next-gen models this can train.<p>On the LLM frontier, we’re starting to hit the limits of reasoning abilities in the current gen.

Unfortunate that they don&#x27;t mention the running times for any of the applications they benchmark (e.g., PageRank). Does anyone in the know have some idea how long this takes?

So basically 2x faster than H100