Nvidia Announces A100 80GB GPU for AI

While these are amazing performance numbers, I seriously wonder if all this expensive AI magic will be cost effective in the end.<p>Their lead example is recommendation systems, but I can&#x27;t say that I have received many good suggestions recently.<p>Spotify and Deezer both suggest the chart hits, regardless of how often I dislike that kind of music.<p>Amazon keeps recommending me tampons (I&#x27;m a guy) ever since I&#x27;ve had a coworking office with a female colleague in 2015.<p>For all the data that they collect and all the AI that they pay for, these companies get very little revenue to show for it.

I am more wondering why hasn&#x27;t AMD massively invested into porting common ML frameworks to OpenCL. Nvidia has outrageous margins on their datacenter GPUs. They&#x27;ve even banned the use of lower-margin gamer-oriented GPU in datacenters [0]. Given that tensor arithmetic is essentially an easily abstractable commodity, I just don&#x27;t understand why they don&#x27;t offer a drop-in replacement.<p>Most users won&#x27;t care what hardware their PyTorch model runs on in the cloud. All that matters for them is dollars per training epoch (or cents per inference). This could be a steal for an alternate hardware vendor.<p>[0] <a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20201109023551&#x2F;https:&#x2F;&#x2F;www.digitaltrends.com&#x2F;computing&#x2F;nvidia-bans-consumer-gpus-in-data-centers&#x2F;" rel="nofollow">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20201109023551&#x2F;https:&#x2F;&#x2F;www.digit...</a>

It occurs to me that if anyone is going to release an ARM based CPU that is competitive with Apple, it&#x27;s Nvidia. A Microsoft&#x2F; Nvidia partnership could create some pretty impressive Surface laptops, and if Nvidia were allowed to sell those CPUs to OEMs for other 2 in 1s or laptops, Microsoft might just get some traction on their ARM efforts.

Does processing power translate to actual power? Will most stock market gains in the end go to the people with the fastest computers? Will wars be won by the groups with the most processing speed? Was Cyberpunk (slightly) wrong and it&#x27;s just about having more memory and more instructions per millisecond than the rest? Are sophisticated circuits the new oil?

&gt;&quot;The new A100 with HBM2e technology doubles the A100 40GB GPU’s high-bandwidth memory to 80GB and delivers<p><i>over 2 terabytes per second of memory bandwidth</i>.&quot;

The workstation they announced in the press release [0] sounds and looks incredible. I&#x27;m sure it costs over $100k, but I wish that kind of case design and cooling system could be available at a more reasonable price point. I wonder how long it will take for a computer with the performance specs of the DGX Station A100 to be available for under $3,000. Will that take 5 years? 10 years? Based on the historical trends of the last decade, those estimates strike me as pretty optimistic.<p>[0] <a href="https:&#x2F;&#x2F;www.nvidia.com&#x2F;en-us&#x2F;data-center&#x2F;dgx-station-a100&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.nvidia.com&#x2F;en-us&#x2F;data-center&#x2F;dgx-station-a100&#x2F;</a>

Why do they still produce GPU rather than specialized ASIC for neural networks like Google does with their Tensor Processing Units?

Legit question: Why are these still called &quot;GPU&quot;s? Shouldn&#x27;t they rightly be called &quot;AIPU&quot;s, or &quot;IPU&quot;s?

It is amusing to see this so soon after a post on how workstations were dead : <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=24977652" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=24977652</a><p>$US 200K for startling performance this time.

Any word on how expensive this board will be?

For only the low low cost of everything in your bank account!