How much did AlphaGo Zero cost? (2018)

Alpha Go Zero inspired the development of an open source version, Leela Go Zero which Leela Chess Zero is forked from by the same guy who made Stock Fish.<p>Lots of people contribute what I imagine are amounts of CPU Power&#x2F;money to the Leela Chess Zero project[1].<p>Would love to see Alpha Chess vs Leela Chess.<p>[1] <a href="https:&#x2F;&#x2F;training.lczero.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;training.lczero.org&#x2F;</a><p>[edit] I&#x27;ve caused terrible confusion by melding Leela Go and Leela Chess when Leela Chess was originally forked from Leela Go and that&#x27;s basically when similarities end.<p>Edited for a bit more clarity.

Are they using the on-demand price instead of the preemptible price? It seems like the sort of job that can run on preemptible machines, just because it&#x27;s a batch job. Also, should the cost really be calculated using public market prices at all, as opposed to the running costs of the TPUs? It is not guaranteed at all that the opportunity cost to Google of using all those TPUs is equal to the price that you or I would pay Google to use them. I understand it cost a lot, but I&#x27;m not convinced by the headline figure of $36M.

Achieving a new breakthrough in computing is often very expensive. Deep Blue is estimated to cost IBM over $100 million over a decade [1].<p>And in comparison to large tech company R&amp;D budgets, the amount cited in the article is a drop in the bucket. Consider the fact that Google spent $26 billion in R&amp;D budget in 2019 alone [2]. Microsoft spent almost $17 billion [3].<p>[1] <a href="https:&#x2F;&#x2F;www.extremetech.com&#x2F;computing&#x2F;76552-project-deep-blitz-chess-pc-takes-on-deep-blue&#x2F;2" rel="nofollow">https:&#x2F;&#x2F;www.extremetech.com&#x2F;computing&#x2F;76552-project-deep-bli...</a><p>[2] <a href="https:&#x2F;&#x2F;www.statista.com&#x2F;statistics&#x2F;507858&#x2F;alphabet-google-rd-costs&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.statista.com&#x2F;statistics&#x2F;507858&#x2F;alphabet-google-r...</a><p>[3] <a href="https:&#x2F;&#x2F;www.statista.com&#x2F;statistics&#x2F;267806&#x2F;expenditure-on-research-and-development-by-the-microsoft-corporation&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.statista.com&#x2F;statistics&#x2F;267806&#x2F;expenditure-on-re...</a>

Another way of thinking about how efficient the brain is: By the article’s numbers, about 5.5 million TPU hours were required to train the machine to play as well as a Go champion.<p>A Go champion might have trained for 8 hours a day, for 15 years (age 5 to 20). That is about 40 000 hours.<p>In other words, machines required 137 times longer to learn the game, and at twice the power consumption! There is still a lot of room for improvement.

I might be wrong, but I think this cost calculation is way off:<p>Their running cost estimate of a single TPU in a machine with 4 &quot;TPUs&quot; is based off the price of a cloud TPU v2-8, but a v2-8 is actually 4 ASICS on 1 board.<p>Also, because of the date of publication being around the time v2s were announced, and the fact that the TPU is only used for inference and GPU is used for training, I think self play was likely done on TPU v1s, which use 5x less power per ASIC and so are likely much cheaper<p>I also think the way they calculated the number of TPUs required is wrong, it looks like they assume 1 machine with 4 TPUs makes 1 move in 0.4 seconds, but since making 1 move only requires a forwards pass through a moderately sized CNN with 19x19(tiny) input, 1 TPU should be able to make thousands of moves in parallel per second.

Alpha Go Zero*, which was trained from scratch, without human games.<p>I&#x27;ve also heard rumors that AlphaStar (<a href="https:&#x2F;&#x2F;deepmind.com&#x2F;blog&#x2F;article&#x2F;alphastar-mastering-real-time-strategy-game-starcraft-ii" rel="nofollow">https:&#x2F;&#x2F;deepmind.com&#x2F;blog&#x2F;article&#x2F;alphastar-mastering-real-t...</a>) was essentially put on hold because it was too expensive to improve&#x2F;train. The bot wasn&#x27;t able to beat StarCraft champions and _only_ got to a grandmaster level.

The amount was removed from the submission title, which sucks if you&#x27;re like me and don&#x27;t like to visit yet another possibly JS-heavy site and drain your battery.<p>For others: It&#x27;s $36M.

I&#x27;ll quibble with a little bit of this.<p>&quot;AlphaGo Zero showed the world that it is possible to build systems to teach themselves to do complicated tasks.&quot;<p>It didn&#x27;t do any such thing. The game of go has a huge number of potential moves and outcomes, but the rules themselves are trivial, the board position can be measured in a handful of bytes and gameplay always and only progresses in one direction. And judging a good vs bad outcome is just a matter of comparing two numbers.<p>Go is challenging and interesting for humans, but it&#x27;s not remotely as &quot;complicated&quot; as driving a car or translating a language.

It’s a shame that this ‘Next big thing’ is the complete opposite of the internet. Instead of opening up the world for anyone to create things, letting small companies compete with large, it is only going to concentrate power with the richest companies and leave small companies unable to get involved.

I wonder if the code and network weights will ever see the light of day. I wonder what the eventual value proposition of working on this sort of stuff is. I suppose they are just going to try to apply the algorithms to better things.<p>I&#x27;ve been interested in the application of AlphaZero to chess. It&#x27;s sad that this many resources were devoted to something which we can&#x27;t even use to play chess as of now. Leela (the open source reengineer) is really strong, but the crushing results presented in the AlphaZero paper never materialized. And this article just shows how hard they are to replicate.

&gt; Each move during self-play uses about 0.4 seconds of computer thinking time.<p>&gt; Over 72 hours, 4.9 million matches were played.<p>One of this claim must be incorrect or misinterpreted, I highly doubt they used so many TPU&#x27;s as the article claims. That would be not only impractical but also it would raise a lot of other issues like networking, disk speed... etc...<p>My statement is not against this article, if anyone can confirm they used so many TPUs in parallel feel free to post it

The (synchronous) 0.4s per move number is misleading (and wrong), that&#x27;s not what the paper is saying. The &quot;footnote 1&quot; of the article is wrong.

Our main compute doesn&#x27;t go towards machine learning, but we do rely heavily on GPU power. I recently had to come up with the figures for us to invest in an expansion of our compute power, and it turned out that buying the machines ourselves would be cheaper than renting them from Google in 3-4 months.<p>We don&#x27;t run on those fancy V100 cards though, just regular old gaming cards suffice, and I suppose if we bought the &quot;industrial&quot; nvidia versions it would a take a bit longer to recoup, but still definitely within the year.<p>Anyway what I&#x27;m saying is that it&#x27;s probably possible to to this a lot cheaper than 36M, though maybe not in such a short time. Our startup is extremely cash intensive, and I bet machine learning companies are as well (I suppose machine learning experts aren&#x27;t cheap ;)), so if we can put in some work and safe a big portion off our hardware costs that really goes the distance.

Now, consider that this is the cost of the <i>final</i> model reported in the paper. This doesn&#x27;t account for all the iterations of trying out e.g. different model architectures, hyperparameter sweeps, etc. The true cost of the experimentation is likely at least an order of magnitude higher.

Does Sarbanes Oxley apply to zero rating ML costs? Alpha go might have unfair kyu ranking, if Google don&#x27;t have to &quot;pay&quot; to acquire rank. (95% joking)

&gt; The power consumption of the experiment is equivalent to 12,760 human brains running continuously.<p>Given the experiment lasts for just days, this actually sounds pretty impressive I think.<p>Many humans studied the game for a big portion of their lives in order to get Go knowledge where it is.

I&#x27;d like to see an AI play Monopoly (the board game) against CEOs of large companies.

I wonder if there&#x27;s some kind of software that takes an advantage of an AI to teach non-beginner players Go. E.g. you could play against the bot and then the AI would translate your mistakes into what you can improve upon.

This is a lot of money.<p>However, if you want to reliably make an AI the best in the world at a range of complicated tasks, can you reasonably expect this to be cheap?

Wondering when researchers will switch from &quot;race to the moon&quot; mode to looking at better optimization techniques instead of just throwing money at the problem.<p>I know some companies are doing that, but I think looking at AlphaGo or AGZ and making it go faster should be an interesting problem in itself.

The interesting part to me, rather than cost, is Energy usage.<p>&gt;The power consumption of the experiment is equivalent to 12,760 human brains running continuously.<p>But the problem is this &quot;brains&quot; unit on AlphaZero doesn&#x27;t seems to take into account of GPU, CPU and Memory involved. It only took the TPU numbers.<p>Then there is another problem.<p>&gt; a TPU consumes about 40 watts,[1]<p>The TPU referred to was a first Gen TPU built on 28nm running at 40W, more like a proof of concept. Currently Google is with Cloud TPU v3 [2], The latest-generation Cloud TPU v3 Pods are liquid-cooled for maximum performance. And each TPU v3 is actually a four chip module. [3]. If a single chip is 100W that is 400W per TPU.<p>Edit: Turns out Wiki list TPU v3 as 250W. [4]. Not sure if that is 250W per chip or 250W for 4 Chips.<p>That is on the assumption they are very high powered and hence would require liquid cooling. Although that might not always be the case.<p>So adding CPU, GPU, Memory, and TPU figures. That original estimate of 12,760 human brains may be off by a factor of 10 if not more.<p>Still pretty impressive. Considering we now only get about 1.8x improvement with each generation node. We would get about 19x by 2030. ( Assuming the same algorithm ). Which means AI is good, but human brain on its own is still very much magical in its efficiency :)<p>Correct me If I am wrong on the numbers.<p>My other questions is, that was how much energy it used to learn Go. But what about energy it used during the Game?<p>How would AlphaGo Zero perform if it was limited to 20W?<p>[1] <a href="https:&#x2F;&#x2F;cloud.google.com&#x2F;blog&#x2F;products&#x2F;gcp&#x2F;an-in-depth-look-at-googles-first-tensor-processing-unit-tpu" rel="nofollow">https:&#x2F;&#x2F;cloud.google.com&#x2F;blog&#x2F;products&#x2F;gcp&#x2F;an-in-depth-look-...</a><p>[2] <a href="https:&#x2F;&#x2F;cloud.google.com&#x2F;blog&#x2F;products&#x2F;ai-machine-learning&#x2F;googles-scalable-supercomputers-for-machine-learning-cloud-tpu-pods-are-now-publicly-available-in-beta" rel="nofollow">https:&#x2F;&#x2F;cloud.google.com&#x2F;blog&#x2F;products&#x2F;ai-machine-learning&#x2F;g...</a><p>[3] <a href="https:&#x2F;&#x2F;techcrunch.com&#x2F;2019&#x2F;05&#x2F;07&#x2F;googles-newest-cloud-tpu-pods-feature-over-1000-tpus&#x2F;" rel="nofollow">https:&#x2F;&#x2F;techcrunch.com&#x2F;2019&#x2F;05&#x2F;07&#x2F;googles-newest-cloud-tpu-p...</a><p>[4] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Tensor_processing_unit" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Tensor_processing_unit</a>

I would be more interested in how much LCZero cost?

the article doesn&#x27;t seem to consider cost of hyperparameter optimization prior to the final training...

Ahem, &quot;one time cost&quot;

A: $400M—to acquire DeepMind

You need 35$m to beat the best Stockfish engine which can work on a small computer. Who won?

It is estimated to be 36 million for <i>someone else</i> to train AlphaGo Zero, assuming they use Google TPU instances and pay the sticker price.<p>Google isn&#x27;t operating with that cost, unless we assume that they are prioritizing AlphaGo to the point where they lose such customers 100% of the time.<p>It&#x27;s way more likely that AlphaGo is trained on spare time, the cost for the hardware is sunk anyway, so only the cost for upkeep is real.

&gt; This accomplishment is truly remarkable in that it shows that we can develop systems that teach themselves to do non-trivial tasks from a blank slate, and eventually become better than humans at doing the task.<p>&quot;non-trivial&quot; is a bit of a red herring here. Playing go is pretty trivial compared to something like walking or scratching your face. Winning go may be non-trivial compared to those in some ways but it is very trivial in comparison in other ways.

sorry if it is off topic but I want to learn alpha zero from beginning , I do have little understanding of Machine &amp; deep learning including vision recognition. Unfortunately I don&#x27;t able to understand how monto Carlo tree is used for decision making. where I can start, what shall I learn so that I can learn alpha go (or OpenAI Five - Dota 2 bit).<p>thanks

Misleading numbers, and wrong calculations. The TPU and CPU cost them almost nothing as they use and build them anyway, and renting them out for this PR stunt just cost them the missed rental time, if customers would really pay that much. Maybe around 20.000. Energy cost? I don&#x27;t see much additional costs as those machines run all the time, regardless if improving the model or doing something else.<p>I bet the much higher cost was the PR team, including the film team, press support, TV team, travels, inviting the expert Go players, building the stage, and such. Estimated 100.000.<p>Not counting the man hours, they were just doing their normal job.