Apple M1 support for TensorFlow 2.5 pluggable device API

M1 and AMD GPU support. I&#x27;m personally more interested in the latter as I haven&#x27;t yet upgraded my MacBook Pro and I expect that my Vega 20 to be faster than M1 at ML training.<p>The raw compute power of M1&#x27;s GPU seems to be 2.6 TFLOPS (single precision) vs 3.2 TFLOPS for Vega 20. This can give you an estimate of how fast it would be for training.<p>Just for reference Nvidia&#x27;s flagship desktop GPU(3090)&#x27;s FP32 performance is 35.5 TFLOPS.

I have found the M1 air fine for web browsing but kind of hard to install software on.<p>Following the instructions: -----<p><pre><code> python -m pip install tensorflow-macos ... ERROR: Failed building wheel for numpy Failed to build numpy ERROR: Could not build wheels for numpy which use PEP 517 and cannot be installed directly</code></pre> -----<p>(base) dave@daves-air ~ % uname -a<p><pre><code> Darwin daves-air.lan 20.5.0 Darwin Kernel Version 20.5.0: Sat May 8 05:10:31 PDT 2021; root:xnu-7195.121.3~9&#x2F;RELEASE_ARM64_T8101 arm64</code></pre>

I&#x27;m still trying to find a way to monitor the Neural Engine on my Macbook air M1, but the APIs are non-existent, there&#x27;s barely anything in the docs and no answer from Apple. My models train fast, 3x faster than most i7 computers with GPU, which is excellent for a fanless ultraportable computer but I wish Apple would treat the NE as a 1st class citizen on these machines, with Mac SDK APIs and usage visualization in the Activity Monitor.

I was able to install this fairly easily (much more so then the crap they dumped out here - <a href="https:&#x2F;&#x2F;github.com&#x2F;apple&#x2F;tensorflow_macos" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;apple&#x2F;tensorflow_macos</a>. Just take a look at the 200 github issues that were ignored for the most part...)<p>I also noticed that in my project I got a decent speedup immediately when executing my model, but I have not run any benchmarks.<p>But, where do you go to file bugs? Ask questions? etc. I am not a big Mac developer, so is there something I don&#x27;t know?

Anyone know if this shows up as an actual GPU device? The last tensorflow-macOS thing did not. If you list devices using that, you’ll see only one: CPU:0.<p>Does this give a GPU:0 device? You can check via:<p><pre><code> import tensorflow as tf2 from pprint import pprint as pp tf = tf2.compat.v1 sess = tf.InteractiveSession() pp(sess.list_devices()) </code></pre> I’d check myself, but I’ve been so burnt by tensorflow 2 and M1 problems that I just don’t have the energy to figure out the inevitable compilation issues, and it sounds like at least one other person already has it running. Plus I’m on mobile.

<a href="https:&#x2F;&#x2F;github.com&#x2F;tensorflow&#x2F;tensorflow&#x2F;releases&#x2F;tag&#x2F;v2.5.0" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;tensorflow&#x2F;tensorflow&#x2F;releases&#x2F;tag&#x2F;v2.5.0</a> (Linked from Apple&#x27;s article)<p>Wow, that list of CVEs is 110 lines.

I have some notes on doing ML&#x2F;python work on M1 Macs that may be helpful. <a href="https:&#x2F;&#x2F;twitter.com&#x2F;troyharvey&#x2F;status&#x2F;1380586300911280128?s=21" rel="nofollow">https:&#x2F;&#x2F;twitter.com&#x2F;troyharvey&#x2F;status&#x2F;1380586300911280128?s=...</a>

Very cool. I’m not an ML guy, so apologies if this is a dumb question: how good is performance compared to other hardware here?

Installed these - which seemed to work bar a few messages about `numpy` not being installable - but trying to use `textgenrnn` ran into a whole bunch of Keras problems (for which the internet&#x27;s answer is &quot;use tensorflow.keras&quot; expect `textgenrnn` is already doing that...)<p>I yearn for the day when someone makes a nice, simple, &quot;install this and python ML works fine with your GPU&quot; package.

I find the benchmarks confusing. If we normalize, is Apple close to beating Nvidia?

Can&#x27;t wait to get metal support in PyTorch!

This is pretty cool to have more generic GPU-based support.<p>Might also be related: Tensorflow Lite Core ML delegate enables running TFLite model on CoreML enabled iOS devices.<p><a href="https:&#x2F;&#x2F;www.tensorflow.org&#x2F;lite&#x2F;performance&#x2F;coreml_delegate" rel="nofollow">https:&#x2F;&#x2F;www.tensorflow.org&#x2F;lite&#x2F;performance&#x2F;coreml_delegate</a><p>Would nice to see performance comparisons on M1 Mac&#x2F;iPad for which way is more performance and efficient. (Admittedly TF vs TF-lite is a 100% apple to apple comparison, pun-intended).

The previous beta 2.4 branch would also utilize AMX coprocessor when appropriate. Does anybody know whether the plugin is also based on ML Compute?

finally

Install conda then use pip ... does not make much sense. Conda is so clunky.