Zeta: Functional Neural Networks in Ocaml

A quite bright bachelor&#x27;s student of mine did something similar for his undergraduate thesis[0]. Conceptually, he represented neural networks as functions from input to output, and connecting two networks required that the output of the first matched the input of the second, and produced a new network. Now, to enable gradient descent or other kinds of optimisation, the networks were not <i>literally</i> functions, but rather pairs of functions for running the network forwards and backwards. While limited in some ways (e.g. no recurrent networks), it was quite convenient, and close in performance to TensorFlow on a single GPU.<p>[0]: <a href="https:&#x2F;&#x2F;futhark-lang.org&#x2F;student-projects&#x2F;duc-bsc-thesis.pdf" rel="nofollow">https:&#x2F;&#x2F;futhark-lang.org&#x2F;student-projects&#x2F;duc-bsc-thesis.pdf</a>

I had been messing around with a similar concept not too long ago in OCaml- as it turns out, strong, static typing can be very nice to have in writing ML models (and code in general). I find it reduces some of the cognitive overhead that comes from having to keep track of how the functions interact in the program (or perhaps, it reduces&#x2F;clarifies the <i>potential</i> ways the functions can interact).<p>I&#x27;ll be very interested to see where this project goes.

ML in ML?<p>Jokes aside, see:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;LaurentMazare&#x2F;tensorflow-ocaml" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;LaurentMazare&#x2F;tensorflow-ocaml</a><p>Also<p><a href="http:&#x2F;&#x2F;ryanrhymes.blogspot.com&#x2F;2017&#x2F;03&#x2F;build-neural-network-from-scratch-in-60.html" rel="nofollow">http:&#x2F;&#x2F;ryanrhymes.blogspot.com&#x2F;2017&#x2F;03&#x2F;build-neural-network-...</a><p>for how you can very quickly write something from scratch with &quot;Ocaml&#x27;s numpy&quot;, owl.

I&#x27;m also working on something like this with F# and TensorFlow:<p><a href="https:&#x2F;&#x2F;youtu.be&#x2F;GUMTpTqzsJA" rel="nofollow">https:&#x2F;&#x2F;youtu.be&#x2F;GUMTpTqzsJA</a><p><a href="https:&#x2F;&#x2F;devpost.com&#x2F;software&#x2F;sylvester-tf" rel="nofollow">https:&#x2F;&#x2F;devpost.com&#x2F;software&#x2F;sylvester-tf</a><p>using typed natural number dimensions for arrays, vectors, matrices, tensors et.al

It&#x27;s great to hear that people are trying to use ocaml for ML :) To me it seems like owl is the most mature ocaml project in this area. I recall they even had some GPU support. Unfortunately, I didn&#x27;t have a chance to try it out so I can&#x27;t comment first hand on how &quot;ready&quot; it is.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;owlbarn&#x2F;owl" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;owlbarn&#x2F;owl</a>