Category theory always reminds me of this quip by Feynman:<p><a href="https://www.youtube.com/watch?v=B-eh2SD54fM" rel="nofollow">https://www.youtube.com/watch?v=B-eh2SD54fM</a><p>When I hear people talking in the jargon of category theory, I do not understand what they say, but I have a suspicion that it is something rather mundane that I <i>would</i> understand if they were using the more specific terms for the given context. I understand the appeal of generalization from a mathematical perspective, but from a practical programming perspective, I fail to understand the value proposition.
So, learning about the categorical structure is interesting, but is there a specific advantage to seeing these concepts directly informing the implementation vs as a post-hoc way of explaining what autodiff is doing? E.g. Tensorflow is creating and transforming graphs of computation. Despite being written before most of the categorical work cited was done, isn't it doing the "same" thing, but we just wouldn't find names or comments in the code that closely align with the categorical work?
This is a nice project, but “You only linearize once” is more-or-less the type theory version of “Reverse Derivative Categories”, so JAX really does this already.
I think about cones all the time when doing machine learning. If I have an object O that can be mapped to A and B I can learn a function from A to B or B to A if I can generate Os. That’s my view of self supervised learning.
It remembers me Theano [0].<p>[0] <a href="https://en.wikipedia.org/wiki/Theano_(software)" rel="nofollow">https://en.wikipedia.org/wiki/Theano_(software)</a>