So, this is a bit of an opus -- the authors would definitely like to usher in a new paradigm for compute is my inexpert reading.<p>Overall the idea, if I understand it, is that rather than using linear activations between layers in a deep learning setup (so called MLP - Multi-Layer Perceptrons), you use functions ("splines") which can be learned using some sort of backprop at the nodes.<p>The idea (well, one of many) is that using more complex functions and making them learnable during training means you can encapsulate more complex functions in a lower dimensionality matrix.<p>Upsides: they claim smaller number of training steps and lower dimensionality for a number of toy problems. Also, since you're training functions and these functions might have periodicity to them, and that periodicity can be adjusted during training, and that periodicity might tie to sets of data with varying states, they claim you can update parts of the periodic functions for specific information without impacting the <i>whole</i> function, and therefore this is a better conceptual architecture for dealing with special cases, "forgetting" and so on.<p>Downsides: they mention it's much slower to train than an MLP architecture, although they claim they didn't try very hard to optimize.<p>My totally uninformed complaints: The networks are trained on REALLY simple problems, like approximating x*y. I mean this is a fairly beautifully written and illustrated 40+ page paper, with really quite a lot of LaTeX, and I (mostly) read the whole thing. It feels hard to read it without at least seeing, like, some MNIST training results.<p>Upshot - I'm not sure that we'll all be training Kan networks in 2024.