Neural Ordinary Differential Equations

Senior author here, I&#x27;m happy to answer any questions.<p>We just released source code: <a href="https:&#x2F;&#x2F;github.com&#x2F;rtqichen&#x2F;torchdiffeq" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;rtqichen&#x2F;torchdiffeq</a> . This includes PyTorch implementations of adaptive ODE solvers that can be differentiated through automatically. So you can mix and match these ODE solvers with any other differentiable model component.<p>There&#x27;s already been a bit of follow-up work, turning Continuous Normalizing Flows into a practical generative density model: <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1810.01367" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1810.01367</a><p>And now we&#x27;re mainly working on 1) Regularizing ODE nets to be faster to solve and 2) getting the time-series model to scale up and extend it to stochastic differential equations.

Wow, I need to master ODEs&#x2F;PDEs to keep up with Deep Learning now! Seems like one has to be a master of statistics, operations research, calculus and algorithms to push it forward!<p>Comparison to RNN was impressive! Any well-known real-world models for comparison to state of art?

Some reporting in MIT&#x27;s newsletter on AI: <a href="https:&#x2F;&#x2F;mailchi.mp&#x2F;technologyreview&#x2F;a-new-type-of-deep-neural-network-that-has-no-layers" rel="nofollow">https:&#x2F;&#x2F;mailchi.mp&#x2F;technologyreview&#x2F;a-new-type-of-deep-neura...</a>

I remember something similar for CFD application but haven&#x27;t seen much after that. It would be awesome if we can build a cheap and fast Navier Stokes solver with neural networks.

Hi, I am very interesting about your models. When you do the back propagation, it seems that it still needs complex calculation. Although O(1) memory cost is an important contribution, do you think record some of the intermediate value will significantly boost the training?