Galerkin Approximation

And, guess what? Since, the Galerkin approximation requires one to choose a basis that is appropriate to the problem at hand, we now have a deep learning solution too (since neural network learning is essentially equivalent to learning an adaptive basis).<p>It is called the Deep Galerkin Method [1]. In a nutshell, the method directly minimizes the L2 error over the PDE, boundary conditions and initial conditions. The integral is tricky though, and computed via a Monte Carlo approximation.<p>[1]: <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1708.07469" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1708.07469</a>

Check out the rest of the guy&#x27;s blog. I desperately wish there were more resources where complex mathematical concepts were first introduced without all of the proofs.

A tangent, but I was exposed to the Galerkin approximation when learning about the Finite Element Method, well over 10 years ago.<p>As part of the course I got introduced to the FEniCS project[1].<p>They had Python code looking very much like the math equations generating C++ code at runtime, compiling it into a Python module which got dynamically loaded and executed.<p>This way they got speeds which rivaled or surpassed handwritten C++, as the C++ code could be optimized around the specific problem, but with superior ergonomics of writing the equations almost directly.<p>It really blew my mind. I had heard about Java doing JIT but this was on another level for me. Not terribly fancy these days but at the time it really helped me expand my thinking about how to solve problems.<p>[1]: <a href="https:&#x2F;&#x2F;fenicsproject.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;fenicsproject.org&#x2F;</a>

I find the general idea of treating differential equations as infinitely dimensional linear systems quite powerful. I was first introduced to the concept while studying quantum mechanics, but applications are everywhere.

This made as little sense to me as it did when I was talking the Finite Element Methods class during graduate school.<p>Still don&#x27;t quite understand why you can&#x27;t just use Runge Kutta methods to numerically solve these problems. I became quite good at manipulating the symbols to derive variational solutions while having absolutely no idea what any of it meant.