This is not a very good list. Several items are between "misleading" and "wrong":<p>1. "Pipe" should be called "bar" (IMO)<p>2. "Vector norm" is a much deeper idea than just "2-norm" and if you limit yourself to the 2-norm, you will run into problems<p>3. "Set membership" is not called "epsilon"!<p>4. I wouldn't describe functions as operating on "pools", but that's just me. ("Pools" seems to imply vector arguments, or at least set arguments, which is not how functions are usually thought of operating, though they are defined that way.)<p>5. The description of R^2 as "2-D array" is wrong.<p>6. I've never seen that notation for elementwise multiplication, though that doesn't mean much.<p>7. I've rarely seen that notation for dot products. Usually the center dot is much more common.<p>8. Hat is... nonstandard. It has many, many meanings.
It is faulty to think of the math notation and code being equivalent. The math states something much more general, and often much deeper, than some mundane for-loop. I’d have titled this “Python Recipes to Calculate Values of Simple Vector Math Expressions”.<p>Often times the symbols may not even be concrete numbers. Often the “real” number may not even be representable on a computer. Often a symbol <i>does</i> represent a concrete number but indirectly defined by some set of rules (e.g., x is the smallest eigenvalue of M; or y is the greatest value less than the supremum of f) that may require a whole library of code to calculate.<p>The mathematical notation is “meant” to be flexible and manipulated, not to be interpreted (necessarily) as a rigid computational process. It should also be noted that while some notation comes from convention, a lot of it is improvised and contextual!
I do wonder, at there people writing ML programs in Python without learning basic math notation?<p>It's hard to imagine someone learning important fundamt math concepts without using math notation, so anyone who benefits from this article should probably take a detour through a math book before continuing their Python ML, or else risk having beautiful code that implements nonsensical math.
The article explains \hat{} as meaning a vector of unit length but in my experience in ML it nearly always designates an estimate (including in the first expression in this article).
It’s interesting where the “technical limitations” set in on computers. We can’t have symbolic math accurately represented because “how would you even do that with keyboard input, and how would you encode it?!” Yet we have poop emojis because that’s an absolute essential part of written communication these days.
I see many comments criticizing the post for only implementing simple mathematics. This is true.<p>However, the approach of understanding math through code is still very helpful, I think. Personally, implementing things that are fuzzy mathematically provides immense clarity once I write them down.<p>For example, a simple monty hall simulator[1]. Or implementing matrix multiplication multiple ways to understand why each is equivalent[2], and why multiplying A(BC) can sometimes be faster than (AB)C[3].<p>I am not sure why this helps me. It may be because I was "raised" as a coder, and so that is how my brain works. But I also think that implementing something in code is very close to constructivist mathematics, in spirit. You cannot prove anything if you cannot construct (/implement) it.<p>[1] <a href="https://github.com/mitchellgordon95/implementing-paradoxes/blob/master/monty_hall.py" rel="nofollow">https://github.com/mitchellgordon95/implementing-paradoxes/b...</a>
[2] <a href="https://github.com/mitchellgordon95/implementing-paradoxes/blob/master/matmul.py" rel="nofollow">https://github.com/mitchellgordon95/implementing-paradoxes/b...</a>
[3] <a href="https://github.com/mitchellgordon95/implementing-paradoxes/blob/master/matmul_assoc.py" rel="nofollow">https://github.com/mitchellgordon95/implementing-paradoxes/b...</a>
If you liked this...<p><a href="https://mitpress.mit.edu/sites/default/files/titles/content/sicm_edition_2/book.html" rel="nofollow">https://mitpress.mit.edu/sites/default/files/titles/content/...</a><p>"Structure and Interpretation of Classical Mechanics" by Gerald Jay Sussman and Jack Wisdom<p>> There has been a remarkable revival of interest in classical mechanics in recent years. We now know that there is much more to classical mechanics than previously suspected. The behavior of classical systems is surprisingly rich; derivation of the equations of motion, the focus of traditional presentations of mechanics, is just the beginning. Classical systems display a complicated array of phenomena such as nonlinear resonances, chaotic behavior, and transitions to chaos.<p>> Traditional treatments of mechanics concentrate most of their effort on the extremely small class of symbolically tractable dynamical systems. We concentrate on developing general methods for studying the behavior of systems, whether or not they have a symbolic solution. Typical systems exhibit behavior that is qualitatively different from the solvable systems and surprisingly complicated. We focus on the phenomena of motion, and we make extensive use of computer simulation to explore this motion.<p>They are basically making the computer do the work, with emphasis on unambiguous, computable notation.<p>(It could be considered a companion to SICP.)
Average of a <i>finite</i> series: There's a statistics module in Python 3.4+:<p><pre><code> X = [1, 2, 3]
from statistics import mean, fmean
mean(X)
# may or may not be preferable to
sum(X) / len(X)
</code></pre>
<a href="https://docs.python.org/3/library/statistics.html#statistics.fmean" rel="nofollow">https://docs.python.org/3/library/statistics.html#statistics...</a><p>Product of a terminating iterable:<p><pre><code> import operator
from functools import reduce
# from itertools import accumulate
reduce(operator.mul, X)
</code></pre>
Vector norm:<p><pre><code> from numpy import linalg as LA
LA.norm(X)
</code></pre>
<a href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.linalg.norm.html" rel="nofollow">https://docs.scipy.org/doc/numpy/reference/generated/numpy.l...</a><p>Function domains and ranges can be specified and checked at compile-time with type annotations or at runtime with type()/isinstance() or with something like pycontracts or icontracts for checking preconditions and postconditions.<p>Dot product:<p><pre><code> Y = [4, 5, 6]
np.dot(X, Y)
</code></pre>
<a href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.dot.html" rel="nofollow">https://docs.scipy.org/doc/numpy/reference/generated/numpy.d...</a><p>Unit vector:<p><pre><code> X / np.linalg.norm(X)</code></pre>
Seeing `for i in range(len(lst)): lst[i]...` gives me hives. That's cool if you're wanting to be super explicit about how the indexing works, but in this page it goes on to say "or you can just write sum(lst)" without worrying about the indexing.<p>I would write the explanations like:<p><pre><code> result = 1
x = [1, 2, 3, 4, 5]
for number in x:
result = result * number
print(result)
</code></pre>
which in my opinion is much closer to the way a mathematician would think about the process.
Maths notation can be wonderfully concise and precise, so it worth thinking about following it closely when programming. One of my favorite examples of this is the numpy `einsum` call [1]. It implements Einstein summation convention [2] - thereby making working with the many dimensions of high-rank tensors feasible.<p>E.g. this (Latex):<p>$C_{ml} = A_{ijkl} B_{ijkm}$<p>becomes (in Python):<p>C = einsum('ijkl,ijkm->ml', A, B)<p>[1] <a href="https://docs.scipy.org/doc/numpy/reference/generated/numpy.einsum.html" rel="nofollow">https://docs.scipy.org/doc/numpy/reference/generated/numpy.e...</a>
[2] <a href="https://en.wikipedia.org/wiki/Einstein_notation" rel="nofollow">https://en.wikipedia.org/wiki/Einstein_notation</a>
Solid post. I think someone who's just starting out with ML or even basic mathematically notations would find this very helpful. Especially someone knows programming, but struggles with mathematically jargon
For arrays, the math uses ordinal convention (1-based) but the Python uses offset convention (0-based). That's OK, but the article should explicitly mention that to avoid confusion.
Shameless plug, but I worked for some time on "Thinking in Tensors, Writing in PyTorch" pretty much focusing on turning LaTeX math into executable code.<p>E.g. Gradient Descent: formula, code, and visualization:<p><a href="https://github.com/stared/thinking-in-tensors-writing-in-pytorch/blob/master/2%20Gradient%20Descent.ipynb" rel="nofollow">https://github.com/stared/thinking-in-tensors-writing-in-pyt...</a>
I think it is confusing not to address that math notation often (including the one used here) uses 1-based indexing, while Python uses 0-based indexing.
Given that FORTRAN was designed as basically rote translations from discrete math notation, it is deeply ironic that we now need to explain things in the opposite direction.