About one year ago, I took some relatively expansive classes about linear algebra and analysis. They were pretty proof-heavy, and therefore forced me to think deeply about the concepts that were introduced. I even did pretty well and received a good grade (1.3).<p>Fast forward to the next semester, and I'm taking an introductory class in neural networks. This class, as you can imagine, requires some of the material introduced in those earlier classes.<p>But even though barely 3 months have passed, I struggled to remember how to do basic operations and had to relearn simple concepts. This was especially problematic while trying to understand backpropagation, since I constantly had to interrupt myself because I had to look something up again. But in the end, I managed to get it by sinking more time into it.<p>Fast forward to today, and I failed to remember how backpropagation works while taking another class about machine learning. And I'm not just talking about details, I can't even remember some parts of how it works abstractly.<p>This is extremely frustrating, because it feels like I'm wasting my time by learning something, only to forget it within a few months.<p>Is there anyone who struggled with the same thing and found some way to alleviate this issue?
If you work on systems of increasing complexity you are eventually going to overrun your memory capacity and the way to scale that up is to write it down and depend on that.<p>When it comes to programming languages, for instance, the first thing I teach myself is how to look up answers to questions like "how do I look up the length of a string?" in the official documentation. This is much more reliable than using Google or Bing to look up frequently wrong answers on Stackoverflow and programming splogs. This works really well for Java and Python, but not so well for Clojure (there is a book you've got to buy) and is also tricky for Javascript (MDN's documentation is close to an official manual)<p>Each time you "learn" something and incorporate it into your docs, and then work with it, it becomes a little more familiar and after a long time you really will master the material.<p>Another "hack" is to learn how to derive things yourself. For instance the other day I had to take some derivatives a little more complex than what I usually do and it's been 30 years since I took calculus, I remember some of the rules but I forgot some that I need and just took a few minutes to derive them from first principles.<p>In the case of something like the Fast Fourier Transform or backpropagation you should be able to derive the method from scratch knowing just a few facts about how it works.
> This is extremely frustrating, because it feels like I'm wasting my time by learning something, only to forget it within a few months.<p>I really identify with this feeling. For a time, this meant that I put the minimum effort into studies to get decent grades and completely stopped learning on my own. Thankfully, soon enough I found Anki [0] and everything changed.<p>Through using the app ~daily for over 7 years now, I have a much better idea of how my memory works. It doesn't mean that I never forget anything (even things in Anki will decay if not engaged with), but it makes memory more of a choice rather than something that will automatically aggressively fade.<p>I'd recommend trying it out for 2-3 weeks and seeing if you like it. Using a shared deck that someone has already put together for something that you're already interested in is a great way to start. For me, that was geography [1].<p>[0] <a href="https://apps.ankiweb.net/" rel="nofollow">https://apps.ankiweb.net/</a><p>[1] <a href="https://ankiweb.net/shared/info/2109889812" rel="nofollow">https://ankiweb.net/shared/info/2109889812</a>
It's been said, "Use it, or lose it" [0]<p>It's also been said, "You have to know it to write it or teach it" [1]<p>If you're not <i>staying</i> in the material between classes, you'll have to "relearn" at least parts of it<p>I got great grades in financial accounting 2+ decades ago<p>Doesn't mean I can <i>do</i> financial accounting today (heck! Probably couldn't "do" it even a semester later)<p>I did fantastically in an intro to COBOL class in 2000. Haven't so much as glanced as COBOL since (to the best of my knowledge)<p>You'd got to keep <i>using</i> what you learn, or it goes stale/disappears<p>Your memory's a muscle [2] - you've got to give it a workout, or it will atrophy [3]<p>--------<p>[0] <a href="https://www.flintrehab.com/use-it-or-lose-it" rel="nofollow">https://www.flintrehab.com/use-it-or-lose-it</a> etc<p>[1] <a href="https://www.electronicdesign.com/technologies/dsps/article/21771563/continuing-education-how-to-teach-yourself-almost-anything" rel="nofollow">https://www.electronicdesign.com/technologies/dsps/article/2...</a><p>[2] <a href="https://www.dummies.com/article/body-mind-spirit/physical-health-well-being/aging/work-memory-muscle-224776" rel="nofollow">https://www.dummies.com/article/body-mind-spirit/physical-he...</a><p>[3] <a href="http://www.ucanlearn.net/?page_id=129" rel="nofollow">http://www.ucanlearn.net/?page_id=129</a>
Sometimes when I write notes, just the action of writing them helps burn it into my memory a little. You are not alone, I have a friend who complains about this very problem 55 years after leaving college. Einstein would say that he didn't bother memorizing something if he knew where to look up the information (paraphrased).
- Learn the memory palace technique. It's thousands of years old and really works. People use it to memorize tens of thousands of digits.<p>- Take B. monnieri. It's not a huge effect but it does seem to have a real positive effect on memory if you take it for a while.<p>- Practice n-back<p>- Get good sleep and exercise frequently to optimize your brain health
I don't see this as abnormal. At every stage of abstraction you have to relearn things. Most professors relearn everything every year by teaching a class.<p>Try implementing a basic neural network and backprop in something like numpy.
If you really want to go hardcore, implement it again without numpy.
A sampling of other times this has been asked hereon:<p><a href="https://www.qwant.com/?q=improve+memory+site%3Anews.ycombinator.com" rel="nofollow">https://www.qwant.com/?q=improve+memory+site%3Anews.ycombina...</a>
Look into spaced repetition, and more specifically Anki. It would seem it's the ideal tool for the problem that you're facing : studying for exams, not for the long term.
dude i code for many years now and i still google how some functions and algos work.<p>it doesnt make sense to beat yourself up for not remembering everything<p>one good perspective is you may forget everything about it, but now you still know how to find the answers that lead you to it.
Short tl;dr of my knowledge:<p>- Nothing can really significantly improve your raw baseline memory ability<p>- Familiarity with a domain improves your brain's ability to compress the information and memorize more/faster in short-term memory, but not for longer.<p>- Memorization strategies are powerful. Spaced repetition[1] is probably the most powerful tool known to mankind, since it works around our natural propensity to forget information that doesn't see regular use. Possibly only surpassed by incremental reading/writing. Memory palaces also actually work, and probably could be combined with it.<p>[1] <a href="https://senrigan.io/blog/chasing-10x-leveraging-a-poor-memory-in-software-engineering/" rel="nofollow">https://senrigan.io/blog/chasing-10x-leveraging-a-poor-memor...</a>