What are some of the best resources for getting more familiar with stuff like playing nicely with branch prediction, instruction pipelines, and building data to fit well in with CPU cache?
The best resource in my opinion is still "What every programmer should know about memory." It is now over ten years old but still the most comprehesive and comprehensible post I know about that topic.<p><a href="https://lwn.net/Articles/250967/" rel="nofollow">https://lwn.net/Articles/250967/</a><p>AFAIK there have been no major breakthroughts since that time in memory technology (Transactional memory could be one). While modern processors may be build a bit different (e.g. added shared L3 chache) you will have no problem understanding them.<p>There is also this github :<p><a href="https://github.com/Kobzol/hardware-effects" rel="nofollow">https://github.com/Kobzol/hardware-effects</a><p>for programs which should show such effects. (I haven't used them yet but they may be interesting for you to play with.)<p>IT Hare has published a nice article about those costs including rules of thumb costs.<p><a href="http://ithare.com/infographics-operation-costs-in-cpu-clock-cycles/" rel="nofollow">http://ithare.com/infographics-operation-costs-in-cpu-clock-...</a><p>Then there are the Agner tables. Giving you latencies and reciprocal throughput for your instructions.<p><a href="https://www.agner.org/optimize/instruction_tables.pdf" rel="nofollow">https://www.agner.org/optimize/instruction_tables.pdf</a><p>If you are through these you have to read the intel and AMD optimization guides. (ARM may or may not have something similar.)<p>The paper by Kazushige Goto ("Anatomy of a High Performance Matrix Multiplication") is an example of Cache and TLB considerations in a non-trivial example.<p><a href="http://www.cs.utexas.edu/users/pingali/CS378/2008sp/papers/gotoPaper.pdf" rel="nofollow">http://www.cs.utexas.edu/users/pingali/CS378/2008sp/papers/g...</a>