TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

What Every Programmer Should Know About Memory (2007) [pdf]

82 pointsby zonovarover 3 years ago

7 comments

corysamaover 3 years ago
This comes up over and over. It&#x27;s great. But, the 75% of useful content comes after 25% of diving way too deep into the details of the electrical engineering.<p>&quot;Every programmer&quot; should know the orders of magnitude of cache hierarchy latencies, how RAM loads a whole cache line to service that single byte you requested, roughly how the automatic prefetcher thinks, that MESI and NUMA access are a growing issue, that the TLB cache is a thing, and generally how the memory controller is the interface between the CPU and pretty much everything else --like the NIC, HD and GPU.<p>&quot;Every programmer&quot; does not need to know about DRAM discharge timing, row selection and refresh cycles.<p>Understanding quad-pumped bus frequencies and CAS latencies is great when you are building systems. But, it&#x27;s not something you think about when designing data structures and algos.
评论 #30355615 未加载
评论 #30357873 未加载
评论 #30355594 未加载
评论 #30355269 未加载
评论 #30355675 未加载
评论 #30355105 未加载
jcranmerover 3 years ago
If you have a 114-page paper, you probably don&#x27;t have a document of what <i>every</i> programmer should know about memory, especially since many programmers work in domains where some of the recommendations here aren&#x27;t even possible to follow!<p>Here&#x27;s a brief summary of what <i>every</i> programmer really needs to know about memory:<p>* There is a hierarchy of memory from fast-but-small to slow-but-large. The smallest and fastest is in L1 cache, which is typically on the order of 10-100 KiB per core, and isn&#x27;t growing substantially over time. The largest cache is L3 cache, in the 10s of MiB, and shared between many or all cares, while your RAM is in the 10s of GiB. An idea of the difference in access times can be found here: <a href="https:&#x2F;&#x2F;gist.github.com&#x2F;jboner&#x2F;2841832" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;jboner&#x2F;2841832</a>.<p>* All memory traffic happens on cacheline-sized blocks of memory, 64 bytes on x86. (It might be different on other processor architectures, but not by much). If you&#x27;re reading 1 byte of data from memory, you&#x27;re going to also read in 63 other bytes anyways, whether or not you&#x27;re actually using that data.<p>* The slow speed of DRAM (compared to processor clock speed) means that it&#x27;s frequently the case that trading off a little CPU computation time for tighter memory utilization. For example, storing a 16-bit integer instead of a pointer-sized integer, if you know that the maximum value will be &lt;65,536.<p>* &quot;Pointer chasing&quot; algorithms are generally less efficient than array based algorithms. In particular, an array-based list is going to be faster than linked lists most of the time, especially on small (&lt;1,000 element) lists.<p>That about covers what I think <i>every</i> programmer needs to know about memory. There are some topics that <i>many</i> programmers should perhaps know--cross-thread contention for memory, false sharing, and then NUMA in descending order, I think, but by the time you&#x27;re in the weeds of &quot;let&#x27;s talk about nontemporal stores&quot;... yeah, that&#x27;s definitely not in the ballpack of everyone should know about them, especially if you&#x27;re not going to mention when nontemporal stores will hurt instead of help [1]. Also, transactional memory is something I&#x27;d put on my list of perennial topics of &quot;this sounds like a good idea in theory, but doesn&#x27;t work in practice.&quot;<p>[1] What Drepper omits is that nontemporal stores will evict the cache lines from cache if they&#x27;re already in cache, so you <i>really</i> shouldn&#x27;t use them unless you <i>know</i> there is going to be no reuse.
评论 #30367835 未加载
mhh__over 3 years ago
The fields of microarchitecture and compiler design (specifically relating to microarchitecture also, i.e. we didn&#x27;t go with Itanium or RISC) could really use a new batch of textbooks.<p>Hennessey and Patterson is still good but there&#x27;s only like 1 alternative, and the most up to date &quot;complete&quot; book on memory I&#x27;m aware of is from 2007 (Jacobs et al).<p>Also: I&#x27;ve read about memory quite bit, I get it. What I don&#x27;t get, and am thus asking for recommendations for is: &quot;What every programmer should know about storage&quot;.
mhh__over 3 years ago
You can get an idea of your computers memory hierarchy in your browser here:<p><a href="https:&#x2F;&#x2F;chipsandcheese.com&#x2F;memory-latency-test&#x2F;" rel="nofollow">https:&#x2F;&#x2F;chipsandcheese.com&#x2F;memory-latency-test&#x2F;</a>
dangover 3 years ago
The main past threads appear to be:<p><i>What every programmer should know about memory [pdf]</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27343593" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27343593</a> - May 2021 (7 comments)<p><i>What every programmer should know about memory, Part 1 (2007)</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=25908018" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=25908018</a> - Jan 2021 (26 comments)<p><i>What Every Programmer Should Know About Memory (2007) [pdf]</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=19302299" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=19302299</a> - March 2019 (97 comments)<p><i>What every programmer should know about memory, Part 1 (2007)</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=15300547" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=15300547</a> - Sept 2017 (12 comments)<p><i>What Every Programmer Should Know About Memory (2007) [pdf]</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14622861" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=14622861</a> - June 2017 (26 comments)<p><i>What every programmer should know about memory (2007)</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=10601626" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=10601626</a> - Nov 2015 (53 comments)<p><i>What Every Programmer Should Know About Memory (2007) [pdf]</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=9360778" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=9360778</a> - April 2015 (15 comments)<p><i>What every programmer should know about memory, Part 1</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=3919429" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=3919429</a> - May 2012 (79 comments)<p><i>What every programmer should know about memory [2007]</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=3360188" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=3360188</a> - Dec 2011 (2 comments)<p><i>What every programmer should know about memory</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=1511990" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=1511990</a> - July 2010 (37 comments)<p><i>What every programmer should know about memory, Part 1</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=1394346" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=1394346</a> - June 2010 (4 comments)<p><i>What Every Programmer Should Know About Memory</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=659367" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=659367</a> - June 2009 (1 comment)<p><i>What every programmer should know about memory, Part 1</i> - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=58627" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=58627</a> - Sept 2007 (7 comments)
throwaway81523over 3 years ago
This is from 2007! Still somewhat relevant though.
评论 #30358545 未加载
btdmasterover 3 years ago
`curl cht.sh&#x2F;latency` illustrates potential bottlenecks quite well given how succinct it is.