TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Thundering Herds and Promises

133 pointsby YoavShapiraabout 6 years ago

14 comments

sciurusabout 6 years ago
This is how many HTTP caches and CDNs work. The terminology used to describe it is often request collapsing or request coalescing.<p>Some examples:<p>* varnish: <a href="https:&#x2F;&#x2F;info.varnish-software.com&#x2F;blog&#x2F;hit-for-pass-varnish-cache" rel="nofollow">https:&#x2F;&#x2F;info.varnish-software.com&#x2F;blog&#x2F;hit-for-pass-varnish-...</a><p>* nginx: <a href="http:&#x2F;&#x2F;nginx.org&#x2F;en&#x2F;docs&#x2F;http&#x2F;ngx_http_proxy_module.html#proxy_cache_lock" rel="nofollow">http:&#x2F;&#x2F;nginx.org&#x2F;en&#x2F;docs&#x2F;http&#x2F;ngx_http_proxy_module.html#pro...</a><p>* fastly: <a href="https:&#x2F;&#x2F;docs.fastly.com&#x2F;guides&#x2F;performance-tuning&#x2F;request-collapsing" rel="nofollow">https:&#x2F;&#x2F;docs.fastly.com&#x2F;guides&#x2F;performance-tuning&#x2F;request-co...</a><p>* cloudfront: <a href="https:&#x2F;&#x2F;docs.aws.amazon.com&#x2F;AmazonCloudFront&#x2F;latest&#x2F;DeveloperGuide&#x2F;RequestAndResponseBehaviorCustomOrigin.html#request-custom-traffic-spikes" rel="nofollow">https:&#x2F;&#x2F;docs.aws.amazon.com&#x2F;AmazonCloudFront&#x2F;latest&#x2F;Develope...</a>
评论 #19684223 未加载
评论 #19688751 未加载
thamerabout 6 years ago
In practice there&#x27;s a bit more to it than what the article describes, especially for a distributed cache: you need to have the Promise auto-expire so that if the machine that&#x27;s performing the backend read disappears the other readers don&#x27;t stay stuck forever waiting for it. It&#x27;s also useful to have a feature in the cache itself that blocks the caller until the Promise has been fulfilled, so as to avoid repeated requests asking if the data is finally there.<p>As an aside, Guava loading caches[1] implement per-key locking so that multiple threads accessing a missing key simultaneously[2] would only lead to a single load with all other readers waiting for the loading thread to complete the fetch and populate the cache.<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;google&#x2F;guava&#x2F;wiki&#x2F;CachesExplained" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;google&#x2F;guava&#x2F;wiki&#x2F;CachesExplained</a><p>[2] in the sense of &quot;in the time it takes for the first accessor to complete the backend read&quot;
评论 #19684970 未加载
评论 #19688196 未加载
评论 #19687116 未加载
spankaleeabout 6 years ago
This is how all async caches are supposed to work. You never want concurrent requests for the same uncached resource to all hit the backend.<p>For TypeScripters out there, this is what my team wrote for our static analysis framework: <a href="https:&#x2F;&#x2F;github.com&#x2F;Polymer&#x2F;tools&#x2F;blob&#x2F;master&#x2F;packages&#x2F;analyzer&#x2F;src&#x2F;core&#x2F;async-work-cache.ts" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;Polymer&#x2F;tools&#x2F;blob&#x2F;master&#x2F;packages&#x2F;analyz...</a>
nicwolffabout 6 years ago
Instead – and more usefully given how slow some of our backend APIs are – we cache each value twice, under e.g. `key` with a short TTL and `key_backup` with a long TTL.<p>The first process to miss on `key` renames `key_backup` to `key` (which is atomic and fast on Redis) and goes to the backend for a new value to cache twice and return, while the rest of the herd reads the renamed backup.<p>Yes, this doubles the total cache size, or equivalently halves the number of keys we have room for. That&#x27;s a price we&#x27;re OK with paying to avoid blocking reads while a value is recalculated.
评论 #19688318 未加载
aaron_m04about 6 years ago
This looks like it would be a very useful design, however the article doesn&#x27;t discuss the implementation of the Promise.<p>This is not something memcached or redis support out of the box, as far as I know. It would seem to imply a cache manager service that has its own in-memory table of Promises.
评论 #19683800 未加载
评论 #19686135 未加载
评论 #19683795 未加载
评论 #19683980 未加载
评论 #19683868 未加载
评论 #19683984 未加载
评论 #19683762 未加载
_bxg1about 6 years ago
&quot;instead of caching the actual value, we cached a Promise that will eventually provide the value&quot;<p>I did this exact thing recently in a client-side HTTP caching system for frequently-duplicated API requests from within a single page. Cool to see it pop up elsewhere.
sbovabout 6 years ago
Back when I worked on a similar problem 10 years ago, we solved it by having a quickly expiring memcached key for hitting the database. So if the value wasn&#x27;t cached, and if that key wasn&#x27;t there, it would attempt to add that key. If it was added, it would hit the database and cache the result. Otherwise, if that key was there or it didn&#x27;t successfully add it, it would wait for a short period of time, then re-try the whole process again.<p>There&#x27;s other similar problems elsewhere too though. A cold MySQL start is a bitch when you have and rely upon huge amounts of memory for MySQL to service your requests - this is especially noticeable if you have so much traffic you need to cache some results. Back then it would take us about an hour before a freshly spun up MySQL instance could keep up with our regular traffic, even accounting for stuff being cached.
risabout 6 years ago
Came up with something along these lines at my last place - not actually the hardest thing to do as long as you&#x27;ve got a robust &amp; convenient locking system available to you. In my case I abused the common db instance that all the clients were connected to anyway to synchronize the callers on postgres advisory locks. Sure, this isn&#x27;t the infinitely scalable, Netflix-scale solution that everyone is convinced they need for everything, but it <i>will</i> probably work <i>absolutely fine</i> for &gt;90% of development scenarios.<p><a href="https:&#x2F;&#x2F;gist.github.com&#x2F;risicle&#x2F;f4807bd706c9862f69aa" rel="nofollow">https:&#x2F;&#x2F;gist.github.com&#x2F;risicle&#x2F;f4807bd706c9862f69aa</a>
layoutIfNeededabout 6 years ago
We do the same thing on client side when lazy-loading assets (typically textures) from the disk, cause you don’t want to hold multiple copies of the same asset in memory.
Johnny555about 6 years ago
I&#x27;m not a developer, but to be honest, thought that&#x27;s how all non-trivial caching implementations worked -- instead of going directly to the back end or having each one trigger a read from the back end for a cache-miss, all of the threads that want that resource just waited on it to appear in the cache.
m0meniabout 6 years ago
Interesting. I also ran into this on a much smaller scale 3 years ago and made <a href="https:&#x2F;&#x2F;github.com&#x2F;AriaFallah&#x2F;weak-memoize" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;AriaFallah&#x2F;weak-memoize</a>
jelderabout 6 years ago
It&#x27;s been a while since I used Rails, but think the fragment cache does this &quot;out of the box.&quot;
niklabhabout 6 years ago
i have implemented this in node.js <a href="https:&#x2F;&#x2F;npmjs.com&#x2F;package&#x2F;memoise" rel="nofollow">https:&#x2F;&#x2F;npmjs.com&#x2F;package&#x2F;memoise</a>
z3t4about 6 years ago
In an old apartment I had a lot of stuff on the same extension plug. I had to turn on the devices one by one to prevent power loss ... There&#x27;s also the same problem&#x2F;solution when boarding airplanes. Even if it feels backwards, it&#x27;s actually faster to let one third of the requests go through first, then to let all requests go through at once.