TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Deterministic Aperture: A distributed, load balancing algorithm

143 点作者 theBashShell超过 4 年前

8 条评论

scottlamb超过 4 年前
Interesting to compare with Google&#x27;s approach to the same problems: <a href="https:&#x2F;&#x2F;sre.google&#x2F;sre-book&#x2F;load-balancing-datacenter&#x2F;" rel="nofollow">https:&#x2F;&#x2F;sre.google&#x2F;sre-book&#x2F;load-balancing-datacenter&#x2F;</a><p>In my experience Google&#x27;s deterministic subsetting usually ensures backends have even connection counts. There&#x27;s one significant corner case though: a large backend service that has a large number of small clients, such that client_task_count * subset_size &lt; backend_task_count. Then it basically degrades to random selection. This can happen to infrastructure that supports all of Google&#x27;s products, many of which are relatively tiny.<p>Seems like their &quot;Continuous Ring Coordinates&quot; section onward is designed to address this same problem. I got a bit lost following it though; I might need to read through it sometime my kids are not also wanting my attention...
nvartolomei超过 4 年前
I have implemented an interactive demo of this algorithm which can be found at <a href="https:&#x2F;&#x2F;nvartolomei.com&#x2F;weighted-deterministic-aperture&#x2F;" rel="nofollow">https:&#x2F;&#x2F;nvartolomei.com&#x2F;weighted-deterministic-aperture&#x2F;</a>. Make sure to click &quot;Verbose state&quot; spoiler.<p>Context: Earlier this year I explored using deterministic aperture for balancing writes&#x2F;appends based on storage usage and implemented a visualization to help explain the algorithm to my colleagues.
web007超过 4 年前
[2019] please - I can&#x27;t believe I didn&#x27;t see this before now!<p>I don&#x27;t see any mention of the &quot;load&quot; feedback mechanism to let clients discover a server&#x27;s load, but I assume that comes as part of their framework.
jeffbee超过 4 年前
Isn&#x27;t the squid-like graph in the middle of the article caused simply by using tiny subsets over tiny populations? Would you really bother with subsetting if each participant had fewer than, say, 100 connections? Put a different way, is the reduction in connections from 280k to 25k at the end of the article a thing worth achieving? Any single Linux process can deal with 280k connections, so having that be the aggregate number of connections in a distributed service architecture strikes me as a non-problem.
twic超过 4 年前
What if the backends announced their load via multicast? Then every client could know the load of every backend, and could continue to use P2C or something like that, without needing to maintain a socket for every backend.<p>There&#x27;s still a problem to solve, because you don&#x27;t want to be opening and closing connections to backends all the time - you want to reuse a small number of connections. So maybe you keep a set of connections open, distribute requests between them using P2C or a weighted random choice, but also periodically update the set according to load statistics.
评论 #25605378 未加载
bullen超过 4 年前
I use DNS round-robin for my setup, that way it&#x27;s distributed geographically: <a href="http:&#x2F;&#x2F;host.rupy.se?dark" rel="nofollow">http:&#x2F;&#x2F;host.rupy.se?dark</a>
siscia超过 4 年前
This kind of reports are always interesting, but they ALWAYS miss why they are not following standard control-theory principles.<p>At this point it is a long time since I finished studying control-theory so I may be missing something obvious.
评论 #25603635 未加载
评论 #25602372 未加载
评论 #25601980 未加载
fastest963超过 4 年前
How do they decide how big to make the slice for each service? Is that still manual?
评论 #25602309 未加载