TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Show HN: Exactly-once delivery counter using Watermill messaging library

18 点作者 roblaszczak大约 4 年前

5 条评论

kevincox大约 4 年前
&gt; 1. you need to use a Pub&#x2F;Sub implementation that does support exactly-once delivery (only MySQL&#x2F;PostgreSQL for now),<p>To implement exactly-once delivery you first need exactly once delivery.<p>But this project doesn&#x27;t implement exactly once delivery.<p>&gt; It can be triggered by calling the <a href="http:&#x2F;&#x2F;localhost:8080&#x2F;count&#x2F;{counterUUID}" rel="nofollow">http:&#x2F;&#x2F;localhost:8080&#x2F;count&#x2F;{counterUUID}</a> endpoint.<p>Oh, but what if my network goes down right after I sent that request? Did it succeed? There is no way of knowing. So now I have to pick between sending again (at least once) or not (at most once).<p>You <i>can</i> get exactly once delivery. But you need to involve request IDs which you haven&#x27;t done here. Imagine something like this:<p><pre><code> PUT http:&#x2F;&#x2F;localhost:8080&#x2F;count&#x2F;{counterUUID}?request-id={requestUUID}. </code></pre> The requestUUID must be unique per counter. It should be generated when the action occurred (like when the user clicks a button) and should be reused for retries. (Of course this will still be foiled by a user clicking a button multiple times)<p>Now the server does:<p><pre><code> INSERT INTO increments(counter_id, request_id, counted) VALUES (:counterUUID, :requestUUID, FALSE) ON CONFLICT DO NOTHING </code></pre> Now instead of deleting the record you just mark `counted = TRUE`.<p>In theory you can never delete records from this table. In practice you can add a timeout after which double-counting is acceptable.
评论 #26889507 未加载
mimir大约 4 年前
This seems like another odd definition of &quot;exactly once&quot;. The underlying assumption appears to be that all you work is happening insides the sql transaction, so all the work the consumer does is safe and can be rolled back if a failure happens. Many systems are going to be doing at least some work that can&#x27;t be easily rolled back by a transaction (e.g. calling an external API). In this more interesting world, you really can&#x27;t get exactly once since we already did something outside of our transaction.<p>In my experience, it&#x27;s easier to reason about and build systems when idempotency is an application level concern. For example take a bank that has some messaging system to update account balances. While a exactly once system, if designed perfectly, might achieve this, you could also achieve this by building an idempotent &quot;update balance&quot; system. With application level idempotency, you have more flexibility to later add different paths or technologies without as many re-write headaches.<p>Also -- the messages per day stat seems irrelevant. I&#x27;ve yet to encounter many real world systems that don&#x27;t have irregular bursty patterns. With this slow processing rate, you could basically have a single large burst and then normal traffic, but be unable to return to realtime latency for hours&#x2F;days.
评论 #26889548 未加载
adamcharnock大约 4 年前
I think this ACID-based approach is a reasonable way to achieve exactly-once message processing, as long as one is happy to limit that guarantee to only DB operations.<p>If anyone is interested in a failed attempt to implement this you can see my write up [1]. It contains some learnings from the experimental implementation I created for Lightbus.<p>The implementation was based on this presentation [2]<p>[1]: <a href="https:&#x2F;&#x2F;github.com&#x2F;adamcharnock&#x2F;lightbus&#x2F;issues&#x2F;4" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;adamcharnock&#x2F;lightbus&#x2F;issues&#x2F;4</a><p>[2]: <a href="https:&#x2F;&#x2F;vimeo.com&#x2F;111998645" rel="nofollow">https:&#x2F;&#x2F;vimeo.com&#x2F;111998645</a>
plank_time大约 4 年前
Exactly-once delivery in a distributed environment is impossible. Close-to-exactly-once is possible but there is always a way to have a situation where exactly-once fails once you get a network involved.<p>Changing the definition of exactly-once delivery won’t help your users.
poorman大约 4 年前
Seems like any key value store would be better for this. Redis, DynamoDB, etc.. could get a lot higher throughput on messages.
评论 #26889718 未加载