TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

You Can't Guarantee Webhook Ordering

57 点作者 yurisagalov将近 3 年前

8 条评论

Diggsey将近 3 年前
This is rubbish, we&#x27;ve run with guaranteed webhook ordering for years, so the idea that you can&#x27;t do is laughable.<p>Timestamps don&#x27;t solve the issue, and neither do &quot;thin payloads&quot; since the receiver has no idea how long to wait before assuming that the order is certain, and if you have a problem on the sender side it could cause logic errors for all of your clients.<p>Most of these problems are solved if the receiver doesn&#x27;t process the webhook immediately, but instead queues it internally. You don&#x27;t have issues with the queue being stalled due to one bad webhook, because there is no event-specific processing happening on the receiver (other than perhaps ignoring some events). The queue can still be stalled if there is a wider problem, but as soon as the problem is resolved, the system can catch up on those queued webhooks, and synchronization integrity is maintained.<p>Having said all that, if I were to design a new system I would go with a pull-based system instead. In this system, the client would request a range (start time, max count) of events via an HTTP request, and the response would include the &quot;end time&quot; that can be used in the next query. A &quot;webhook&quot; would contain an empty payload, and would simply indicate that the queue had become non-empty - this could be omitted entirely if realtime updates are not required, instead having the client poll.<p>The advantages of this approach are that it&#x27;s easy for consumers to &quot;replay&quot; a set of events if they accidentally lose them, and it&#x27;s also a lot more efficient, since many events can be sent per request (we gain some of this benefit at the moment by supporting &quot;batch&quot; webhooks containing multiple events, but it requires opt-in from the client.) Additionally, it allows webhooks to be versioned more easily, since you can have versioned endpoints for fetching events, and it also allows you to have an arbitrary number of consumers of the same set of events with no additional complexity.
评论 #32334512 未加载
评论 #32335259 未加载
评论 #32334487 未加载
emadda将近 3 年前
You can also poll an &#x2F;events endpoint to get a consistent ordering.<p>I used &#x2F;events to apply writes from Stripe to a local database for this reason in the tdog CLI:<p><a href="https:&#x2F;&#x2F;table.dog&#x2F;blog&#x2F;principles&#x2F;events-are-better&#x2F;" rel="nofollow">https:&#x2F;&#x2F;table.dog&#x2F;blog&#x2F;principles&#x2F;events-are-better&#x2F;</a>
评论 #32336298 未加载
评论 #32336628 未加载
spiffytech将近 3 年前
This is a big pain point with Stripe&#x27;s webhooks, and I think there&#x27;s ample room for improvement.<p>Senders could guarantee ordering by only sending webhook n+1 after the HTTP request for webhook n completes, rather than sending them concurrently or in arbitrary order. For efficiency, perhaps only guarantee ordering for hooks related to each resource rather than all of a customer&#x27;s hooks.<p>Or, include a monotonic counter in the webhook so the recipient can tell when it would apply an old state on top of a new one.<p>What the recipient does when they receive the webhook is up to them (delays, parallelism, etc.), but at least they&#x27;d know the correct event order.<p>The author raises a good point about what to do in the face of errors, but I&#x27;d vastly prefer to handle special behavior upon recipient error (stall, dead letter queue) to the current Stripe reality of &quot;things come in out of order, and we don&#x27;t give you the info needed to reassemble the order on your end&quot;.
评论 #32336397 未加载
spiffytech将近 3 年前
The solution to receiving webhooks in unknown order is to ignore the payload and refetch the resource. Yet naively implemented, this still leaves race conditions on the recipient end: if two webhooks can come in at once, you have to make sure you process them serially, since your refetch or database write could complete in arbitrary order.<p>That&#x27;s non-trivial engineering to foist upon every recipient of your webhooks.<p>I like the idea of the &#x2F;events pull-based endpoint, which keeps engineering much simpler on for recipient: <a href="https:&#x2F;&#x2F;blog.sequin.io&#x2F;events-not-webhooks&#x2F;" rel="nofollow">https:&#x2F;&#x2F;blog.sequin.io&#x2F;events-not-webhooks&#x2F;</a>
评论 #32336410 未加载
rektide将近 3 年前
&gt; <i>At first glance it seems like a simple, and easy to implement idea — just send the webhooks in order.</i><p>Not webhook specific but a couple hours today figuring out that some our service calls to internal services look like they open &amp; are sent &amp; processing, but the target server doesnt even see the request for a full 8s sometimes. The call itself was not thrle problem, the service just hadnt started until a long time after data was all sent.
topspin将近 3 年前
Would have thought this is self evident. Intermediates exist and they can do essentially anything. Without absolute control over every aspect of the systems involved you have no guarantees about ordering.
评论 #32335327 未加载
davidgu将近 3 年前
Really enjoyed the post Tom. I really like the structure of introducing a potential solution then pointing out the problems with it.<p>P.S. Svix is great, super happy customer here :)
评论 #32338330 未加载
Spivak将近 3 年前
Those who fear or do not know Kafka are doomed to work around its absence. Like I just cannot understand this mindset of &quot;just educate people&quot; when the system doesn&#x27;t meet the requirements of its users. If your users want event ordering just give them event ordering.<p>You can even keep your existing webhook code by providing a synchronous bridge to Kafka so &quot;just send them in order but wait for the 200 before sending the next one.&quot; Boom, now you are guaranteed the events are recorded and processed in order.
评论 #32334660 未加载