TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Webhook Failure Scenarios

100 pointsby luuover 1 year ago

11 comments

losvedirover 1 year ago
Webhooks are fun to think about. A couple more issues off the top of my head:<p>* Ordering. Since network requests can take variable amounts of time, how do you ensure those two &quot;foo.updated&quot; events are processed in order, or that the receiver can tell their order? Especially something to consider if the webhooks will retry a few times on intermittent failures.<p>* Consistency. Always a concern in distributed microservice land, but maybe more acute when generating webhooks right as things are updated: if the receiver uses the webhook to make an API request right back into the system, will the API have the same view of the data?<p>* DDoS. How do you make sure the webhook destination URL is owned by the subscriber? If your system can generate high volumes of webhook traffic, that could be a problem.<p>* Infinite loops. A silly one, but the user could conceivably point the webhook at a URL of the system that sends the webhook, in such a way that it will cause a new webhookable event to be generated.
评论 #37524293 未加载
评论 #37524636 未加载
评论 #37525982 未加载
JimDabellover 1 year ago
Some interesting previous discussions here:<p>- Best Practices for Using Webhooks (stripe.com): <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=32521159">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=32521159</a><p>- Collection of best practices for providing and consuming webhooks (webhooks.fyi): <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=32517645">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=32517645</a><p>- Give me &#x2F;events, not webhooks (sequin.io): <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27823109">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27823109</a>
chunk_waffleover 1 year ago
For a server sending webhooks to endpoints entered in by users, take care that:<p>a: The FQDN does not resolve to an RFC1918 address (You don&#x27;t want to be POSTing payloads to endpoints within your internal network.)<p>b: If you respond to redirect responses (easier to just not do so for other reasons as well) also make sure those don&#x27;t resolve to internal addresses too!
评论 #37525009 未加载
评论 #37525996 未加载
bennyp101over 1 year ago
Those terms are very confusing - to me orign would be the SENDER ... where the message ORIGNates from ...<p>But yes, those are common scenarios, and most services that offer calling a webhook have retries baked in for such things. It is the responsibility of the consumer to verify if they have already received a message or not.<p>You should not be consuming a webhook &#x27;live&#x27; and attempting to work on it the moment you receive it - that way lies dragons!<p>I have dedicated endpoints for services to send to, and they are verified and consumed as quickly as possible and put into a queue to be processed after. That way I can at least see what was sent, and if we missed something.
评论 #37525971 未加载
tasnover 1 year ago
There are so many more failure scenarios!<p>I wrote a post about common TLS errors recently: <a href="https:&#x2F;&#x2F;www.svix.com&#x2F;blog&#x2F;ssl-tls-incomplete-certificate-chain&#x2F;">https:&#x2F;&#x2F;www.svix.com&#x2F;blog&#x2F;ssl-tls-incomplete-certificate-cha...</a>
leetroutover 1 year ago
That needs better naming in my opinion.<p>Just sender and receiver would be enough IMO.<p>Maybe it is just me but it is unusual to see origin used in this context even if it is technically correct I only ever see it used with CDNs and load balancing.
评论 #37525992 未加载
评论 #37522150 未加载
sergiotapiaover 1 year ago
At the most basic level, what I like to do is create a webhook response controller, the controller enqueues a job and immediately returns an HTTP 200.<p>The job is saved in my postgres database, using Oban (Elixir) so I have observability, accurate bug stacktraces are saved in the job record&#x27;s error field, and I can see how many times the job has been attempted.
caseysoftwareover 1 year ago
Very cool article.<p>I&#x27;m one of the creators of <a href="https:&#x2F;&#x2F;webhooks.fyi&#x2F;" rel="nofollow noreferrer">https:&#x2F;&#x2F;webhooks.fyi&#x2F;</a> and will do a PR to add this one. Thanks!
评论 #37559242 未加载
pyrolisticalover 1 year ago
The way I solve all these problems is.<p>Prerequisite: require receiver to be able to process duplicate webhook being resent.<p>Algorithm outline:<p>1. Have webhook body stored by sender<p>2. Attempt to send body to receiver<p>3. If 2x status returned mark webhook as sent. Done<p>4. Otherwise for any other status or timeout increment attempt for webhook and try again later until maximum attempts (which then mark webhook as failed)<p>This covers the tricky case where success response gets dropped in network by having the sender time out and retry later. The receiver gets a duplicate
评论 #37525028 未加载
nonaneover 1 year ago
Any suggestions to alternative to webhooks? I feel a ‘pull’ based model with cursors and long polling would be simpler and more reliable than webhooks.
评论 #37559271 未加载
评论 #37534128 未加载
评论 #37530157 未加载
naikrovekover 1 year ago
right out of the gate: why not use the terms &quot;sender&quot; and &quot;receiver&quot; instead of &quot;client&quot; and &quot;origin&quot;?<p>the author uses &quot;client&quot; to refer to the application <i>sending</i> the webhook, and &quot;origin&quot; to refer to the application <i>receiving</i> the webhook. this is all backwards to me.<p>I often say that naming things is hard, but it isn&#x27;t so hard that this needs be the result.
评论 #37525993 未加载
评论 #37534153 未加载