How do people in production handle the possibility that your service might miss a webhook notification? If you miss a notification you'll end up with stale data and you won't know it.<p>Slack has a retry policy for a while but will then just give up. Another webhook provider I've looked at says nothing at all about this sort of thing. How do folks deal with this in production systems?<p>Seems to me like the best way to address this issue is to use the webhook as a hint that you need to run some other process that guarantees you've got all updates.
I think the "securing webhooks" section is missing some critical tips that we've learned in production.<p>1) Resolve the DNS of the webhook URL, and compare all returned addresses from that resolution against an IP blacklist, which includes all RFC1918 addresses, EC2 instance metadata, and any other concerning addresses.<p>2) Even though it seems like you'd want to, do NOT blindly return an unexpected response to the person configuring the webhook. Say there was an error, what the code was, etc, but returning the response body means you basically just gave someone curl with a starting point on your network (see 1 as well)<p>3) Find ways to perform other validations of those webhooks. Are the URLs garbage? Are they against someone else's system? Create validation workflows that require initial pushes to the URL with a validation token to be entered back into your system, like validating an email address by clicking a link.
Aside: Webhooks are always a pain.<p>Implementing polling is easier for both sides.<p>I routinely have to integrate with random 3rd party systems, some with no or broken webhooks, some with no API at all.
It turns out for my customers (this may not be always the case) eventual consistency is more important than timelyness.<p>What I do now every time I need to sync data from a third party is I always implement some sort of pull first with idempotent logic on my side. It's easier, and it allows me to just re-run things if something fails (e.g. network error, unexpected data in production, etc).<p>Only when that works reliably and only if required by the customer I implement a webhook, but I usually throw away most of the message and just wake up my polling worker that is otherwise polling relatively slowly.
Sort of disagree with the send-everything-in-the-payload approach. It opens your system up to all sorts of weird edge case bugs like receiving hooks out of order which could mean stale data is considered fresh. It also means you have to care a lot more about verifying the authenticity of the request.
Also, in your documentation, please show what the webhook events will look like since developers actually want to write code and not guess at what we will get.<p><i>cough</i> Stripe. (<a href="https://stripe.com/docs/api#events" rel="nofollow">https://stripe.com/docs/api#events</a>)
This is going to sound bizarre, but why do webhooks and not just an AMQP queue? I get that receiving HTTP POSTs is easier, but it just seems better to setup a publisher/subscriber relationship. That way, if a subscriber goes down, they can always catch up. And publishers can allow messages to sit in the queue with a TTL and max_size. It seems like a win-win for everyone.
Webhook only makes sense if you don't care a single bit about missing updates. If not, it's deeply flawed.<p>A pull model (polling, long-polling, SSE, etc) is strictly superior for synchronisation. You just can't "miss" updates, can restart from the beginning again and reinterpret past events in a different light, the client goes at its own pace, etc.
I did a talk a little while back on providing a good developer experience around webhooks that covers a lot of the same topics. I wish I had a recording of it, but the slides are here: <a href="https://speakerdeck.com/johnsheehan/crafting-a-great-webhooks-experience-1" rel="nofollow">https://speakerdeck.com/johnsheehan/crafting-a-great-webhook...</a><p>Edit: found the video <a href="https://www.youtube.com/watch?v=xc5ezyJjz1k&feature=youtu.be&t=1266" rel="nofollow">https://www.youtube.com/watch?v=xc5ezyJjz1k&feature=youtu.be...</a>
I'd like to follow up on the statement that the OpenAPI tools do not support webhooks. This is slated to change in an upcoming version of the OpenAPI-specification. Check out <a href="https://github.com/OAI/OpenAPI-Specification/pull/763" rel="nofollow">https://github.com/OAI/OpenAPI-Specification/pull/763</a> to see details. As soon as this is released, it only be a matter of time before Swagger and the rest support webhooks.
To be fully client based (serverless) you need a middle-man for web-hooks. Websockets are a better alternative for stand-alone web clients. There are also "push notifications" via web workers but they are vendor dependent.
What I am most interested in is how to test/debug webhooks during development.<p>How do I tell webhook providers to send test notifications to my local development instance without tampering with the production setup on both sides?
So "push technology" is called "webhooks" now?<p>How does this all integrate with HTTP 2? Can you get your notifications over a channel you already have open for other reasons?