I can never understand why enforcing (sane and secure) TLS and adding a (shared) secret request header isn't the usual simple answer.<p>As far as I know, TLS prevents all these security issues, except authentication of the webhook caller, hence the shared secret which your webhooks should check.<p>I've verified in the TLS specs that every issue I've seen raised by people is actually covered, yet people still think (and write) that TLS doesn't sign traffic (it does), or TLS doesn't prevent replay attacks (it does), etc.<p>Usually I give them pointers to the specs, and they say "it doesn't hurt to add even more security" but they're really rolling out their own encryption, which you shouldn't do.<p>I'd say enforce TLS and authenticate the webhook caller...
Hey HN - I'm the author of this post. (Founder of WorkOS)<p>Lots of people have written about how to scale event-based systems or how to design distributed infrastructure for queue management. But what's often overlooked is the subtle-yet-important product decisions that go into designing webhooks for a great developer experience.<p>We don't actually offer webhooks-as-a-service at WorkOS, but we provide webhooks to developers and I've built this feature twice now, previously at Nylas and now at WorkOS. Hopefully this article helps the next person who builds them. :)<p>For anyone curious how we do it at WorkOS, here's our webhooks API reference: <a href="https://workos.com/docs/reference/webhooks/connection" rel="nofollow">https://workos.com/docs/reference/webhooks/connection</a>
SSRF is a tremendous problem with webhooks. Stripe also solves this: <a href="https://fly.io/blog/practical-smokescreen-sanitizing-your-outbound-web-requests/" rel="nofollow">https://fly.io/blog/practical-smokescreen-sanitizing-your-ou...</a>
Something else Stripe and most other "serious" webhook senders do is only send Webhooks from a predefined set of IP addresses [1].<p>This isn't the security guarantee that signing, etc. are but it's a substantial help in adding some Security In Depth.<p>1 - <a href="https://stripe.com/docs/ips" rel="nofollow">https://stripe.com/docs/ips</a>
Maybe a stupid question but... why not using an MQTT server instead of webhooks? Clients could be notified in RT, TLS is already backed in the protocol with client auth... the client doesn't need to expose an endpoint, only to connect to a server...
AWS EventBridge announced API destinations recently [1]. I can't find anywhere in their docs about SSRF, but given it's a managed service running outside of your network it sounds like a good option. Anyone care to comment if this would be a good solution for webhooks?<p>1 - <a href="https://aws.amazon.com/blogs/compute/using-api-destinations-with-amazon-eventbridge/" rel="nofollow">https://aws.amazon.com/blogs/compute/using-api-destinations-...</a>
Hehe, just did a show HN[1] for Diahook (webhooks as a service) earlier today. I wonder if that's what triggered this posting. :P<p>Anyhow, I wish more people had better webhooks, it's such an important part of any API, so please read this blog post, understand the challenges, and provide a great webhooks experience!<p>[1] <a href="https://news.ycombinator.com/item?id=26399672" rel="nofollow">https://news.ycombinator.com/item?id=26399672</a>
Kind of wild that there's no mention of SSRF. A quick search shows it's a pretty frequent security issue in Webhooks: <a href="https://www.google.com/search?q=ssrf+webhook" rel="nofollow">https://www.google.com/search?q=ssrf+webhook</a>
Not just logging the outbound webhooks to STDOUT and hope they get ingested into logstash or something, but actually log them in your SoR/RDBMS so that you can present that data to a debugging developer.
This was a great read – thanks for sharing!<p>I've always been a huge fan of how Stripe handles their webhooks. They are pretty much doing everything listed in your article.
What people totally omit when thinking about webhooks is the parallels to web sockets and client side updates. In the last round of our system updates, we have "attached" the web sockets and webhooks to the same message bus which has simplified the overall design of the system.
At FastComments we send an your API token in the webhook request, and if your API doesn't return a 401 with an invalid token the integration won't activate.
This was an interesting read! If you ever decide to update this article & include tools for working with webhooks, Hookdeck(<a href="https://hookdeck.io" rel="nofollow">https://hookdeck.io</a>) should be at the top of the list. Hookdeck is a complete webhook infrastructure that makes working with webhooks painless. we pride ourselves in reliability. We provide tooling for easy monitoring and troubleshooting of webhooks