> Lastly, in order to help monitor the behavior of the hundreds of partner APIs that IFTTT connects to, we collect information about the API requests that our workers make when running Recipes. This includes metrics such as response time and HTTP status codes, and it all gets funneled into our Kafka cluster.<p>> This way if you query Elasticsearch to find all API errors in the last hour, it can find the answer by looking at a single index, increasing efficiency.<p>This is a really good way to know if third party APIs are having problems. Staying up to date with all those APIs they support must take up significant amount of engineering effort. Many APIs are just second-class citizens for their product owners. Bugs are introduced, changes are made without announcements, and even if there are announcements when you're dealing with so many different APIs it's hard work keeping track of them all and making changes in your app to keep it running especially when APIs are turned off, or schema changes are happening. This seems to be the hard problem IFTTT is solving, integrating into APIs.<p>I'd shy away from starting a project that involves so many other companies APIs just because of how hard of a problem that is to manage, but IFTTT is doing a great job here.