TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Realtime Data Processing at Facebook

5 点作者 codepie将近 9 年前

1 comment

levbrie将近 9 年前
Facebook's targeting of seconds rather than milliseconds and their reasoning behind it ought to serve as a template for companies with similar requirements. Discussions of latency in Spark often center around micro-batching and just how low you can go. In the context of Spark, that makes perfect sense. If your system needs to guarantee sub-80 millisecond latency, you don't want to commit to a system that isn't designed for it. But in the larger context, there are plenty of systems that can provide close to ideal performance with a latency of a few seconds. We've actually divided up our realtime stream processing infrastructure in a similar way - we have features that require milliseconds of latency and they actually go through a completely different pipeline than the realtime features that require heavy processing and provide all of their intended benefits with a few seconds of latency. What really matters is knowing which is which.