Scribe: Transporting petabytes per hour via a distributed, buffered queueing

braggy PR is misleading: the 25GB/s coming from CERN is after they filter the data down from 600TB/s because there are no commercial systems that can capture data at higher rates.

Keep in mind the network cost of petabytes per hour cross continent.<p>Those of us who don&#x27;t own our own cross-ocean fiber can&#x27;t afford to design systems like this.

Naming is hard! Facebook used to have a _different_ Scribe (<a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Scribe_(log_server)" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Scribe_(log_server)</a>).<p>We used it at a company I worked for, but it had long-since been deprecated, so I was confused when I saw this Scribe.

Is it normal for these internal system to not implement any kind of access control? From the post it seems every reader can access every stream?

why can&#x27;t Producer&#x2F;Scribed write directly to LogDevice storage. Any reasons for routing through WriteService.

So, if I&#x27;m reading this correctly, 2.5GB&#x2F;s of log data being generated? If we assume (aggressively) that they have 5mil machines in their infrastructure, doesn&#x27;t that mean that each machine would have to be generating 500kB&#x2F;s of log data?<p>Despite that, I find the claims to be underwhelming. So your system can process massive amounts of data by scaling massively horizontally...neat.

How does this compare to a robust Splunk infrastructure?