TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

YouTube's Database “Procella”

3 pointsby marklitabout 3 years ago

1 comment

crorellaabout 3 years ago
I find lots of commonalities with other systems aimed at processing big datasets. The only thing I find different is their management of the metadata at a centralized place, which can bring lots of network IO savings since you don&#x27;t have to query the stripe stats (like in ORC) in order to enable stripe skipping.<p>The risk I see with that is that you have to be extra careful when processing updates to the metastore, as sometimes there are competing processes writing metadata about the same partition and if not done properly you&#x27;ll end up with a corruption that might cause problems (ie. you skip a whole file because the metadata was incorrect, causing a correctness problem down the road).
评论 #31419404 未加载