TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Ask HN: Protobuf as a Database Wire Protocol?

1 pointsby fearthetelomereover 4 years ago
Why is Protobuf a bad choice for a distributed database wire protocol?<p>I understand it&#x27;s a popular choice for service development through the use gRPC, Twirp, etc. I assume there are good performance reasons why Protobuf is not used in database node-to-node communication, but I&#x27;m curious to know what they are.<p>As an aside, if I don&#x27;t want to commit to one protocol like Postgres or MySQL or Memcached, is there an extensible wire protocol out there that can accommodate these different protocols under one umbrella? I&#x27;m considering generically pre-pending each frame with a set of bits to identify which protocol the frame uses. This seems like something that&#x27;s probably already been solved, and I&#x27;m just not aware of it. Does anyone have any ideas?

2 comments

aq9over 4 years ago
So, a lot would depend on what you plan on passing the protobufs over. Plain TCP? gRPC? gRPC kinda sucks for this, the serialization&#x2F;deserialization can be moderately expensive (CPU); and gRPC&#x27;s multiplexing of multiple flows onto the same connection can be hard to reason about (i.e. debug); and the performance characteristics are not great either.
评论 #26108960 未加载
kstenerudover 4 years ago
Have a look at Apache Arrow<p><a href="https:&#x2F;&#x2F;arrow.apache.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;arrow.apache.org&#x2F;</a>