TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

History of Apache Arrow: Developing a standard for in-memory columnar processing

6 pointsby riboflavinalmost 7 years ago

1 comment

kwilletsalmost 7 years ago
&gt;In the Big Data world, we didn’t see much talk about columnar data until Google’s Dremel paper in 2010<p>Try 2005:<p><a href="http:&#x2F;&#x2F;db.csail.mit.edu&#x2F;projects&#x2F;cstore&#x2F;#papers" rel="nofollow">http:&#x2F;&#x2F;db.csail.mit.edu&#x2F;projects&#x2F;cstore&#x2F;#papers</a><p>We had a very large Vertica installation up in 2009.<p>But as Abadi has noted it&#x27;s an in-memory format, and it removes a lot of bottlenecks at the CPU-Memory interface. In that sense it&#x27;s quite new; we&#x27;ve been wrestling with tricky in-memory formats for some time.