TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Thrill – Big Data Processing with C++

127 点作者 brakmic超过 8 年前

6 条评论

lorenzhs超过 8 年前
The companion paper to Thrill, with more details on its architecture and some benchmarks and comparisons to Spark and Flink: <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1608.05634" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1608.05634</a>
评论 #13628599 未加载
codepie超过 8 年前
There&#x27;s also Blogel [0] which is a distributed graph processing framework in C++ and it runs significantly faster than its counterpart in Java, Apache Giraph [1].<p>I have started wondering if the big data developers really care about the speed; the advantages of these Java softwares start to fade out when compared with their C++ counterparts.<p>[0] - <a href="http:&#x2F;&#x2F;www.cse.cuhk.edu.hk&#x2F;blogel&#x2F;" rel="nofollow">http:&#x2F;&#x2F;www.cse.cuhk.edu.hk&#x2F;blogel&#x2F;</a><p>[1] - <a href="http:&#x2F;&#x2F;www.cse.cuhk.edu.hk&#x2F;blogel&#x2F;papers&#x2F;blogel.pdf" rel="nofollow">http:&#x2F;&#x2F;www.cse.cuhk.edu.hk&#x2F;blogel&#x2F;papers&#x2F;blogel.pdf</a>
评论 #13627635 未加载
adrianN超过 8 年前
There is also the STXXL [1] for times when your data is big but not &quot;big&quot;. It contains containers and algorithms optimized for external storage.<p><a href="http:&#x2F;&#x2F;stxxl.sourceforge.net&#x2F;" rel="nofollow">http:&#x2F;&#x2F;stxxl.sourceforge.net&#x2F;</a>
评论 #13627703 未加载
pzh超过 8 年前
Does anybody know how this is different from Spark? These Distributed Immutable Arrays sound suspiciously similar to Spark&#x27;s Resilient Distributed Datasets. Is it just the choice of C++ as opposed to Scala that would make this more efficient?<p>Also, I wonder if and how they implemented the concept of lineage (unless these DIAs are not really very resilient)... I thought Spark relied on Scala&#x27;s delayed evaluation to do that, though I may be mistaken.
评论 #13626854 未加载
评论 #13627713 未加载
Mikeb85超过 8 年前
Very cool. Will have to remember this, maybe write an R package that makes use of it.
评论 #13627489 未加载
tmsldd超过 8 年前
The Force is strong with this one