TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Show HN: Interactive map for architecting big data pipelines

144 点作者 ddrum001将近 8 年前

12 条评论

lobster_johnson将近 8 年前
This is very useful.<p>I wish it had some information about supported languages. Most of the processing systems are JVM-based and require that you write your program in a JVM language. Some have Python support. But I have yet to encounter one that allows you write your pipelines in Go, Rust or JavaScript, for example. One notable exception is Storm, which supports pluggable runners, including one that talks to an external program over standard I&#x2F;O. My impression that aside from Python, today&#x27;s pipelines require a large amount of JVM buy-in, something I&#x27;m personally not interested in.<p>I&#x27;d also love some kind of metric for &quot;aliveness&quot;. For example, my impression is that Storm was hot for about a week, and then Spark and Flink happened, and now nobody is talking about it, and Twitter itself has apparently replaced it with Heron.
评论 #14641819 未加载
评论 #14643410 未加载
评论 #14642285 未加载
评论 #14644030 未加载
dsacco将近 8 年前
Wow, this is awesome. What a simple yet useful idea.<p>This format lends itself to data processing, but I think it would be really nice to apply it a variety of workflows. For example, you could model the software deployment process across different languages and frameworks. It could be a good complement to StackShare.<p>A bit of constructive feedback: I&#x27;m not a stickler for UX or design, but maybe spruce up the gray boxes a bit. I&#x27;ve never been a designer though, so take that for what you will.
vosper将近 8 年前
If you&#x27;re aiming to be comprehensive, then you may want to add Onyx under streaming processors. It&#x27;s not as popular as the options you&#x27;ve listed though, so I understand why it might be left off.<p><a href="http:&#x2F;&#x2F;www.onyxplatform.org" rel="nofollow">http:&#x2F;&#x2F;www.onyxplatform.org</a>
评论 #14646288 未加载
lolptdr将近 8 年前
This is awesome. Great aggregating of so many buzzwords and brand names that I&#x27;ve heard over the years. Nice job!<p>Keep it simple and hierarchical. I suggest additional filters for each component of the data engineering flow that can discern unique features or commonalities.
评论 #14640680 未加载
greggyb将近 8 年前
Interesting that Microsoft&#x27;s only showing in this map is for Azure Blob Storage.
评论 #14641909 未加载
评论 #14641442 未加载
jnatkins将近 8 年前
StreamSets Data Collector is another useful open-source ingest tool. I&#x27;m biased, but people seem to like it.
trwoway将近 8 年前
Strange that Apache Flink and Google Dataflow don&#x27;t figure in the Stream Processing list
评论 #14640736 未加载
rahilb将近 8 年前
Storm also has a Scala api, but is filtered when selecting Stream Processing and Scala.
Aegeaner将近 8 年前
Why is there no Flink in Streaming processing framework?
评论 #14647030 未加载
Faaak将近 8 年前
What do you think of Arctic for the Data point ?
rjbwork将近 8 年前
Kind of cool, but only 2 entries from Azure that aren&#x27;t on other places.<p>Kind of useless for us on Azure.
lima将近 8 年前
Citus DB is missing.