TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Map Reduce and Stream Processing

30 点作者 paulsb超过 14 年前

3 条评论

xal超过 14 年前
Really interesting article. I can't help but notice that he used ruby in his example code. Anyone know if this is simply for illustration purposes or is it already possible to use ruby to write hadoop map/reduce jobs?
评论 #1879980 未加载
pama超过 14 年前
We've sometimes used a hack that is a minor variant of the micro-batch processing in MapReduce. We: a) map the latest batch of data; b) in the reducer, join it with a cache from a previous reduction; and c) reduce in part, save a new cache, and proceed with further reductions. (We use our homegrown MapReduce implementation that allows multiple rounds of reduction and access to the filesystem, so I'm not sure this would work in Hadoop.)
Waywocket超过 14 年前
I read that as 'steam processing', and was sorely disappointed.