TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

6.824 Lab 1: MapReduce

4 pointsby AliCollinsover 10 years ago

1 comment

gopalvover 10 years ago
Very interesting development - this is essentially writing an AppMaster (as in YARN apps) for MapReduce, but in Go.<p>I&#x27;ve been playing with writing custom MR (style) streaming apps in Go &amp; finding that the go-routine impl fits neatly into long chains of map&#x2F;reduce&#x2F;filter pipes.<p><a href="https://github.com/hortonworks/gohadoop" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;hortonworks&#x2F;gohadoop</a><p>I&#x27;m mostly missing a proper HDFS implementation and a proper distributed RPC model (fork of the JNI wrapped libhdfs works, but is single threaded &amp; synchronous).<p>In good time, I hope - perhaps I get it in by Feb :)