Yahoo Open Sources S4 Real-Time MapReduce Framework

the entire point was that it's not mapreduce, it's a stream processing system. try <a href="http://wiki.s4.io/Manual/S4Overview#toc1" rel="nofollow">http://wiki.s4.io/Manual/S4Overview#toc1</a> for a "<i>what is S4</i>" intro.<p><i>The architecture resembles the Actors model, providing semantics of encapsulation and location transparency, thus allowing applications to be massively concurrent while exposing a simple programming interface to application developers.</i><p>and<p><i>S4 is a general-purpose, distributed, scalable, partially fault-tolerant, pluggable platform that allows programmers to easily develop applications for processing continuous unbounded streams of data.</i><p>go stream processing. also see <i>Flume</i> for related recent works.

That's not a readme! At best that's a weak INSTALL.<p>Anyone care to illuminate to me how this works? "Real-time MapReduce" sounds almost like an oxymoron.<p>The project's site ( <a href="http://s4.io/" rel="nofollow">http://s4.io/</a> ) isn't very helpful either. Full source:<p><pre><code> the S4 open source project. coming soon.</code></pre>

It's kind of funny how almost no one is advised to write code in Java these days, and on the other hand really cool infrastructure projects like this or Cassandra are almost always written in Java.<p>I understand they have to perform better than the average webapp, but still...funny.

@atarashi: S4 is a stream processing platform that has been developed at Yahoo!. The website and github repository are under development. Follow us on twitter <a href="http://twitter.com/s4project" rel="nofollow">http://twitter.com/s4project</a>

Website is live: <a href="http://s4.io/" rel="nofollow">http://s4.io/</a>