Frank McSherry was a researcher at MSR Silicon Valley; when the office was closed in 2014, most (all?) of the researchers were also let go. Frank was a member of a research group working on differential dataflow; not long before the office was closed they released a really awesome open-source library, Naiad [0], which looked extremely promising as a means for efficiently implementing certain stream-processing tasks.<p>The team had a nice blog [1] going too, explaining how different aspects of Naiad make it more efficient than other stream-processing systems (at least, for certain classes of algorithms).<p>[0]: <a href="https://github.com/MicrosoftResearch/Naiad/" rel="nofollow">https://github.com/MicrosoftResearch/Naiad/</a><p>[1]: <a href="https://bigdataatsvc.wordpress.com/" rel="nofollow">https://bigdataatsvc.wordpress.com/</a>
So... After following the link I am dropped in the middle of the development diary of some project. After skimming the notes I can see that the project is<p>a) related to data processing<p>b) has not yet reached the stage of practical applicability<p>c) is fun to hack on<p>And not much more. Could someone provide more context?