MapReduce and Spark

I like Spark over Hadoop just from an interface point of view, particularly the ability to just start up a (Scala) shell and start playing around. Hadoop can be very effective, but even getting "hello world" to run requires an intimidating array of setup.

I often read that spark avoids the costly synchronization required in mapreduce, since it uses DAG's. Can someone explain how is that achieved. If the application so demands that you can launch jobs together, that can be done even with hadoop/mapreduce. If one job requires the output of another, then the job has to wait for synchronization whether its mapreduce or DAG.

Some interesting use cases are also described on Cloudera&#x27;s developer blog, at <a href="http://blog.cloudera.com/blog/2013/11/putting-spark-to-use-fast-in-memory-computing-for-your-big-data-applications/" rel="nofollow">http:&#x2F;&#x2F;blog.cloudera.com&#x2F;blog&#x2F;2013&#x2F;11&#x2F;putting-spark-to-use-f...</a>.

Although spark is nice, I&#x27;m also looking forward to mpi&#x2F;orted integration with hadoop...<p>&quot;Performance: Launches ~1000x faster, runs ~10x faster&quot;<p>&quot;Launch scaling: Hadoop (~N), MR+ (~logN)&quot;<p>&quot;Wireup: Hadoop (~N2), MR+ (~logN)&quot;<p><a href="http://slurm.schedmd.com/slurm_ug_2012/MapRedSLURM.pdf" rel="nofollow">http:&#x2F;&#x2F;slurm.schedmd.com&#x2F;slurm_ug_2012&#x2F;MapRedSLURM.pdf</a>

What I would love to know is if Mahout works out of the box with Spark or if there&#x27;s a third party library that bridges the two.