I really think this is where pure FP shines.<p>If you look at the architecture of something like Apache Beam, while you describe your computations in a language like Java or Python, you're really using a DSL for creating an immutable DAG that processes chunks of immutable data, persisiting each chunk (being loose with terms here) for durability, 'modifying' it in the immutable sense and then passing it on to the next edge and so on.<p>In a single standalone system, many argue that a purely immutable conceptual approach has pros and cons. In the Big Data world, I've never heard anyone argue for an imperative approach. Immutability is necessary, and as soon as you need immutability you want all the other goodies that facilitate it in a clean way like monads, lenses, and the like.
I really hope Unison succeeds. It looks like a really interesting take on FP / distributed systems. The fact that Paul Chiusano and Runar Bjarnason are at the helm is exciting, especially to Scala / FP devs like me who know them as the writers of the legendary "red book". Just based on the rare quality of that book, I'm super interested in other FP projects that they're involved in.
I think there were a lot of great ideas presented in these small examples.<p>1. Explicitly marking non-local _data_ vs. in-memory. A very cool idea -- languages and libraries I've used all seem to try to make this transparent (unsuccessfully, as indicated by the random serialization errors you get)<p>2. Making the noise around serialization and RPC transparent to the user<p>3. Bring the code to the data, not the data to the code<p>4. Immutability, FP, and I like the syntax -- no coincidence in its similarities to Haskell and Scala, I'm sure :)<p>These are all pain points in a lot of existing "big data systems" I've used, where they either solve the problem half-baked or don't address at all. I'm excited to see where this project goes!
Recommend starting with part 1 of the article if you want more of an overview: <a href="https://www.unison-lang.org/articles/distributed-datasets/" rel="nofollow">https://www.unison-lang.org/articles/distributed-datasets/</a>
I dislike the term "immutable data structure", because it has two widely used conflicting meanings:<p>1. Read-only data structures, which are used by everyone in every language all the time.<p>2. Persistent data structures, which make the underlying immutable structures kind of mutable by creating new versions and reusing parts from the old versions of the structure.