Differential Dataflow

Differential Dataflow is about differential computation which itself is a generalization of incremental computing, most commonly known from Excel.<p>A good introduction to the overall topic is incremental [0] an OCaml library by Jane Street and the corresponding blog posts and videos [1]. They even use it for webapps [2].<p>[0]: <a href="https:&#x2F;&#x2F;github.com&#x2F;janestreet&#x2F;incremental" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;janestreet&#x2F;incremental</a><p>[1]: <a href="https:&#x2F;&#x2F;www.janestreet.com&#x2F;tech-talks&#x2F;seven-implementations-of-incremental&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.janestreet.com&#x2F;tech-talks&#x2F;seven-implementations-...</a> (Great to understand fundamentals of an incremental computation graph)<p>[2]: <a href="https:&#x2F;&#x2F;www.janestreet.com&#x2F;tech-talks&#x2F;intro-to-incr-dom&#x2F;" rel="nofollow">https:&#x2F;&#x2F;www.janestreet.com&#x2F;tech-talks&#x2F;intro-to-incr-dom&#x2F;</a>

This is also the tech powering Materialize, which is an awesome materialised view maintenance engine with a Postgres like interface:<p><a href="https:&#x2F;&#x2F;materialize.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;materialize.io&#x2F;</a>

I was kind of caught up on comment &quot;if (m2, m1) and (m1, p), then output (m1, (m2, p))&quot;. When input is [(m2, m1), (m1, p)] and is pairwise switched and joined with unreversed input.<p>If m2 manages m1 and m1 manages p then m2 manages p via m1, but I could not quite understand what this example output should mean. M1 manages m2 and p? M1 is managed by m2 and p. Neither seems correct.

What are people here using differential dataflow for?

How is this different than for example rule engines where you have certain inputs, a calculation&#x2F;solver engine and then you get the outputs for it?

this is also one of the main ideas behind Noria[1] (built by @jonhoo, who also gave his PhD dissertation[2] on this recently)<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;mit-pdos&#x2F;noria" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;mit-pdos&#x2F;noria</a><p>[2] <a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=GctxvSPIfr8" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=GctxvSPIfr8</a>

This sounds like streaming and what kafka streaming has been doing for a while. Everything is eventually consistent and what&#x27;s served up is the current state at the time of the request.

Can someone explain in simple english what is the difference between this and Automatic Materialized as implemented in SqlServer and Oracle