Apache Arrow Datafusion 5.0.0 release

The new core we&#x27;re building for InfluxDB (named InfluxDB IOx) uses Datafusion for query execution. We have multiple team members contributing to this release and we&#x27;re super excited to be involved with it.<p>I think it&#x27;s a really exciting time for new OLAP systems because of Arrow, Rust, and the rise of object store + ephemeral compute for analytical and time series data.

One of the Arrow Datafusion committers here. Happy to help answer any question.

I want to understand something, why use Spark and DataFusion? As the way to interact with them is SQL.<p>Why not just use traditional DBMS like PostgreSQL?<p>Are there explainer articles on this topic?<p>See Quora: <a href="https:&#x2F;&#x2F;www.quora.com&#x2F;unanswered&#x2F;Whats-the-advantage-of-doing-SQL-in-Spark-and-Apache-Arrow-DataFusion-vs-doing-SQL-on-a-traditional-DBMS-like-MySQL-or-PostgreSQL" rel="nofollow">https:&#x2F;&#x2F;www.quora.com&#x2F;unanswered&#x2F;Whats-the-advantage-of-doin...</a>

What&#x27;s the relationship between Datafusion and Ballista? They seem to have been merged into a single repo. Do they share a release schedule? Are they a single product or still separate?

How would you compare the goals, vision, and current status of DataFusion with DuckDB? (www.duckdb.org) Could DuckDB be an execution engine for Ballista?

Congrats to the launch! The python binding to datafusion is also soon to be released. Exciting time!