PipelineDB 1.0 – High-Performance Time-Series Aggregation for PostgreSQL

PipelineDB = Insert data with time component to be aggregated on the fly into always up-to-date summary tables using a variety of aggregation functions. Raw data is not persisted.<p>TimescaleDB = Store data with time component into &quot;hypertable&quot; that is automatically partitioned by time, for faster queries when limited by time range. Single node and has helper methods to make time based bucketing and aggregation easier.<p>Citus = Store data in distributed tables automatically partitioned and spread across multiple nodes, by any single column. Join across nodes with non-distributed tables.<p>Can definitely use PipelineDB for real-time summaries and TimescaleDB or Citus for raw long-term storage in the same database.<p>Side note: It would be nice if Postgres had package manager for extensions.

Has anyone tried to mix pipelinedb with timescale[1] , I think both are working on different side of playing with timeseries data ?<p>[1]<a href="https:&#x2F;&#x2F;www.timescale.com&#x2F;how-it-works" rel="nofollow">https:&#x2F;&#x2F;www.timescale.com&#x2F;how-it-works</a>

I&#x27;ve been following Pipeline since the beginning and it&#x27;s so fricking cool. Please, if you can&#x27;t think of a good use of Pipeline, use it instead of a count(*)! :D

Interesting, this seems to be the other side of the postgres time series extension coin.<p>TimescaleDB for writes, PipelineDB for reads.

How does this compare to TimescaleDB?<p>Are they solving the same problem in different ways or are they complementary projects? If it&#x27;s the latter, what would that look like?

Hoping this gains some traction as a defacto extension for cloud hosted postgresql. I think this is probably as useful as plv8 for a lot of use cases.

And it supports Postgres 10.x!<p><a href="http:&#x2F;&#x2F;docs.pipelinedb.com&#x2F;installation.html#install-postgresql" rel="nofollow">http:&#x2F;&#x2F;docs.pipelinedb.com&#x2F;installation.html#install-postgre...</a><p>Can&#x27;t wait for Postgres 11 support.

Looking over this cursorily, looks super cool.<p><pre><code> INSERT INTO events_stream (ts, value) VALUES (now(), &#x27;0ef346ac&#x27;); </code></pre> &gt; As soon as the continuous view reads new incoming events and the distinct count is updated the reflect new information, the raw events will be discarded.<p>So you create a table, insert into it, and it&#x27;s always empty. Is that right?<p>Does this work for any table in pg? How does pg know that the insert should NOT actually insert a row?

Fantastic guys, thank you! I’ve been looking forward to it becoming an extension for half a year. This is great news.<p>This basically means Postgres now has continuous views and a toolkbox of functions for running calculations. Combined with PG11 partitioning features and better parallel gusty execution, PG is an even more formidable choice for medium sized data.

I work closely in the space of providing time series databases as managed solutions. I can say that I am very happy to see this recent development of new tsd&#x27;s and this with timescale is a huge bump to the industry&#x2F;segment. Everybody currently measures some analytics and mostly user data and there is so much abuse with it, yet there is so much more you can measure and do and it is still very early stage. Farms, industrial applications, IoT and so much more. I&#x27;d love to just measure temperature and wind speed at unprecedented resolution.

Very interesting. Does it aggregate per day? If so, I wonder how it handles time-zone, I mean when to create a new day when you have agents on different time-zones.

Running functions on top of the transaction log(in transaction order) is a really powerful thing.

Is most of the intermediate processing done in memory, or is it limited by hd write speed?

How does this compare to Citus?

Congrats!<p>Also, how&#x27;s stride.io doing?

Big question for me: does it work on Heroku postgres?