Building Observability with ClickHouse

I see lot of hype around ClickHouse these days. Few years ago I remember TimescaleDB making the rounds, arguably being predecessor for this sort of "observability on SQL" thinking. The article has short paragraph mentioning Timescale, but unfortunately it doesn't really go into comparing it to ClickHouse. How does HN see the situation these days, is ClickHouse simply overtaking Timescale on all axis? That sounds bit of a shame; I have used Timescale a bit and enjoyed it, but just on such small scale that it's operational aspects did not really come up.

Interesting read, I was comparing some of these tools earlier for small web shop use while I didn&#x27;t proceed to setup any of them just yet. Demoed Elastic, SigNoz and Grafana Loki, of which Alloy+Loki seemed to make most sense for my needs and didn&#x27;t cause too much headache setting up on a tiny VM, so that I would have collection going in the first place and a decent method to grep through it.<p>Currently collecting just exception data from services to GlitchTip (Sentry fork), seems most valuable sysadmin-wise while having most security etc. concerns outsourced to managed hosting companies.<p>Was left curious what anomaly detection methods Elastic has built-in would take to DIY &lt;<a href="https:&#x2F;&#x2F;www.elastic.co&#x2F;guide&#x2F;en&#x2F;machine-learning&#x2F;current&#x2F;ml-ad-finding-anomalies.html" rel="nofollow">https:&#x2F;&#x2F;www.elastic.co&#x2F;guide&#x2F;en&#x2F;machine-learning&#x2F;current&#x2F;ml-...</a>&gt; with data frame &#x2F; statistics &#x2F; ML libraries (Clojure Noj).

Interestingly, I recently interviewed Samuel Colvin, Pydantic&#x27;s author, and he said when designing his observability Saas called LogFire, he tried multiple backends, including ClickHouse.<p>But it didn&#x27;t work out.<p>One of the reasons is LogFire allows the users to fetch the service data with arbitrary SQL queries.<p>So they had to build their own backend in rust, on top of DataFusion.<p>I used ClickHouse myself and it&#x27;s been nice, but it&#x27;s easy when you get to decide what schema you need yourself. For small to medium needs, this plus Grafana works well.<p>But I must admit that the plug and play aspect of great services like Sentry or LogFire make it so easy to setup it&#x27;s tempting to skip the whole self hosting. They are not that expensive (unlike datadog), and maintaining your observability code is not free.

ClickHouse + Grafana is definitely a fantastic choice, here is another blog from ClickHouse talking about dogfooding their own technology and save millions:<p><a href="https:&#x2F;&#x2F;clickhouse.com&#x2F;blog&#x2F;building-a-logging-platform-with-clickhouse-and-saving-millions-over-datadog" rel="nofollow">https:&#x2F;&#x2F;clickhouse.com&#x2F;blog&#x2F;building-a-logging-platform-with...</a><p>(Full disclosure: I work for ClickHouse and love it here!)

Another project I want to give shout out to is Databend. It&#x27;s built around the idea of storing your data at S3-compatible storage as Parquet files, and querying as SQL or other protocol.<p>Like many popular Data Lake solutions, but it&#x27;s open-source and written in Rust, which means quite easy to extend for many who know it already.

Such a PITA. Unless you have a dedicated team to handle observability, you are in for pain, no matter the tech stack you use.

There is at least one basic factual error in this blog post, which makes me discount the whole thing.<p>&quot;But if you will use it, keep in mind that [InfluxDB] uses Bolt as its data backend.&quot;<p>Simply not true. The author seems to have confused the storage that Raft consensus uses for metadata with that used for the time series data. InfluxDB has its own custom data storage layer for time series data, and has had so for many years. A simple glance at the InfluxDB docs would make this clear.<p>(I was once part of the core database team at InfluxDB and have edited my comment for clarity.)

Completely rewriting a system because you don&#x27;t like JSON is a bit extreme