Hi HN, Mike and Warren here! We're excited to share some (early) work towards our next major version of HyperDX. HyperDX makes it easy to visualize/search logs & traces on top of Clickhouse (so incident & bug investigations hopefully go by a little easier). For example, if a team is thinking of migrating to Clickhouse for their observability data warehouse [1][2][3] usually due to cost or data privacy reasons, they can easily throw HyperDX on top to do the UI layer for analysis and dashboarding in a dev-friendly way (aka not needing to type paragraphs of SQL to find some logs)<p>Over the past year we've seen a ton of excitement in companies adopting Clickhouse-based observability stacks - but one of the biggest challenges we've seen is that the UI layer on top of Clickhouse is either clunky to use for observability use cases (ex. BI tools), or too tied to a specific ingestion architecture to scale to every use case (we used to be in this category!). For companies that needed more flexibility in how their data is ingested and stored (usually due to running at a large scale), there's really no good options for a developer experience (DX) focused observability layer on top of Clickhouse (Shopify spent 3 years building it in-house!)<p>Our current release works completely in the browser - and it does this by building on top of Clickhouse's HTTP interface, which our React app can directly talk to. This means you can actually try HyperDX in your browser on your own Clickhouse with no installation! This was fortunately easy for us to accomplish due to being full stack Typescript, making it incredibly easy to shift between server and client code. On top of this we've been spending time baking in performance optimizations to ensure that HyperDX can continue to leverage Clickhouse efficiently at larger data volumes. We do a few tricks like only fetching columns that are needed for the current search, and re-querying to expand the entire row if needed to fully leverage Clickhouse's columnar nature (40% faster, ymmv!) - or rewriting queries to use materialized columns to speed up Map column access when available (10x faster!).<p>On the DX side: we support querying using both Lucene (ex. `fullText property:value`) and SQL syntax. We've found the former to be our favorite for how concise it is. Similarly for charts, our chart builder has been upgraded to accept SQL expressions as well, so you can leverage the full power of SQL, while avoiding typing paragraphs of boilerplate SQL for time series data. We also make it easy to switch between UTC/local timestamps! Lastly, we've added high cardinality outlier analysis by charting the delta between outlier and inlier events (a la bubble up) - which we've found really helpful in narrowing down causes of regressions/anomalies in our traces.<p>We have a lot more planned for the full release - but wanted to get this out early to hear your feedback and opinions!<p>In Browser Live Demo: <a href="https://play.hyperdx.io/search">https://play.hyperdx.io/search</a><p>Github Repo: <a href="https://github.com/hyperdxio/hyperdx/tree/v2">https://github.com/hyperdxio/hyperdx/tree/v2</a><p>Landing Page: <a href="https://hyperdx.io/v2">https://hyperdx.io/v2</a><p>[1]: <a href="https://www.uber.com/blog/logging/" rel="nofollow">https://www.uber.com/blog/logging/</a>
[2]: <a href="https://blog.cloudflare.com/log-analytics-using-clickhouse/" rel="nofollow">https://blog.cloudflare.com/log-analytics-using-clickhouse/</a>
[3]: <a href="https://www.youtube.com/watch?v=LDj3_jMsCXg&list=PLvQF73bM4-5X9mt0lweCXL_v8xdvrLEvB" rel="nofollow">https://www.youtube.com/watch?v=LDj3_jMsCXg&list=PLvQF73bM4-...</a>
Can you clarify: Does the full-text search for logs linearly search all logs like Loki does, or can it speed it up with an index?<p>The docs at <a href="https://www.hyperdx.io/docs/search">https://www.hyperdx.io/docs/search</a> don't seem to talk about this key design decision.<p>I have a couple 100 GB to few TB logs (all from `journald` or JSON lines), just want to store them forever, and find results fast when searching for arbitrary substrings.<p>Loki does not use an index, so it's pretty slow at finding results in TB-sized logs (does not return results within a few seconds, so it's not interactive).<p><a href="https://quickwit.io" rel="nofollow">https://quickwit.io</a> is one thing I'm looking at integrating, that can solve much of the index-based log search.<p>(Note I'm not super familar with the capabilities of ClickHouse itself regarding indexed full-text search.)
Hmm, this is not the "Kibana" alternative I imagined.<p>Kibana was supposed to be an easy UI. You go to Discover, and the data automatically shows in chronological order, I can explore it with different options.<p>Kibana is very suitable for non-tech or less-tech people. I hope your product find a clear target audience. With too much ES query JSON or SQL it would scare people off.
A happy HyperDX customer here. Can't recommend it enough.<p>We wanted something good for tracing and logs, without the price tag we were used to from datadog. We've been pleasantly surprised by how easy it was to set up and start pumping telemetry.<p>The UI is super intuitive and the OOTB dashboards are great as well.
It is actually really great! Works out of the box, does it with a single-page UI, and it is not slow. It's very close to a log viewer I always dreamed of. The UI is much better than Grafana.<p>I connected it to the system.text_log table, and it took zero time with no problems.
I've had browser and Linux VM logging to HyperDX with great success, but have been struggling to get OTel logging working with Azure Functions. Turns out, new (currently in Preview) "Flex Consumption" functions [0] natively support OTel and work with HyperDX.<p>[0] <a href="https://azure.microsoft.com/en-us/updates/public-preview-azure-functions-brings-new-flexibility-with-azure-functions-flex-consumption/" rel="nofollow">https://azure.microsoft.com/en-us/updates/public-preview-azu...</a>
Neat! I was looking to replace DataDog with an open source alternative. I'm collecting the logs and batch sending them to DataDog using their batch http-intake API. I'm looking for the quickest way to switch over - is there anything similar on HyperDX?<p>Also, I'd like to improve my observability using OTel in Cloudflare Workers but it looks like the example is out of date using an deprecated library which points to a new one to use instead. Might be worth updating the docs on that when you get a chance.<p>Deprecated: <a href="https://github.com/RichiCoder1/opentelemetry-sdk-workers">https://github.com/RichiCoder1/opentelemetry-sdk-workers</a>
New: <a href="https://github.com/evanderkoogh/otel-cf-workers">https://github.com/evanderkoogh/otel-cf-workers</a>
HyperDX looks great! Are you going to add support for other backends (aka data sources) in order to become "Grafana for logs" solution? For example, I'd be glad to see the support for VictoriaLogs in HyperDX. It provides rich set of HTTP querying APIs, which could be used for building efficient UX at HyperDX - <a href="https://docs.victoriametrics.com/victorialogs/querying/#http-api" rel="nofollow">https://docs.victoriametrics.com/victorialogs/querying/#http...</a>
We use HyperDX for our observability! We had been using google's observability suite before, because we're using GKE anyways, but HyperDX's search over traces is just waaaay better and I can't go back.
Super neat! Does the v2 branding mean that the more "fully featured" observability product is going away? Or is it all going to be rebuilt on top of clickhouse?
If you want an opensource / non AGPL licensed alternative for Kibana, Opensearch also includes a fork of Kibana in the form of Opensearch Dashboards.<p>Clickhouse not being Elastic/Opensearch based means they would need to reinvent that wheel in any case because Kibana cannot use Clickhouse for storage. So this isn't so much an alternative but an essential component to make Clickhouse useful. Since you can't use Kibana for that. From various accounts here; they seem to have done a decent job.<p>Of course the key strength of Kibana is that it builds on features that Elasticsearch has; like aggregations that are probably more limited in Clickhouse. Same with Opensearch Dashboards. It depends on your use case whether you actually need that of course.<p>One point of concern with Clickhouse is that, like Elastic, they require contributors to sign contributor agreements. This basically allows them to re-license the code base if they want to at some point. Which is of course what Elastic did several times now (they changed it back to AGPL a few weeks back). Like Elastic are well funded by VC money but still pre-IPO. Just saying that if you moved to Clickhouse because of the Elastic licensing debacle, you might just have moved that problem instead of solving it.
Whats hilarious is that Kibana started out as a Open Source.<p>Hard to trust anything released as OSS these days that hits this site run by a for profit company.. Its all destined to have a rug pull after some VC funding. Considering HyperDX is a for profit company, I'm sure we won't have to wait long!