Hi folks, Quickwit cofounder here.<p>We started Quickwit 3 years ago with a POC, "Searching the web for under $1000/month" (see HN discussions [0]), with the goal of making a robust OSS alternative to Elasticsearch / Splunk / Datadog.<p>We have reached a significant milestone with our latest release (0.7) [1], as we have witnessed users of the nightly version of Quickwit deploy clusters with hundreds of nodes, ingest hundreds of terabytes of data daily, and enjoy considerable cost savings.<p>To give you a concrete example, one company is ingesting hundreds of terabytes of logs daily and migrating from Elasticsearch to Quickwit. They divided their compute costs by 5x and storage costs by 2x while increasing retention from 3 to 30 days. They also increased their durability, accuracy with exactly-once semantics thanks to the native Kafka support, and elasticity.<p>The 0.7 release also brings better integrations with the Observability ecosystem: improvements of the Elasticsearch-compatible API and better support of OpenTelemetry standards, Grafana, and Jaeger.<p>Of course, we still have a lot of work to be a fully-fledged observability engine, and we would love to get some feedback or suggestions.<p>To give you a glance at our 2024 roadmap, we planned to focus on Kibana/OpenDashboard integration, metrics support, and pipe-based query language.<p>[0] Searching the web for under $1000/month: <a href="https://news.ycombinator.com/item?id=27074481">https://news.ycombinator.com/item?id=27074481</a><p>[1] Release blog post: <a href="https://quickwit.io/blog/quickwit-0.7" rel="nofollow">https://quickwit.io/blog/quickwit-0.7</a><p>[2] Open Source Repo: <a href="https://github.com/quickwit-oss/quickwit">https://github.com/quickwit-oss/quickwit</a><p>[3] Home Page: <a href="https://quickwit.io" rel="nofollow">https://quickwit.io</a>
> To give you a concrete example, one company is ingesting hundreds of terabytes of logs daily and migrating from Elasticsearch to Quickwit. They divided their compute costs by 5x and storage costs by 2x while increasing retention from 3 to 30 days<p>I guess that's to be expected. Almost anything is more storage-efficient than Elasticsearch, FTS is so expensive.
Looking at the docker compose, this seems like a very complicated tool to run.<p>You'd need Kafka, zookeeper, and Jaeger. All would need to be HA. Then also this service. Not mentioning postgres because in theory you can use aurora or the like.<p>How quick have your current customers been able to get up and running so far? And how much maintainence have they needed?
What I am really missing in these (really nice) alternatives to ELK is Kibana and its Lens.<p>I tried to tapping this with Grafana but never learned to have graphs as easily as with Kibana. Maybe I was not trying hard enough?<p>Has anyone replaced Kibana with Grafana for non time based graphs?
I am impressed with this work, especially given our current use of Loki with Prometheus and Grafana.<p>Few questions come to mind. Firstly, is Quickwit compatible with any S3 compatible object storage, such as Cloudflare's R2? Are there particular considerations to keep in mind for this kind of setup? Secondly, do you see Quicwit being used for analytics, such as tracking daily visits or analyzing user retention?<p>Your insights on these would be greatly appreciated.
When open source providers say 'alternative' to a commercial solution, do they consider the serious engineering needed to scale such systems? I mean I used SigNoz, and it's comparable to Datadog feature wise but nowhere close in performance and scalability.
Glad this is getting some love. This is seriously good software. Have you guys supported generic substring search yet? I recall it was not supported as of a few months ago.
Congrats on the launch, we'll have to get you integrated with Rootly :). We can enable incident responders to fetch metrics while they respond to incidents in Slack!
I find presenting this as an open source alternative to commercial solutions a little disingenuous when any commercial use of it also requires a paid license. Like many other cases it seems like the AGPL is functioning more as a trial license.