How does Greptime handle dynamic schemas where you don't know most of the shape of the data upfront?<p>Where I work, we have maybe a hundred different sources of structured logs: Our own applications, Kubernetes, databases, CI/CD software, lots of system processes. There's no common schema other than the basics (timestamp, message, source, Kubernetes metadata). Apps produce all sorts JSON fields, and we have thousands and thousands of fields across all these apps.<p>It'd be okay to define a small core subset, but we'd need a sensible "catch all" rule for the rest. All fields need to be searchable, but it's of course OK if performance is a little worse for non-core fields, as long as you can go into the schema and explicitly add it in order to speed things up.<p>Also, how does Greptime scale with that many fields? Does it do fine with thousands of columns?<p>I imagine it would be a good idea to have one table per source. Is it easy/performant to search multiple tables (union ordered by time) in a single query?
Am I the only one that got, "This article smells like it was written by an AI told to 'compare these two products'"?<p>Something around the sentence structure just is offputting.
Any reason to use this like in Azure over their cloud native options such as with AKS that has fluentd built into the ama-pod? It already sends logs to Azure Monitor/LogA. Azure Managed Grafana can take in Kusto queries. AMA can monitor VMs. Further you can use DCE/DCRs for custom logs. Azure provides Azure native ElasticSearch too. It seems to own this market.<p>You can predictably control costs and predict costs with these models.
For logs I'd be more likely to choose <a href="https://www.gravwell.io" rel="nofollow">https://www.gravwell.io</a> as it's log agnostic and I've seen it crush 40Tb/s a day, whereas it looks like greptime is purpose-tuned for metrics and telemetry data.
Reading the web site, I just noticed the open-source version does not have "Log query endpoints".<p>Does that mean you have to use SQL (or the visual SQL builder) to query logs, and you don't get access to a log query language the way Kibana gives you KQL and Lucene syntax?<p>If so, I think it's a little disingenuous to write an article comparing the ELK stack, which <i>is</i> open source and comes with a perfectly usable query UI, to Greptime's equivalent, which is not.