Looks pretty nice. The codebase is half typescript and half Java? Guessing that is because Kafka is written in Java.<p>Another question: how might this compare to the Confluent web GUI? Obviously this is open source which is great.
<a href="https://lenses.io/" rel="nofollow">https://lenses.io/</a> Kafka UI is pretty solid. Nice to see an open source alternative here.
There are several alternatives posted here already, but after comparing several ones maybe a year ago I found Kowl very nice: Nice UI, backend written in Go (quick startup, low memory usage), viewing and filtering of messages, support for schema registry, comes with a Docker image, usable locally and Helm chart for Kubernetes.<p><a href="https://github.com/cloudhut/kowl" rel="nofollow">https://github.com/cloudhut/kowl</a>
Neat! Just be wary around licensing, we’ve been using a similar management tool for a couple years <a href="https://github.com/yahoo/CMAK" rel="nofollow">https://github.com/yahoo/CMAK</a> which was originally Kafka manager but they were forced to rename it.
Where I work we use Confluent Cloud who has their own proprietary UI. I've always found it to be lacking, hard to use and not very good.
We substituted it with AKHQ <a href="https://akhq.io/" rel="nofollow">https://akhq.io/</a> , which is miles ahead of anything I've seen. The main issue with it are the interesting UX decisions that requires learning. For example, a lot of links require a double click, which isn't a common behavior in Web Apps. Besides that, it's absolutely wonderful and goes beyond just Kafka. We use Kafka Connect very heavily and AKHQ seems to implement the full CRUD for Connect.
I rolled out kPow initially for my team of 10 to help us manage and monitor our clusters.<p>We’ve now rolled it out to our 400+ engineers who use it every day to help them gain visibility and insight to their data and topics, manage offsets, amount other things. It handles HUGE clusters with no issue, and the RBAC features are great.<p>I’ve tried them all. While there are some ok alternatives, I honestly find them toys compared to kPow. Do yourself a favor and check it out!
We use Kafdrop for viewing topics/messages/info and it's really nice. It's read-only though, so if you need to configure a topic or push a message you have to use something else.<p><a href="https://github.com/HomeAdvisor/Kafdrop" rel="nofollow">https://github.com/HomeAdvisor/Kafdrop</a>
What is the difference between this and <a href="https://github.com/yahoo/CMAK" rel="nofollow">https://github.com/yahoo/CMAK</a>
I'm using kafdrop but modify it a bit. Reasons:
- the front page is very slow if there are a lot of topics (the kafka cluster I'm using has 6k topics). It's because kafdrop tries to get a lot of information of every topic even though it doesn't display them on the screen. I modified it like this: after display 6k topic names, I send ajax requests to get info (# of partitions, start & end offsets of each partition, number of messages in each partition) of 20 topics each.
- kafdrop doesn't have the feature to send a message to a topic. I added a text area below each topic name so that I can type the mesaage there and send.
- If there's a new message, kafdrop cannot tell me which partition that new message goes to. So I added this feature: suppose that a topic has 3 partitions with 3, 4 and 5 messages. If there are some new messages coming to that topic, I can refresh that topic in the UI and the UI will show something like partition 2: 3 -> 4 messages. So I know that 1 new message goes to partition 2.<p>I like the multi-cluster management feature of this project. Unfortunately I don't know react (angular + typescript is ok with me), so I'm wondering if I should try this one knowing that I cannot modify it.
The reasons I modified kafdrop are also the features I like to see in a new Kafka UI.
A few pieces of feedback.<p>Positive:<p><pre><code> - Multi-cluster support is a great feature! That's the main thing missing from Kowl, in my opinion
- The UI feels clean
- Message streaming feels smooth, and the search feature is a great addition. Also love the ability to see a message for a given partition/offset, and that JSON messages are nicely rendered as JSON!
</code></pre>
Negative/Feature Requests:<p><pre><code> - The CleanupPolicy logic doesn't handle "delete,compact" (I see bug number 829 addressed a slightly related issue, maybe that logic could be expanded [1])
- When viewing Cluster -> Topic -> Consumers, it seems like far more is being loaded than just consumer groups for that particular topic. It takes much longer to load than, e.g., Kowl does for pulling the same information
- Similar feedback for when clicking on an individual consumer group (example URL path: "/ui/clusters/dev/consumer-groups/my-group"). It takes a very long time to load
- If any permissions issues are encountered while loading the individual consumer group, the entire request fails. That wouldn't necessarily be an issue, but it is when you're loading all consumer groups, and not just the one requested
- I don't see any docs on how to access a topic that is secured with certificates (Kowl's relevant config [2])
- If a schema registry uses a self-signed certificate (or one signed by a corporation's CA), there's no way to pass the certificate through a config or skip validation (Kowl's relevant config [3]). As it is, you get a 500 error in the API call, but the UI doesn't show an error
</code></pre>
Overall, it seems like a fair bit of thought went into this! But there are a few quirks that make it very tough to use in a corporate environment with (perhaps excessively) large Kafka clusters with hundreds/thousands of topics, or secured with internal certificates<p>1. <a href="https://github.com/provectus/kafka-ui/blob/master/kafka-ui-api/src/main/java/com/provectus/kafka/ui/model/CleanupPolicy.java#L9" rel="nofollow">https://github.com/provectus/kafka-ui/blob/master/kafka-ui-a...</a><p>2. <a href="https://github.com/cloudhut/kowl/blob/10776b81e45abfa716548d171d7f7e1326bf8553/docs/config/kowl.yaml#L56-L61" rel="nofollow">https://github.com/cloudhut/kowl/blob/10776b81e45abfa716548d...</a><p>3. <a href="https://github.com/cloudhut/kowl/blob/10776b81e45abfa716548d171d7f7e1326bf8553/docs/config/kowl.yaml#L68-L73" rel="nofollow">https://github.com/cloudhut/kowl/blob/10776b81e45abfa716548d...</a>
<a href="https://kpow.io" rel="nofollow">https://kpow.io</a> - we build kPow for Apache Kafka, a drop-in replacement for Confluent Control Center + much more.<p><a href="https://kpow.io/features" rel="nofollow">https://kpow.io/features</a><p>- Multi-Cluster, Multi-Schema, Multi-Connect.<p>- Multi-Tenancy, RBAC, all the Authz - LDAP, SAML, etc.<p>- Very fast streaming multi-topic search (~1M messages / minute)<p>- Built in JQ search filters (we implemented a subset of JQ in the browser and JVM)<p>- Live Kafka Streams / Consumer Group topology visualisation (including RocksDB metrics for state stores, etc)<p>- Offset management, resetting, scheduled mutations.<p>- Consumer Group read and Topic write freshness / n-minutes inactivity metrics.<p>- Heaps of admin functions like Staged Mutations, Temporary RBAC policies, Data Masking for PII, etc.<p>- Prometheus / OpenMetrics endpoints.<p>- kREPL (slightly mad programmatic interface to Kafka)<p>- 7k commits, 100k docker pulls, 97.4% Clojure.<p>- Unlimited users, unlimited features.<p>- Commercially supported, annual licenses or available as hourly-metered (0.16c/hr) on the AWS Marketplace or Red Hat Marketplace.