I'm the author of the blog post.<p>The focus of the article really is about JNI in Rust.<p>I see most questions are about "Why did you not use X language instead?", so let me try and address this.<p>To answer the "Why not just Rust", I should first mention that Rust was still in its early days (before 1.0), and it was a risky bet to choose an emerging language.<p>The project was started by Vlad (our CEO) who had a background writing high performance Java in the trading space. The Zero-GC techniques - whilst uncommon in open source software - are mature and a staple of writing high performance code in the financial industry. The product evolved organically, feature after feature.<p>I personally joined the team from a C and C++ background, having previously moved from a project that suffered from minute-long compile times from single .cpp files due to template overuse. Whilst I do miss how expressive high-level C++ can be, Java has really good tooling support.<p>When writing systems-style software most of what matters in terms of performance is how we call system calls, manage memory and debug and profile. This is an area where Java really shines. Don't get me wrong: In the absolute sense I think C++ tools tend to be better (Linux Perf is awesome!), but Java tooling is _there_. IntelliJ makes it trivially easy to run a test under the debugger reliably and consistently. It's equally easy to run a profiler and to get code coverage. The same tools work across all platforms too, might I add. It's not necessarily better, but it's easier. Turns out that while a little quaint, using Java turned out to be a pretty good choice in my opinion in practice.<p>Times have moved on. The Rust community really cares about tooling, and it's one of the reasons why we've picked it over expanding our existing C++ codebase: We just want to get stuff done and have enough time left in our dev cycle to properly debug and profile our code.
> We seldom use the new keyword and objects are designed to be pooled and reused.<p>Just note that depending on the selection of the GC, this kind of usage may make the GC work <i>more</i> than when allocating new objects, not less. In particular, with the newer GCs -- G1 and ZGC -- mutating existing objects may be more costly than allocating new ones depending on circumstances. In general, the new GCs are optimised to work the least and give the best performance when the allocation rate is neither too high nor too low; the new GCs also reuse memory better than object pools. Reusing objects also precludes scalarization optimisations, i.e. not every `new Foo` actually results in a heap allocation, and can be optimised to work directly in registers.<p>So while on very old JVMs (such as Java 8) a "zero allocation" strategy may result in better performance and in "zero GC", on newer JVMs it may result in worse performance and <i>more</i> GC work (in fact, it will almost surely not yield zero GC). While it depends on many variables, I would advise against a zero allocation strategy on newer JVMs as the default path toward better performance or even better latency. It's an approach that seems to be very strongly coupled to the way the JVM was designed over a decade ago, but a lot has changed since then.<p>Additionally, Java now offers manual memory management and efficient FFI that are significantly better than what JNI offered: <a href="https://openjdk.org/jeps/442" rel="nofollow noreferrer">https://openjdk.org/jeps/442</a>
I mean no disrespect to the authors, but this seems like an extremely painful way to write an application. How common is it to write Java apps this way? It seems like there must have been a better language choice than trying to work against the GC and rewriting parts of the standard lib? And now adding in Rust via JNI? It just feels very painful to me.
Very interesting to see how the Rust-JNI interface gets used in a production environment (e.g., the topic of unifying logging typically doesn't come up in "your first Rust-JNI app" tutorials, for instance)<p>I do have one question around the assignment to `static mut CALL_STATE`. Don't you need some form of synchronization/memory fence/memory barrier to make sure that other threads see that assignment?<p>On x86/x64 it probably doesn't matter (total store order), but other architectures are less lenient.
I'm not sure I understand why they're using java if they avoid GC? Doesn't sound like the best fit, especially the foreign memory in java isn't too pleasant to work with.
I've never heard of QuestDB until this post, but I very much like what you guys are doing.<p>InfluxDB 1.x had a chance to be great: the wrote a time series database that used a SQL-like dialect, they had a really nice alerting platform, they had a really nice visiualization tool. Then they abandoned all of that to jump on the hype train of "Write a new programming language!" which was the hot thing like 5 or 6 hype cycles ago. We've _never_ upgraded to their 2.x product because it literally threw away our investment.<p>I think if you guys get pick up where they departed you'll be tremendously successful.
> JNI<p>Have you guys benchmarked FFI in Java 21 (preview, now release) yet? :) Yes I know it's super new, but I'm curious if there is a benefit in terms of:<p>1. performance<p>2. ease of maintenance<p>3. ease of finding production problems
Nowadays there's a much better technology than JNI for doing this sort of thing in Java.<p><a href="https://docs.oracle.com/en/graalvm/jdk/21/docs/reference-manual/native-image/" rel="nofollow noreferrer">https://docs.oracle.com/en/graalvm/jdk/21/docs/reference-man...</a>