I really like the look of vortex[1]! One of my industry pet peeves is all the useless utf-8 server log bytes. I'd like to log data in a sane, schemaful, binary format and this looks like it could be a good way to do that. Bonus points if we can wire this up as a physical layer for e.g. datafusion[2] so I can analyze my logs with the dataframe abstraction.<p>EDIT: Question about FSST--lets say I build a strings table like:<p><pre><code> struct Strings {
compressor: fsst::Compressor,
compressed: Vec<Vec<u8>>
}
</code></pre>
Is there some optimal length for compressed given the 255 symbols limit?<p>[1] <a href="https://github.com/spiraldb/vortex">https://github.com/spiraldb/vortex</a>
[2] <a href="https://github.com/apache/datafusion">https://github.com/apache/datafusion</a>