Sonic: Fast, lightweight and schemaless search back end in Rust

Always nice to see more alternatives to Elasticsearch. That project could be so much better with some proper planning and focus.<p>There&#x27;s also Toshi: <a href="https:&#x2F;&#x2F;github.com&#x2F;toshi-search&#x2F;Toshi" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;toshi-search&#x2F;Toshi</a> which is built on top of Tantivy: <a href="https:&#x2F;&#x2F;github.com&#x2F;tantivy-search&#x2F;tantivy" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;tantivy-search&#x2F;tantivy</a><p>And for C++, there&#x27;s Xapiland: <a href="https:&#x2F;&#x2F;github.com&#x2F;Kronuz&#x2F;Xapiand" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;Kronuz&#x2F;Xapiand</a><p>And for Go, there&#x27;s Blast: <a href="https:&#x2F;&#x2F;github.com&#x2F;mosuka&#x2F;blast" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;mosuka&#x2F;blast</a> built on Bleve: <a href="https:&#x2F;&#x2F;github.com&#x2F;blevesearch&#x2F;bleve" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;blevesearch&#x2F;bleve</a>

Huh. When this was initially posted it had a weird and commercially restrictive license, but it looks like that&#x27;s been reverted, possibly after (polite) discussion on &#x2F;r&#x2F;rust. It&#x27;s MPL 2 now.<p><a href="https:&#x2F;&#x2F;github.com&#x2F;valeriansaliou&#x2F;sonic&#x2F;issues&#x2F;52#issuecomment-475911660" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;valeriansaliou&#x2F;sonic&#x2F;issues&#x2F;52#issuecomme...</a>

Performance figures are awesome. Also language support is great.<p>&gt; Sonic only keeps the N most recently pushed results for a given word<p>This index discards old entries. This is fine for messages, in which aging items lose relevance. Yet the developer uses it for a help desk, which I think should give equal importance to all items.<p>In this area I would say the main comperitor is Groonga. It can be integrated into PostgreSQL with the PGroonga extension, and it indexes all of the data. However it consumes way more ram.

Wow. I’ve spent the last three weeks building a custom search solution in Kotlin - in my case I’m using tokenizers from Lucene and using a radix trie as an index. I actually looked at using Bleve (another rust search lib) initially but it didn’t have the right language support<p>Glancing over this it looks like a nearly perfect fit for my use case I just wish I had seen this a couple of weeks earlier!

Talk about perfect timing!<p>I was looking for something just like this for a project in my team. We had been using this setup where a huge chunk of the data was being stored in triplicate: some of it in ES, some more of it in another database and finally the whole dataset in our data warehouse.<p>Hopefully I can use this to only provide the index + full text capability and just use the warehouse itself as the main db because the query performance is similar enough and the warehouse is criminally underused for what we pay for it.

This looks like a breath of fresh air. Elasticsearch won’t even start if it can’t preallocate 2 GB

I would like to understand the code for the project. What approach would help?

This looks pretty cool! I&#x27;d like to try it, but it looks like I&#x27;ll have to wait for the Python client library first.