I tried Meilisearch, unfortunately despite being build on LMDB it does not work at scale.<p>For any collection above 1.5M documents, insert times are crazy, like 60 seconds to index additional 6000 documents. There is something very wrong with the core data model and single writer LMDB bottlenecks IMO.
Co-founder and Tech Lead here,<p>Thanks to Louis' work and the design of LMDB, we can efficiently use the virtual address space of the OS to let it manage in the best way the memory Meilisearch is using.<p>We continuously improve the indexing speed, index size, memory usage, language support, and search speed. Perfection is a goal, and we will eventually reach it. Unfortunately, computer science is the science of trade-offs, and that's the hard part: choosing the most critical feature. We decided that search speed and accuracy are.