I have been working on AI products for some time, and there is always a RAG / search component to build.<p>With the plethora of tools out there by now, it's now pretty instinctive to set up with a basic stack.<p>However, I did see feedback from end users mentioning the search part didn't provide good results. And I also saw several engineering teams going from a standard stack to something way more complex.<p>Also, there always seems to be edge cases where hybrid search doesn't cut it. (Like asking "search for contracts that DO NOT mention a termination fee" for example).<p>I'm curious to understand what the community thinks about retrieval systems:<p>- is it still considered to be a hard problem?
- are teams actively trying to solve it, if so how?
- will we ever have a system that basically can find any information across any corpus even with minimal indications from users?