Hey HN! We just released the beta version of collie.ai (<a href="https://collie.ai" rel="nofollow">https://collie.ai</a>).<p>Collie.ai recursively scrapes your website's entire contents and files, then gives you a search bar that seamlessly integrates into your website's design.<p>It uses scrapy to build the spider (<a href="https://github.com/scrapy">https://github.com/scrapy</a>) then indexes them all into the mixpeek API (<a href="https://mixpeek.com" rel="nofollow">https://mixpeek.com</a>), which is powered by Lucene's KNN (<a href="https://issues.apache.org/jira/browse/LUCENE-9004" rel="nofollow">https://issues.apache.org/jira/browse/LUCENE-9004</a>)<p>I've been in the search engineering space for quite some time. One thing I noticed is how tedious integrating full text search is. You have to build the scrapers, parsers, indexer, embedding service, then API. Not to mention the mobile responsive UI to make it all searchable.<p>We're the collie team and we have some exciting features coming out:<p>- Personalization and recommendations: using learn-to-rank systems to re-order results based on user interest<p>- Analytics: understand how users are interacting with your search<p>- A/B Testing: integrate search into your conversion funnel to optimize for what you care about<p>many many more.<p>If you want a white-label collie search bar for your internal, secure data (Drive, S3, Box, etc.) head over to collie.ai/beta