This doesn't seem to use local LLMs... so it's not really local. :-\<p>Is there a deep searcher that can also use local LLMs like those hosted by Ollama and LM Studio?
I’m curious how this compares to the open-source version made by HuggingFace [1]. As I can tell, the HF version uses reasoning LLMs to search/traverse and parse the web and gather results, then evaluates the results before eventually synthesizing a result.<p>This version appears to show off a vector store for documents generated from a web crawl (the writer is a vector-store-aaS company)<p>[1] <a href="https://github.com/huggingface/smolagents/tree/main/examples/open_deep_research">https://github.com/huggingface/smolagents/tree/main/examples...</a>
I think the magic of Grok's implementation of this is that they already have most of the websites cached (guessing via their twitter crawler) so it all feels very snappy. Bing/Brave search don't seem to offer that in their search apis. Does such a thing exist as a service?
Nice - I like people's different twist on Deep Research. Here is mine...with Flow I'm trying a new workflow.<p><a href="https://github.com/btahir/open-deep-research">https://github.com/btahir/open-deep-research</a>
Considering all the major AI companies have basically created the same deep research product, it would make sense that they focus on a shared open source platform instead.
There's two blog posts that go with this, check it out:<p><a href="https://milvus.io/blog/i-built-a-deep-research-with-open-source-so-can-you.md" rel="nofollow">https://milvus.io/blog/i-built-a-deep-research-with-open-sou...</a><p><a href="https://milvus.io/blog/introduce-deepsearcher-a-local-open-source-deep-research.md" rel="nofollow">https://milvus.io/blog/introduce-deepsearcher-a-local-open-s...</a>
Have been searching for a deep research tool that I can hook up to both my personal notes (in Obsidian) and the web and this looks like this has those capabilities. Now the only piece left is to figure out a way to export the deep research outputs back into my Obsidian somehow.
I actually tried using this and I came into some issues and I had to replace the openAI text embeddings with the MilvusEmbedding.<p><a href="https://gist.github.com/zitterbewegung/086dd344d16d4fd4b8931b7830d512b7" rel="nofollow">https://gist.github.com/zitterbewegung/086dd344d16d4fd4b8931...</a><p>The QuickStart had a good response. [1]
<a href="https://gist.github.com/zitterbewegung/086dd344d16d4fd4b8931b7830d512b7?permalink_comment_id=5456928#gistcomment-5456928" rel="nofollow">https://gist.github.com/zitterbewegung/086dd344d16d4fd4b8931...</a>
I'm wondering about the practical implications of integrating web crawling. Could this, in theory, be used solely for reading papers from Sci-Hub and producing valid graduate-level research?<p>It could be useful for comparing reports built using DeepSeek R1 vs. GPT-4o and other large models. The code being open source might highlight the limitations of different LLMs much faster and help develop better reasoning loops in future prompts for specific needs. Really interesting stuff.
Cloudflare is going to ruin self hosted things like this and force centralization to a few players. I guess we'll need decentralized efforts to scrape the web and be able to run it on that.