Roaming RAG – Make the Model Find the Answers

RAG is a pain to set up, so I tried something different. Instead of dealing with vector DBs and all that complexity, just let the LLM navigate well-structured docs like a human—exploring outlines and diving into sections. It’s simple, and works great for stuff like technical manuals or llms.txt.

I independently "invented" this 2 months ago, also for parsing 100 page long PDFs. Give each page a title and summary, but prepend as context the title+summary of previous 3 pages. In the end combine all titles+summaries to get a master index using the LLM. At runtime you first use the index to zoom into the desired sections to give the answer. I think this makes the RAG "smarter", not based on simple similarity.