So, I am hitting limits with the amount of data I am sending to Claude via the API.<p>I am trying to send about 5000 comments but I would love to send more but I am hitting limits of the message size.<p>What are some good design patterns when sending large sets of data to an LLM?<p>I ideally need to send all the data together at it gives context to the overall prompt.
Some approaches we’ve used:<p>- Group the comments by theme, then pass groups to the LLM to summarize/deduplicate then pass the outputs of that as context.<p>- RAG where only relevant parts are included in the context.<p>- Use an LLM with a larger context window (like Gemini pro)