Like many of you I use ChatGPT for specific questions, completing a function from comments, etc. But I'm reading that LLMs will soon become actual developers.<p>How can that be? Let's forget about quality, hallucinations, etc. The largest context window from an accessible/affordable LLM is 32k (Mixtral or GPT4). That's barely enough for a TODO app, let alone a real project. The smallest project I work on, a desktop app, has 60k LOC/6M characters/1.5M tokens.<p>So what changes are coming that would allow an LLM modify an existing codebase, e.g. to modify a feature and write its tests? (without having to spoonfeed it the perfect context the way we do now in ChatGPT)
Your question is posed as a hypothetical, but the problem is already solved...<p>Add a dependency graph of different agents and tools. Use summarization (either selecting subsections or rewriting). Give it a scratch space. Use RAG.<p>Why would it need to load the whole code base into memory? We can build very complex architectures on top of this task that mix LLMs with software.<p><a href="https://arxiv.org/abs/2402.09171" rel="nofollow">https://arxiv.org/abs/2402.09171</a><p>This isn't hypothetical; all of these