(Not an expert, just a fellow JS dev)<p>What exactly are you wanting to infinite scroll/cache? You said chatbot, but your questions make it sound like you have a lot more than that going on in the frontend?<p>If it's the actual chatbot prompts and responses you want to cache (like the actual messages in a ChatGPT-like app), I think the challenge there would be more about reasoning through the server/client sync and how to paginate an infinitely long document (each individual chat) so that the client can fetch and cache chunks at a time. Or maybe you don't even have to worry about that at first, since the chats are probably going to be mostly natural language and code, which should easily compress over the wire anyway. It's something you can optimize and paginate later.<p>But what I'm getting is that I think the challenge is more going to be about figuring that server/client logic than worrying about clientside state in and of itself. Unless your chat engine or long-term storage is in the frontend itself (and at that point, why not just deploy as a desktop app altogether), the chat app is really just a thin client to the backend operations where the read/writes and scaling would actually happen.<p>For the client-server stuff, RTK-query in particular is really good at that. It's designed to easily sync up client state with API responses and handle caching effortlessly, with great developer ergonomics. I hated RTK on its own but add RTK-query and it becomes wonderful, like a more powerful/mature version of useSWR. But I don't really think that's really the crux of your problem, if I'm understanding your use case right (which I might not be)...<p>You're not planning on implementing the backend using NextJS too, are you? Next.js is great for write-rarely, read-often monoliths (like blogs, web apps that rarely need to save state to the server, etc.). It's primarily a Node server backed by a great CDN integration, with some serverless helpers. But if you try to implement real-time chat on top of that, with multiple users constantly writing and reading in real time, that's not the sort of thing that's going to be easy to scale on that architecture. Vercel does offer various hosted/integrated storage solutions (Postres, KV, etc.), but if you go down that route, you basically end up paying Vercel lots of money to handle scaling for you (which is fine if that's what you want to do). In that case they'll take care of the backend scaling for your dollars, and the clientside state lib is less important (because each chat's authoritative state is really on the server, and the client just displays read-only fragments of that at any given time). If you build it all on serverless/managed storage, presumably the bottleneck then wouldn't be from the Vercel network to each user, but from the Vercel network to your chatbot server. Where's that part actually going to live? (Or are you just using someone else's LLM API?)<p>If you just want to plug in someone else's API, I also wonder if you might not be reinventing the wheel... on the frontend, there's already open source kits like <a href="https://github.com/lobehub/lobe-chat">https://github.com/lobehub/lobe-chat</a> that take an API key and give you a functional chat interface. Might be worth looking into if you haven't already?