Please stress test it - like hammer it hard: https://jazz-ai-chat.vercel.app<p>It was so delightful to build thanks to jazz.tools.<p>Vercel's Fluid compute makes it run efficient and scale endlessly<p>How does it work?
Each message is created locally and added to the chat. At the same time I "wake up" a Vercel Function which consumes the chat, runs AI and writes directly into Jazz<p>The client just renders the state from Jazz. No real streaming required.<p>Source: https://github.com/tobiaslins/jazz-ai-chat<p>Whats Fluid Compute?
When you run Serverless Functions, usually one invocations serves one request. With Fluid, multiple requests can be served by one function invocation, which is super useful when using AI (low CPU usage, only waiting for upstreams)<p>Whats Jazz?
It's a new way to build apps. It's persistence (like a graph database) with permissions and sync built in.<p>You can access/write Jazz even from backend workers like here from Vercel Functions.
It needs a spinning circle or dots to show that the AI is thinking. (It can be just an animated gif :) .)<p>Sometimes it doesn't reply until I add a second question like "????"