Hi HN! Here's a local-only stack I built over the weekend - hope it can be useful for you!<p>I have been building a lot of AI apps - <a href="https://github.com/a16z-infra/ai-town">https://github.com/a16z-infra/ai-town</a>
<a href="https://github.com/a16z-infra/companion-app">https://github.com/a16z-infra/companion-app</a>
...<p>And there were definitely times I spent way too much $$ before deploying the app to production. So I was looking for a "local only" stack and found a few tools that worked well together. I used the following set of tools but may add more options later:<p>- Inference: Ollama<p>- VectorDB: Supabase pg-vector<p>- LLM orchestration: langchain<p>- Framework: nextjs<p>My learning from building ai-town was also that many people don't have the luxury of swiping a credit card to test an app with a lot of AI workloads -- not surprisingly, many models on Ollama are already very good - I used llama2-7b for majority of my dev work.<p>Hope this can lower the barrier to entry for building AI apps. :)
very cool that you used supabase for this yoko - thank you.<p>it could be worthwile adding an an index to the migration script[0]. if you decide to do this, it will need to be HNSW since ivfflat indexes should be created _after_ the table has some data[1]. HNSW is faster as well, so it's a sensible default.<p>also it's worth mentioning that the gte-small[2] embedding model isn't great with non-english embeddings. I don't know if many developers will get tripped up on that, but hopefully it's clear that they can swap that for any huggingface model<p>[0] <a href="https://github.com/ykhli/local-ai-stack/blob/main/supabase/migrations/20231028002341_create_vector_db.sql">https://github.com/ykhli/local-ai-stack/blob/main/supabase/m...</a><p>[1] <a href="https://github.com/pgvector/pgvector#ivfflat">https://github.com/pgvector/pgvector#ivfflat</a><p>[2] <a href="https://github.com/ykhli/local-ai-stack/blob/1a23061442ce1d98981f0f3af83f2cbbc2ed5c18/supabase/functions/embed/index.ts#L15">https://github.com/ykhli/local-ai-stack/blob/1a23061442ce1d9...</a>