We recently did a review of our AI tech stack at (both internal and external).<p>Some of our favorites and more prolifically used were OpenAI API, ChatGPT, Midjourney, and Jasper.<p>Curious to hear what other AI tools (and frameworks like LangChain etc) y'all are heavily dependent on
I am all local<p>VoltaML for stable diffusion imagegen (as AITemplate is <i>really</i> fast, and controlnet and LORAs are super useful)<p>Llama.cpp for textgen... I use the koboldcpp frontend (and sometimed AI Horde) for experimenting with models, though I am still figuring out the exact stack I want for productivity. I am watching this PR closely:<p><a href="https://github.com/ggerganov/llama.cpp/pull/1773">https://github.com/ggerganov/llama.cpp/pull/1773</a><p>As well as Starcoder/Openllama finetunes for summarization, codegen and such.
OpenAI + PineconeDB -> Langchain -> Quivr to get a decent UI<p>right now i'm trying to prove to the company i work that you can turn the endless rules and daily information stream into something people can find easily using sematic search (vector DB) and an AI to summarize categorize, generate embeddings and auto update the DB, while also interpreting the content of the search result and giving a more digestible answer and a source link if the end user needs more information.<p>Keyword based search is a pain when different products have similar names or the internal search doesn't filter words like "of" and "and" from the keyword search.<p>After that, if the tests work and i can push the idea forward, it'll probably be OpenAI + a local vector DB (Chroma?) + custom made search page for internal usage
For us the complexity has all been about the infra around the AI stack. For example, having a k8s cluster for running inference for open source models (Stable Diffusion, Bark, etc.). Beyond that it’s been custom abstractions around Open AI/PaLM APIs and vector stores (pinecone and Faiss).<p>This stack is evolving so quickly but we are convinced it goes back to a distributed systems/infra problems when it comes to complexity, unless you’re training your own foundation models.
A deeper look into our stack if you're interested: <a href="https://www.commandbar.com/blog/harnessing-ai-magic">https://www.commandbar.com/blog/harnessing-ai-magic</a>