TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

Develop+Deploy RAG Bots with LlamaEdge: Across OSes, NPUs, GPUs Using Vector DB

1 pointsby 3Sophonsover 1 year ago

1 comment

3Sophonsover 1 year ago
LlamaEdge is not just a lightweight and portable runtime for chatbots. It is an app dev platform for portable LLM apps! RAG chatbots and AI agents can be developed on Mac or Windows, compiled to Wasm, and then deployed across Nvidia &#x2F; AMD &#x2F; ARM devices, fully taking advantage of on-device GPUs, NPUs etc.<p>Hence, besides the LLM runtime, those LLM apps also need to manage embeddings in vector db. This example shows how to access the Qdrant vector db from your portable Wasm apps!<p><a href="https:&#x2F;&#x2F;github.com&#x2F;WasmEdge&#x2F;wasmedge-db-examples&#x2F;tree&#x2F;main&#x2F;qdrant">https:&#x2F;&#x2F;github.com&#x2F;WasmEdge&#x2F;wasmedge-db-examples&#x2F;tree&#x2F;main&#x2F;q...</a>