TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Develop+Deploy RAG Bots with LlamaEdge: Across OSes, NPUs, GPUs Using Vector DB

1 点作者 3Sophons超过 1 年前

1 comment

3Sophons超过 1 年前
LlamaEdge is not just a lightweight and portable runtime for chatbots. It is an app dev platform for portable LLM apps! RAG chatbots and AI agents can be developed on Mac or Windows, compiled to Wasm, and then deployed across Nvidia &#x2F; AMD &#x2F; ARM devices, fully taking advantage of on-device GPUs, NPUs etc.<p>Hence, besides the LLM runtime, those LLM apps also need to manage embeddings in vector db. This example shows how to access the Qdrant vector db from your portable Wasm apps!<p><a href="https:&#x2F;&#x2F;github.com&#x2F;WasmEdge&#x2F;wasmedge-db-examples&#x2F;tree&#x2F;main&#x2F;qdrant">https:&#x2F;&#x2F;github.com&#x2F;WasmEdge&#x2F;wasmedge-db-examples&#x2F;tree&#x2F;main&#x2F;q...</a>