TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Ask HN: Where to Host Llama 2?

3 点作者 retrovrv将近 2 年前
There's ollama/ggml etc for local setup, but other than Replicate, what are the other options for hosting Llama 2?

1 comment

brucethemoose2将近 2 年前
vast.ai is a popular and economical option.<p>A single 3090 will host 70B reasonably well, two will fit it completely in vram.<p>Another thing I suggest is hosting on AI horde with koboldcpp, if the UI&#x2F;API works for you and the finetune is appropriate for public use. You will get priority access to your host, but fulfilling other prompts in its spare time will earn you kudos to try other models people are hosting, or to get more burst throughput.<p><a href="https:&#x2F;&#x2F;lite.koboldai.net&#x2F;#" rel="nofollow noreferrer">https:&#x2F;&#x2F;lite.koboldai.net&#x2F;#</a>
评论 #37127167 未加载