TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Docker Model Runner

100 点作者 kordlessagain大约 1 个月前

10 条评论

jpgvm28 天前
I&#x27;m looking at using OCI at $DAY_JOB for model distribution for fleets of machines also so it&#x27;s good to see it&#x27;s getting some traction elsewhere.<p>OCI has some benefits over other systems, namely that tiered caching&#x2F;pull-through is already pretty battle-tested as is signing etc, beating more naive distribution methods for reliability, performance and trust.<p>If combined with eStargz or zstd::chunked it&#x27;s also pretty nice for distributed systems as long as you can slice things up into files in such a way that not every machine needs to pull the full model weights.<p>Failing that there are P2P distribution mechanisms for OCI (Dragonfly etc) that can lessen the burden without resorting to DIY on Bittorrent or similar.
评论 #43681299 未加载
评论 #43682381 未加载
israrkhan28 天前
Be aware of licensing restrictions. Docker Desktop is free for personal use, but it requires a paid license if you work for an organization sized 250+. This feature seems to be available in Docker Desktop only.
评论 #43680750 未加载
leowoo9128 天前
I don&#x27;t understand why add another domain-specific command to a container manager and go out of scope for what the tool was designed for at first place.
评论 #43679076 未加载
评论 #43680621 未加载
评论 #43690303 未加载
rockwotj28 天前
Looks exactly like ollama but built into Docker desktop? Anyone know of any differences?
评论 #43678593 未加载
评论 #43677457 未加载
评论 #43677209 未加载
评论 #43677230 未加载
tgmatt28 天前
Seems like <a href="https:&#x2F;&#x2F;kitops.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;kitops.org&#x2F;</a> but fewer features.
Havoc27 天前
Can’t say I&#x27;m a fan of packaging models as docker images. Feels forced - a solution in search of a problem.<p>The existing stack - a server and model file - works just fine. There doesn’t seem to be a need to jam an abstraction layer in there. The core problem docker solves just isn’t there
评论 #43690314 未加载
评论 #43680346 未加载
avs73327 天前
I&#x27;m going to take a contrarian perspective to the theme of comments here...<p>There are currently very good uses for this and likely going to be more. There are increasing numbers of large generative AI models used in technical design work (e.g., semiconductor rules based design&#x2F;validation, EUV mask design, design optimization). Many&#x2F;most don&#x27;t need to run all the time. Some have licensing that is based on length of time running, credits, etc. Some are just huge and intensive, but not run very often in the design glow. Many are run on the cloud but industrial customers are remiss to run them on someone else&#x27;s cloud<p>Being able to have my GPU cluster&#x2F;data center be running a ton of different and smaller models during the day or early in the design, and then be turned over to a full CFD or validation run as your office staff goes home seems to be to be useful. Especially if you are in anyway getting billed by your vendor based on run time or similar. It can mean a more flexible hardware investment. The use casae here is going to be Formula 1 teams, silicon vendors, etc. - not pure tech companies.
superb_dev28 天前
Looks like Docker is feeling left out of the GenAI bubble. It’s a little late…
评论 #43680119 未加载
waffletower27 天前
I have used Replicate Cog, built on docker, fairly heavily and and find it is a decent compromise of features. Docker taking this use case more seriously is quite welcome, though surprisingly late. Local metal GPU support (where available to the containerized application APIs), not currently available in Cog, is attractive though it would require generalization of application code to support containers executable via Cuda and Metal etc.
评论 #43688508 未加载
tuananh28 天前
they are about ~2 years late.
评论 #43681049 未加载
评论 #43690317 未加载