TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Hacked Nvidia 4090 GPU driver to enable P2P

829 点作者 nikitml大约 1 年前

38 条评论

andersa大约 1 年前
Incredible! I&#x27;d been wondering if this was possible. Now the only thing standing in the way of my 4x4090 rig for local LLMs is finding time to build it. With tensor parallelism, this will be both massively cheaper and faster for inference than a H100 SXM.<p>I still don&#x27;t understand why they went with 6 GPUs for the tinybox. Many things will only function well with 4 or 8 GPUs. It seems like the worst of both worlds now (use 4 GPUs but pay for 6 GPUs, don&#x27;t have 8 GPUs).
评论 #40015272 未加载
评论 #40013494 未加载
评论 #40014382 未加载
评论 #40014157 未加载
评论 #40024825 未加载
评论 #40013602 未加载
评论 #40014481 未加载
评论 #40013415 未加载
chriskanan大约 1 年前
This is great news. As an academic, I&#x27;m aware of multiple labs that built boxes with 4090s, not realizing that Nvidia had impaired P2P communication among cards. It&#x27;s one of the reasons I didn&#x27;t buy 4090s, despite them being much more affordable for my work. It isn&#x27;t nvlink, but Nvidia has mostly gotten rid of that except for their highest end cards. It is better than nothing.<p>Late last year, I got quotes for machines with four nvlink H100s, but the lead time for delivery was 13 months. I could get the non-nvlink ones in just four months. For now, I&#x27;ve gone with four L40S cards to hold my lab over but supply chain issues and gigantic price increases are making it very hard for my lab to do it&#x27;s work. That&#x27;s not nearly enough to support 6 PhD students and a bunch of undergrads.<p>Things were a lot easier when I could just build machines with two GPUs each with Nvlink for $5K each and give one to each student to put under their desks, which is what I did back in 2015-2018 at my old university.
评论 #40032037 未加载
评论 #40033098 未加载
jstanley大约 1 年前
What does P2P mean in this context? I Googled it and it sounds like it means &quot;peer to peer&quot;, but what does that mean in the context of a graphics card?
评论 #40011694 未加载
评论 #40011682 未加载
评论 #40013903 未加载
评论 #40011680 未加载
评论 #40013120 未加载
评论 #40011691 未加载
评论 #40016187 未加载
userbinator大约 1 年前
I wish more hardware companies would publish more documentation and let the community figure out the rest, sort of like what happened to the original IBM VGA (look up &quot;Mode X&quot; and the other non-BIOS modes the hardware is actually capable of - even 800x600x16!) Sadly it seems the majority of them would rather tightly control every aspect of their products&#x27; usage since they can then milk the userbase for more $$$, but IMHO the most productive era of the PC was also when it was the most open.
评论 #40013286 未加载
评论 #40016272 未加载
评论 #40032353 未加载
评论 #40014441 未加载
No1大约 1 年前
The original justification that Nvidia gave for removing Nvlink from the consumer grade lineup was that PCIe 5 would be fast enough. They then went on to release the 40xx series without PCIe 5 and P2P support. Good to see at least half of the equation being completed for them, but I can’t imagine they’ll allow this in the next gen firmware.
HPsquared大约 1 年前
Is this one of those features that&#x27;s disabled on consumer cards for market segmentation?
评论 #40013182 未加载
评论 #40014215 未加载
评论 #40021738 未加载
评论 #40012281 未加载
ivanjermakov大约 1 年前
I was always fascinated by George Hotz&#x27;s hacking abilities. Inspired me a lot for my personal projects.
评论 #40013397 未加载
评论 #40012299 未加载
评论 #40012381 未加载
评论 #40013039 未加载
llm_trw大约 1 年前
Skimming the readme this is p2p over PCIe and not NVLink in case anyone was wondering.
评论 #40011303 未加载
评论 #40011151 未加载
jsheard大约 1 年前
It&#x27;ll be nice while it lasts, until they start locking this down in the firmware instead on future architectures.
评论 #40013713 未加载
jagrsw大约 1 年前
Was it George himself, or a person working for a bounty that was set up by tinycorp?<p>Also, a question for those knowledgeable about the PCI subsys: it looked like something NVIDIA didn&#x27;t care about, rather than something they actively wanted to prevent, no?
评论 #40013948 未加载
评论 #40011139 未加载
评论 #40012035 未加载
评论 #40011557 未加载
rfoo大约 1 年前
Glad to see that geohot is back being geohot, first by dropping a local DoS for AMD cards, then this. Much more interesting :p
评论 #40013100 未加载
评论 #40012557 未加载
modeless大约 1 年前
What are the chances that Nvidia updates the firmware to disable this and prevents downgrading with efuses? Someday cards that still have older firmware may be more valuable. I&#x27;d be cautious upgrading drivers for a while.
xipho大约 1 年前
You can watch this happen on the weekends, typically, sometimes, for some very long sessions, sometimes. <a href="https:&#x2F;&#x2F;www.twitch.tv&#x2F;georgehotz" rel="nofollow">https:&#x2F;&#x2F;www.twitch.tv&#x2F;georgehotz</a>
klohto大约 1 年前
fyi should work on most 40xx[1]<p>[1] <a href="https:&#x2F;&#x2F;github.com&#x2F;pytorch&#x2F;pytorch&#x2F;issues&#x2F;119638#issuecomment-2051196015">https:&#x2F;&#x2F;github.com&#x2F;pytorch&#x2F;pytorch&#x2F;issues&#x2F;119638#issuecommen...</a>
thangngoc89大约 1 年前
&gt; You may need to uninstall the driver from DKMS. Your system needs large BAR support and IOMMU off.<p>Can someone point me to the correct tutorial on how to do these things?
评论 #40017586 未加载
评论 #40017097 未加载
tanelpoder大约 1 年前
I also love that it can be done with just a few code line changes:<p><a href="https:&#x2F;&#x2F;github.com&#x2F;NVIDIA&#x2F;open-gpu-kernel-modules&#x2F;commit&#x2F;1f4613dacec2638569a74b5e3dbcab01832f72a7?diff=unified&amp;w=1">https:&#x2F;&#x2F;github.com&#x2F;NVIDIA&#x2F;open-gpu-kernel-modules&#x2F;commit&#x2F;1f4...</a>
gigatexal大约 1 年前
as a technical feat this is really cool! though as others mention i hope you don&#x27;t get into too much hot water legally<p>seems anything that remotely lets &quot;consumer&quot; cards canibalize anything with the higher end H&#x2F;A-series cards Nvidia would not be fond of and they&#x27;ve the laywers to throw at such a thing
xmorse大约 1 年前
Finally switched to Nvidia and already adding great value
clbrmbr大约 1 年前
If we end up with a compute governance model of AI control [1], this sort of thing could get your door kicked in by the CEA (Compute Enforcement Agency).<p>[1] <a href="https:&#x2F;&#x2F;podcasts.apple.com&#x2F;us&#x2F;podcast&#x2F;ai-safety-fundamentals-alignment&#x2F;id1680794263?i=1000651665081" rel="nofollow">https:&#x2F;&#x2F;podcasts.apple.com&#x2F;us&#x2F;podcast&#x2F;ai-safety-fundamentals...</a>
评论 #40011209 未加载
评论 #40011206 未加载
评论 #40016851 未加载
评论 #40014674 未加载
BeefySwain大约 1 年前
Can someone ELI5 what this may make possible that wasn&#x27;t possible before? Does this mean I can buy a handful of 4090s and use it in lieu of an h100? Just adding the memory together?
评论 #40013357 未加载
waldrews大约 1 年前
Would this approach be possible to extend downmarket, to older consumer cards? For a lot of LLM use cases we&#x27;re constrained by memory and can tolerate lower compute speeds so long as there&#x27;s no swapping. ELI5, what would prevent a hundred 1060-level cards from being used together?
评论 #40019801 未加载
namibj大约 1 年前
And here I thought (PCIe) P2P was there since SLI dropped the bridge (for the unfamiliar, it looks and acts pretty much like an NVLink bridge for regular PCIe slot cards that have NVLink, and was used back in the day to share framebuffer and similar in high-end gaming setups).
评论 #40014428 未加载
perfobotto大约 1 年前
What stops nvidia from making sure this stops working in future driver releases?
评论 #40014468 未加载
评论 #40016269 未加载
lucifer_is_back大约 1 年前
so basically rtx 4090 x6 = 144 GB ram which would cost $1599<i>6 = $9594 ( only the nvidia 4090s) and currently the tiny box gives *TinyBox* &gt; GPU RAM | 144 GB &gt; Price | $15,000 $25,000 Nvidia 4090x6<p>&gt; GPU RAM | 144 GB &gt; Price | $9594<p>so a * 36.04%* decrease in price from team red tinybox ( $15k) and *61.624% *decrease in price from the team green tinybox ( $25k)</i>
ewalk153大约 1 年前
Does this appear to be intentionally left out by NVidia or an oversight?
评论 #40011364 未加载
评论 #40011354 未加载
jeffs4271大约 1 年前
It is cool seeing hacks like this. But this is something to be careful with, as GH100 had hardware changes to meet CUDA fence requirements.
aresant大约 1 年前
So assuming you utilized this with (4) x 4090s is there a theoretical comparative to performance vs the A6000 &#x2F; other professional lines?
评论 #40024290 未加载
评论 #40013976 未加载
lawlessone大约 1 年前
This is very interesting.<p>I can&#x27;t afford two mortgages though ,so for me it will have to just stay as something interesting :)
cavisne大约 1 年前
How does this compare in bandwidth and latency to nvlink? (I’m aware it’s not available on the consumer cards)
评论 #40065191 未加载
评论 #40016975 未加载
spxneo大约 1 年前
does this mean you can horizontally scale to GPT-4-esque LLM locally in the near future? (i hear you need 1TB of VRAM)<p>Is Apple&#x27;s large VRAM offering like 196gb offer the fastest bandwidth and if so how will pairing a bunch of 4090s like in the comments work?
qxfys大约 1 年前
I am amazed how people always find a way to make this kind of thing work. kudos!
musha68k大约 1 年前
OK now we are seemingly getting somewhere. I can feel the enthusiasm coming back to me.<p>Especially in light of what&#x27;s going on with LocalLLaMA etc:<p><a href="https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;LocalLLaMA&#x2F;comments&#x2F;1c0mkk9&#x2F;mistral_8x22b_already_runs_on_m2_ultra_192gb_with" rel="nofollow">https:&#x2F;&#x2F;www.reddit.com&#x2F;r&#x2F;LocalLLaMA&#x2F;comments&#x2F;1c0mkk9&#x2F;mistral...</a>
gururise大约 1 年前
How long before Nvidia patches this?
m3kw9大约 1 年前
In layman terms what does this enable?
arthurcolle大约 1 年前
Does this work on 4060?
c0g大约 1 年前
Any idea of DDP perf?
vladgur大约 1 年前
curious if this will ever make it to 3090s
theturtle32大约 1 年前
WTF is P2P?
评论 #40018418 未加载