TE
TechEcho
Home24h TopNewestBestAskShowJobs
GitHubTwitter
Home

TechEcho

A tech news platform built with Next.js, providing global tech news and discussions.

GitHubTwitter

Home

HomeNewestBestAskShowJobs

Resources

HackerNews APIOriginal HackerNewsNext.js

© 2025 TechEcho. All rights reserved.

What Flips Your Bit: Cosmic Ray Errors at Mozilla

233 pointsby dannyobrienabout 3 years ago

19 comments

IncRndabout 3 years ago
&quot;Bitsquatting is a form of cybersquatting which relies on bit-flip errors that occur during the process of making a DNS request. These bit-flips may occur due to factors such as faulty hardware or cosmic rays. When such an error occurs, the user requesting the domain may be directed to a website registered under a domain name similar to a legitimate domain, except with one bit flipped in their respective binary representations.<p>&quot;A 2011 Black Hat paper detailed an analysis where eight legitimate domains were targeted with thirty one bitsquat domains. Over the course of one day, 3,434 requests were made to bitsquat domains.&quot; [1]<p>Cisco presented a paper on bitsquatting at defcon, &quot;Examining the Bitsquatting Attack Surface&quot;. From the paper, &quot;The conclusion is that the possibility of bitsquat attacks is more widespread than originally thought, but several techniques exist for mitigating the effects of these new attacks.&quot; [2]<p>[1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Bitsquatting" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Bitsquatting</a><p>[2] <a href="https:&#x2F;&#x2F;media.defcon.org&#x2F;DEF%20CON%2021&#x2F;DEF%20CON%2021%20presentations&#x2F;DEF%20CON%2021%20-%20Schultz-Examining-the-Bitsquatting-Attack-Surface-WP.pdf" rel="nofollow">https:&#x2F;&#x2F;media.defcon.org&#x2F;DEF%20CON%2021&#x2F;DEF%20CON%2021%20pre...</a>
评论 #31017814 未加载
评论 #31023931 未加载
评论 #31021250 未加载
评论 #31017811 未加载
评论 #31018126 未加载
tconfreyabout 3 years ago
I worked at Sonus Networks (now Ribbon[0]) in the early 2000&#x27;s building VoIP solutions for telcos. We had a bunch of unexplained errors in a new installation in Denver. After much head scratching the engineers on the problem concluded that the higher altitude significantly increased the likelihood of impact by alpha particles and that that was the cause of the problem!<p>(IIRC we increased the shielding on the devices.)<p><a href="https:&#x2F;&#x2F;ribboncommunications.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;ribboncommunications.com&#x2F;</a>
评论 #31021386 未加载
评论 #31017102 未加载
评论 #31016631 未加载
geophileabout 3 years ago
As the article points out, using collected client data is problematic, because some errors will often be undetectable, as in numeric data. And in general, you would have to control for bit flips somehow caused by software.<p>I wonder whether a SETI approach would be useful here. Allocate, say, 1MB of memory. Fill it with some known bit pattern. Periodically check the memory and look for discrepancies. Do this once an hour, on 10M devices, and that is a LOT of monitoring. Report discrepancies along with time, location (including elevation), hardware and OS information.<p>I would think that this approach would provide a lot of interesting information about when and where bit flips occur, especially when matched against information on solar and atmospheric events (as in the article). Perhaps sensitive hardware and OS environments would be detected. Even completely negative results would be interesting: no bit flips observed would suggest that purported bit flips elsewhere might have other explanations.
评论 #31020914 未加载
评论 #31018922 未加载
robotsteve2about 3 years ago
Any sort of hardware or software error seems much more likely. Computers are incredibly complex and approximations are used everywhere (in the design of the hardware, in the theory of operation). I don&#x27;t think inference-based experiments or analysis on cosmic ray bit flips are appropriate.<p>You really need some kind of dedicated cosmic ray detector nearby as a control. If the flux of cosmic rays into the detector is orders of magnitude lower than the rate of bit errors you ascribe to cosmic rays, it&#x27;s probably some hardware&#x2F;software issue and not the cosmic rays.
评论 #31017386 未加载
评论 #31019865 未加载
评论 #31019890 未加载
zepearlabout 3 years ago
I don&#x27;t know folks.<p>2 years ago I took a laptop which I wasn&#x27;t using (16 GiB RAM non-ECC) =&gt; I created in Linux with Python an array (&quot;bytes&quot;? Don&#x27;t remember exactly anymore) of ~10 or 12 GiB containing random integers =&gt; computed the array&#x27;s hash and saved it.<p>Then for ~1-2 months I recomputed from time to time the hash of that array (inbetween the laptop was in suspend-to-RAM) and compared it to the original result =&gt; it always matched, I never had any bitflips.<p>I therefore doubt that the estimation of &quot;1&#x2F;256MB&#x2F;month&quot; is correct - I could not prove that, at least not with my laptop.
评论 #31020347 未加载
评论 #31019817 未加载
评论 #31019871 未加载
评论 #31020140 未加载
评论 #31021411 未加载
评论 #31024375 未加载
评论 #31021545 未加载
spullaraabout 3 years ago
I bit squatted cloudfront.net years ago and got many, many requests. Most of them *.js which would, if I were malicious, have allowed me to do just about anything. It was interesting to see that the errors definitely happened in different places. For instance, sometimes the Host header was the original domain and sometimes it matched my domain.
axg11about 3 years ago
This is fascinating and hints at a future possible scientific study: using phones across the globe to map cosmic ray events. I&#x27;m not a physicist so I can&#x27;t speak for the value of such data. If cosmic ray events do not occur uniformly across the globe then mapping events from 100,000s of phones could give interesting insights.
评论 #31016930 未加载
li2uR3ceabout 3 years ago
&gt; In almost every case we cannot find any plausible explanation or bug<p>Observe the natural state of every software developer. I kid... or do I?<p>&gt; What if it wasn’t just some fantastical explanation?<p>Doesn&#x27;t sound nearly as fantastical but bad RAM is probably more common than one would expect. You seldom really know the quality of hardware you run on. Just say&#x27;n, sometimes you don&#x27;t need a helping cosmic ray.
grog454about 3 years ago
On the subject of bit flips, I am able to detect these in the client to server UDP packets in my game. With specific logging enabled I would see an error about once per minute while receiving about 15,000 of one type of packet per second. I was able to estimate about 1&#x2F;1,000,000 packets contained a single flipped bit.
评论 #31023529 未加载
legalcorrectionabout 3 years ago
I suspect without great evidence that cosmic ray bitflips are mostly a scapegoat for imperfect hardware and are in fact one or two orders of magnitude less common than popular wisdom would suggest.
incomingpainabout 3 years ago
I had the opportunity to design my SOC from scratch. Mostly ripping off Berkeley&#x27;s public design.<p>Something I have documented in the last 2 years. Solar flare activity is what causes problems. All memory is ECC but it still happens.<p>Faraday cage incoming?<p>Wait? Faraday cage racks million $ idea?
评论 #31018729 未加载
ThePhysicistabout 3 years ago
One of the first things you&#x27;ll learn when studying experimental physics is how to come up with all kinds of alternative mechanisms that might explain the result you&#x27;ve observed in your experiment, and then think of ways to test that the results weren&#x27;t actually caused by those unwanted mechanisms. Most Nobel-prize winning physics experiments were carefully designed to compensate for any relevant secondary effects, and I would even go as far as saying that this is often the largest challenge when doing high-precision experiments.<p>So the first question I&#x27;d ask myself when thinking about cosmic-ray induced errors is how I would ensure that the bit flips are not caused by e.g. problems on the hard drives or the NAND array (which are probably much more likely to occur than cosmic ray events, at least on the surface of the earth).
评论 #31020683 未加载
withinboredomabout 3 years ago
We see a correlation between (major) solar activity and hash&#x2F;signature verification failures from clients -- on the order of millions of verifications per day, only 30k failures per day, max.<p>I just finished looking into it in our reporting and was pretty impressed to see spikes lineup with dates here: <a href="https:&#x2F;&#x2F;www.spaceweatherlive.com&#x2F;en&#x2F;solar-activity&#x2F;top-50-solar-flares&#x2F;year&#x2F;2020.html" rel="nofollow">https:&#x2F;&#x2F;www.spaceweatherlive.com&#x2F;en&#x2F;solar-activity&#x2F;top-50-so...</a>
anonymousiamabout 3 years ago
This is why I always buy ECC&#x2F;EDAC capable servers. SEUs are a real thing.
Avlin67about 3 years ago
What about overclocking ? does it cause bit flip ? especially low grad DDR4 pushed to its limit…
dextercdabout 3 years ago
The &#x27;1 error for every 256MB memory a month&#x27; sounds like way tko much to me.<p>A program I wrote launches every time I start my computer. It allocates some memory and scans it periodically for unexpected changes. After an equivalent of 15.8 256MB&#x2F;months no anomalies have been found yet.<p>Would really like to see more authoritative figures for modern consumer hardware.
评论 #31167274 未加载
评论 #31022889 未加载
eatonphilabout 3 years ago
If I wanted to reproduce bitflipping (from any source) on my laptop (any computer, really) over the shortest time frame possible, how could I conduct that experiment? Any pointers welcome.
评论 #31024128 未加载
trolliedabout 3 years ago
I know HN has a decent Factorio fanbase. Factorio properly stresses PC hardware, and borderline memory is usually ok for a casual gamer until you start a Factorio megabase. A decent example is Warger who does speedruns: <a href="https:&#x2F;&#x2F;forums.factorio.com&#x2F;viewtopic.php?f=7&amp;t=100646" rel="nofollow">https:&#x2F;&#x2F;forums.factorio.com&#x2F;viewtopic.php?f=7&amp;t=100646</a> <a href="https:&#x2F;&#x2F;www.speedrun.com&#x2F;factorio#100" rel="nofollow">https:&#x2F;&#x2F;www.speedrun.com&#x2F;factorio#100</a> Those that have played the game - speedruns are amazing to watch, if you haven&#x27;t already.
评论 #31033307 未加载
0xdeadbeefbabeabout 3 years ago
How do they know it&#x27;s cosmic rays and not something else?