We once had a customer who would call us in a panic a couple of times a year saying our inspection equipment was experiencing unusually high false rejects and they were generating very high scrap rate. By the time we got a technician on site the next day, everything was working flawlessly and the customer couldn't reproduce the problem either. This went on for almost three years with various levels of escalation to the current management. Finally, one day a technician was on site for another project when the customer came up to him and said "It's happening right now! Come fix it!" The technician rushed over to the equipment and discovered that the sun was shining at exactly the right angle to cause a lens flare in one of our cameras. This happened twice a year as the sun moved along its trajectory. A strategically placed piece of opaque plastic fixed it permanently.
The phase of the moon really can affect performance. A friend of mine worked on wireless links in Scotland and was struggling with loss at certain times of day, but not exactly the same time every day. When they graphed loss against time, the pattern was really periodic over many days. The periodicity turned out to be 12 hours 25 minutes, which they eventually realized is exactly the time between low tides. The problem was at low tide the reflected path off the water interfered with the line-of-sight path causing signal fading, whereas at high tide it interfered much less. In particular, see figure 2 of their paper for the correlation between tide height and SNR: <a href="https://homepages.inf.ed.ac.uk/mmarina/papers/EDI-INF-RR-1365.pdf" rel="nofollow">https://homepages.inf.ed.ac.uk/mmarina/papers/EDI-INF-RR-136...</a> As tide height really does depend on the phase of the moon, presumably their loss did too, if they measured for long enough.
I heard a great story a while back for a digitization project where historic content was being provided by many libraries around the world, including one in Russia.<p>The quality of the scanned books was excellent, except for a weird distortion every so often where part of the page would be shifted partway through as if someone had shifted half the page in Photoshop. This was only noticed in books over a certain size so people were checking to see if there was some kind of mechanical problem with the scanner (these were robots with automatic page turners so it was plausible that there could be something which was only an issue past a certain position), trying to figure out of there was some way that the software had some kind of memory leak or other issue which would explain the long and inconsistent intervals.<p>Eventually they were on a long-distance phone call to Moscow and not turning up anything when there was a loud rumble in the background. “What was that?” lead to the realization that the library's scan center was close to a subway tunnel. The vibration of a passing train was enough to cause a glitch but only if you happened to be scanning at the exact time it went by: the reason longer books were noticed was simply because having more pages meant that at any point in time a long book was more likely to be sitting in the scanner and the technicians running the scanner were apparently tuning out the trains as background noise. This was reportedly the first project they'd done with one of the scan robots which can process an entire book unattended so it was plausible that smaller past projects simply hadn't been scanning frequently enough to hit this problem or that some previous technician had noticed and immediately redone the page.
I thought that this was going to be different story.<p>There was a program I heard about back in the 90s which would literally crash depending on the phase of the moon!<p>The story is that it wanted to print a date. The programmer happened to have an astronomy library available that gave a string containing the date. So the programmer called that, and then parsed out the date.<p>Unfortunately the astronomy library wrote its result as a string to a point. The result included the phase of the Moon. The pointer was not declared to be long enough. And therefore, would crash if the name of the phase of the moon was too long!
This reminds me of something I lived through as a nerdy teenager working a summer job as first line IT support at the headquarters of a multinational, in the mid-90s...<p>One day, I started receiving calls (through my pager!) from rather many people about intermittent networking problems. The state of the art 10mbit wired UTP network would have frequent bursts of 90% package loss.<p>What was weird: only people on the fifth floor would have this issue..!? Our first thought was that they were on a single hub/switch that might have broken. But no, they were connected to the same uplinks as the computers on the problem-free surrounding floors. Furthermore, laptop users (who were of course also wired at the time) were reporting no problems whatsoever.<p>We were pretty much out of ideas by that point, but did an experiment just to test our assumptions: we took a PC and hooked it up with a long network cable and a power extension cable on the fourth floor and started pinging it. Flawless. Then we started walking up the stairs, and, yes indeed, somewhere around halfway up the stairs packets started to drop. (But not at all times, sometimes it would be fine, like all PCs on the fifth.)<p>If you want to guess at the cause, this is your chance. :-)<p>We brought in a company specialized in EM interference. It turns out that a GSM antenna placed on the roof of the four story building opposite to ours about half a year ago, had just been turned on. Its height aligned to our fifth flour. Whenever someone was using this mast to make a call (which certainly wasn't all of the time back then), it would cause interference on a specific model of network card that we were using in all of our PCs. It had a relatively large metal component that was apparently a pretty good 900 MHz antenna.<p>When confronted, the mobile operator quickly adjusted the antenna to not be directed at us. I believe all network cards were replaced soon after. Fun times!
> Not strictly the cycle of the moon but close.<p>Meh. Just the old 49.7 days cycle that it takes to overflow 32 bits when measuring miliseconds.<p>I was hoping for a "it works when I buy vanilla icecream and doesn't when I buy other flavour".
I used to work in aerospace. One of my projects involved running avionics bench tests at a customer facility, basically the avionic subsystem of the aircraft in a big room on shelves. We were using a laptop for data logging and started getting dropouts in the data every 5 minutes. This was worrying because a) this hadn't happened at our site on similar equipment and b) this was a final customer-facing check before doing a real test flight.<p>We spent about a week trying to debug the system and the software and at a certain point while I was just sitting and thinking about what to do next, Flying Toasters popped up in the data logging PC (the lid was normally closed because of the space on the bench).<p>The Windows screensaver was hogging so much CPU that the datalogger couldn't keep up.
Tangentially related, here's another fun bug that inexplicably cares what time it is: Open Office cannot print on Tuesdays (<a href="https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161/comments/28" rel="nofollow">https://bugs.launchpad.net/ubuntu/+source/cupsys/+bug/255161...</a>)
Reminds me of a story where a company's internet would regularly drop at the same time every day -- let's say 3pm.<p>Nobody could figure it out so they called in an expert.<p>After lots of attempts and figuring, one day the person in question happens to look out the window at the time in question ... and sees a service truck park exactly in line-of-sight between the business and their internet-signal pickup broadcast point.<p>Ah ha!
> Bugs based on a time calculations can often show themselves later when view through a longer lens and scope of time...sometimes WAY longer than you'd expect.<p>When I worked for BBN in '97-'98, someone from outside the company as I recall came to talk to a room of engineers about the wide variety of calendar-related behaviors in various UNIX systems that were expected to cause problems for Y2K.<p>It was a very, very long list, often subtle issues, and I recall the concern in the room about the number of old systems in use by the DoD and others.<p>Anyway, no real point to this other than date handling is one of the hardest things to get right in computing, ranking right behind <i>testing</i> for the correct behavior.
> What an interesting and insidious bug! Bugs based on a time calculations can often show themselves later when view through a longer lens and scope of time...sometimes WAY longer than you'd expect.<p>My personal anecdote. I like playing online games, and as you know latency is the killer. I enjoyed playing in the evenings after work, and inexplicably I started noticing my latency spike from around 50ms to > 1s. Extremely frustrating.<p>I had no idea what caused this so I set up a simple ping command and had it save it to a graph.<p>Well, the next day I noticed the pings were steady throughout the whole day, then in the evenings I'd get these chunks of bad time. It turns out when my wife would watch Netflix in the other room (and it was <i>only</i> Netflix), it'd cause something to go awry with the router and latency would spike for me. (The really weird thing was that it was a combination of a Roku, Netflix, and a wired switch - change any of those and the problem went away).<p>Later during the pandemic, I also diagnosed drop-outs on my network due to kids in my neighborhood being online during school hours. Like clockwork I'd get a bad network from around 10pm and it'd be fine ending around 3 or 4. On school holidays and weekends my network was fine.
In the analogue days, before pixels existed, a customer had trouble with their phone line not working when the moon was full.<p>The problem was that they lived on the coast, and a subsurface junction box would get wet during king tides, causing the telephone line to fail.
F'n A. Reading the comments on this thread makes me love humanity. So much ingenuity, raw engineering horsepower, creativity. Goddamn, you are great people and you should be proud of yourselves. Reading this makes me believe we will survive as a species.
> Did you know (I know because I'm old) that Windows 95, for a time, was unable to run longer than 49.7 days of runtime?<p>Yep! Because foone did a whole month-long Twitter thread on it, even had a livestream showing the crash.<p><a href="https://twitter.com/Foone/status/1413694652822163459" rel="nofollow">https://twitter.com/Foone/status/1413694652822163459</a><p><a href="https://www.youtube.com/watch?v=Hb46tX7-d_o" rel="nofollow">https://www.youtube.com/watch?v=Hb46tX7-d_o</a>
At a previous job, we used a bug tracking tool called "Remedy". On September 8, 2001, it started reporting dates incorrectly; its idea of the current date jumped back to 1973 and started advancing at 1/10 the normal rate.<p>It used Unix timestamps (seconds since 1970) and assumed they could only be 9 decimal digits. When the time reached 10 digits, the last digit was quietly dropped.<p>(It was fixed within a few days.)
I'm immediately reminded of NetHack [1] (which you can play online here [2])! Real-life phase of the moon has a small but important effect on gameplay in NetHack. The public game server on alt.org even tracks the phase of the moon to provide a handy reference.<p>A little bit disappointing to discover that the code from the article does not actually depend on the phase of the moon. I'm really interested to see the other stories here where it actually is the case that the phase of the moon is affecting people's code.<p>[1] <a href="https://www.nethack.org" rel="nofollow">https://www.nethack.org</a><p>[2] <a href="https://alt.org/nethack/" rel="nofollow">https://alt.org/nethack/</a>
Now, this is a true phase-of-the-moon bug: <a href="http://ftp.informatik.rwth-aachen.de/jargon300/phaseofthemoon.html" rel="nofollow">http://ftp.informatik.rwth-aachen.de/jargon300/phaseofthemoo...</a>
Funny enough, Nethack[1] had this implemented too (w:spoilers). See also wmoonclock[2](wmaker) for a nice "moon-clock".<p>[1]<a href="https://nethackwiki.com/wiki/Time" rel="nofollow">https://nethackwiki.com/wiki/Time</a><p>[2]<a href="https://www.dockapps.net/wmmoonclock" rel="nofollow">https://www.dockapps.net/wmmoonclock</a>
I've resolved several "celestial body" problems with routers and modems in East/West Africa over the years by pointing USB fans at them – between the sun and the workday generating heat with higher load in lower-end routers or insufficiently-air-conditioned units, can work surprisingly well to improve the network at almost no cost.
I have seen interference from one part of set top box to cause noise on flash input lines and sometimes issue a command to clear flash on the device.<p>Months of debugging, dozen people involved, tens of thousands of devices bricked, tens of millions lost.<p>All due to a single line of code that configured flash to not require special magic before each command. This feature made to improve resistance to interference also hindered performance. Somebody thought it a good idea to disable to get some points for improved performance.
Once in a while my PC performs several contiguous execution of a key pressed on my Bluetooth keyboard. At other times, it ends up missing some key presses. Upon investigation, I've discovered this usually happen when ceiling fan is running :).
As long as you don't traumatize the hard drives by yelling at them, feat. brendangregg & bcantrill:<p><a href="https://youtu.be/tDacjrSCeq4" rel="nofollow">https://youtu.be/tDacjrSCeq4</a>
Is there a database or "online myth/story" archive of wacky bugs like this and in the comments? These would make for great "cocoa at night" reading!
I worked for satellite television. One of our servers would freak out once a year. It was found that the actual satellite was in line with sun at that time causing large amount if power sent over cable close to the server due to need to use backup antenna.
Neat Thread!<p>What i love about these problem-solving anecdotes is how a seemingly totally different domain is the key to the solution. It always makes me marvel at how interconnected everything in our World is. Strengthens my belief that "Cross-Disciplinary" knowledge is where "Wisdom" lies and is the key to our Future.<p><i>“From a drop of water, a logician could infer the possibility of an Atlantic or a Niagara without having seen or heard of one or the other. So all life is a great chain, the nature of which is known whenever we are shown a single link to it.”</i>
--- Sherlock Holmes in A Study in Scarlet
Reminds me of the story where emails couldn't be sent more than 500 miles away: <a href="https://web.mit.edu/jemorris/humor/500-miles" rel="nofollow">https://web.mit.edu/jemorris/humor/500-miles</a>
I love/hate such bugs. Often hard to debug, but totally great to finally find the root cuase and a solution. I've once got a ticket stating "this script always crashes before 10 a.m." [1]<p>[1] <a href="https://darekkay.com/blog/script-crashes-before-10/" rel="nofollow">https://darekkay.com/blog/script-crashes-before-10/</a>
See also a case where the hardware worked differently due to the moon <a href="https://www.nytimes.com/1992/11/27/us/moon-is-blamed-for-blips-in-a-particle-accelerator.html" rel="nofollow">https://www.nytimes.com/1992/11/27/us/moon-is-blamed-for-bli...</a>
At my work they had a server outage every so often, where it seemed to just go down (back in the 90s). Turned out the cleaning lady came in the evenings when there was no one there, unplugged the server before plugging in her vacuum cleaner. And then plugging the server back in.
My office internet drops out on my computer every afternoon at about 3pm....doesnt affect any other computer, just mine. I am getting Father Merrin in next week to give the place a proper going over.
Signed integers should be used as sparingly as floating-point. They should not be used in ordinary code because ordinary code has no use for them until they break something.<p>The most notable exception would be languages which allow negative indexing, but IMHO if that were syntactic instead of relying on actual signed integers, it would be safer (I.e., [- $int] would be a different syntax from [(-$int)] and the latter would not be correctly typed.)