Out of control Java processes when dealing with a leap second?

Leap second + Java + Linux fix (without reboot):<p>/etc/init.d/ntp stop; date; date `date +"%m%d%H%M%C%y.%S"`; date;<p>-- then restart java

I had a lovely morning (2am-4.30) rebooting about 50 boxes by hand. All wearing some shade of java.

Reddit is also down due to Java <a href="https://twitter.com/redditstatus/status/219244389044731904" rel="nofollow">https://twitter.com/redditstatus/status/219244389044731904</a>

We saw this on every Ubuntu Lucid machine we run. Restarting the JVM was insufficient - we ended up fully rebooting the machines.

Would love to read a root cause analysis of the relevant source code. Anyone have a link?

I didn't really think about it at the time, but right about then (I'm in GMT+10) a Minecraft server (OpenJDK 64-bit, Ubuntu 11.10) I had running got rather upset ...<p><pre><code> 2012-07-01 09:59:59 [WARNING] Time ran backwards! Did the system time change? 2012-07-01 10:01:14 [WARNING] Can't keep up! Did the system time change, or is the server overloaded? </code></pre> ... and stuck on 100% CPU until I noticed it and restarted it this morning. Thankfully no other issues that I've noticed.

I saw the same across all of our Ubuntu servers. Some running Tomcat others running SmartFox Server (a java based game server). A reboot + app restart fixed it.

I was just wondering why everything stopped working at night. System.exit() didn't work anymore in any of the java processes, or just random hangs.<p>wow.

Even if you're not running java in your stack, it's worth a quick check on your servers. Our SoftLayer servers use an Adaptec RAID card and the monitoring software uses the JVM and will suck up all of your CPU. All of our MySQL/Cassandra/Redis/Kestrel servers needed to be bounced because they had RAID cards. Frustrating.

Same problem here. I am supposedly on vacation in Barcelona and was summoned to help :/ Gotta love your java.

At the risk of shooting myself, all our jvms were fine on windows and Ubuntu.

Fun that a bug for my team happens to be the thing linked in this article. :)

its the futex() call that is bugged, so therefore things that are highly multi-threaded... for example mysql and java!<p>Many unix things are not as multithreaded as those examples.

I was able to restore my java app boxen (and all other boxen) by only setting the date to the current date. No restarts necessary.

What's the explanation of that wacky date format?<p>minute day hour month year . second

I experienced no problem today. Is this Java-specific?

Has anyone seen this not on Ubuntu?