TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Tracking down a memory leak in Ruby's EventMachine

158 点作者 nelhage大约 12 年前

10 条评论

andrewvc大约 12 年前
I found a bug in eventmachine during the time I spent writing em-zeromq, the eventmachine binding for zeromq. The important thing to understand here is that ZMQ is, in-essence, a userland socket. Normal sockets are efficiently monitored using the epoll system call (or one of its older variants, say select, or poll). However, as a user-land program<p>ZMQ 'sockets' aren't compatible with those calls, they use a userland equivalent of those kernel level edge triggered pollers. Integrating ZMQ with a traditional event library (like eventmachine) presents a problem at this point, as software like EM or Node typically require IO to be across real file descriptors from real sockets, something a userland library can't provide. The ZMQ devs however realized this was a hotly requested feature and so devised a way around this limitation.<p>The compatibility layer in ZMQ takes the form of performing some internal communication across traditional unix IPC, in the case using a pipe IIRC. In other words, for some of its internal messaging rather than simply use a function call, ZMQ will push data across a pipe. This pipe can then be exposed as a proxy for a ZMQ socket.<p>The downside of this strategy is that exposing FDs across software requires extreme care. Generally, it is assumed that <i>one</i> piece of software will have responsibility for an FD.<p>The actual issue in my case was that <i>any</i> ruby exception would cause the entire process to crash with an error about closing an already closed FD. What was happening was that given an exception <i>both</i> ZMQ and EM were trying to shut down all the FDs they knew about. Closing an FD that's already closed causes ZMQ to assert and crash instantly. It sounds simple once you're in the right frame of mind, but it took a good number of evenings to track down to that cause. It turned out the the EM option to not shut-down FDs was non-functional in the end. A one character patch provided the fix.
twoodfin大约 12 年前
Like danso, I admire the detective-work here. I would like to point out, though, that XCode's Instruments utility has a fantastically useful "Leaks" mode that will identify leaked allocations, including a stack trace. It can attach to a running process and has a non-disastrous impact on performance, though like most such tools it's voracious for memory.<p>Other platforms likely have similar tools, though I have yet to stumble across one as easy to use.
评论 #5339282 未加载
评论 #5341211 未加载
评论 #5339336 未加载
gingerlime大约 12 年前
<i>"If you’ve stared at too many Linux coredumps, as I have, that number looks suspicious. Interpreted in little-endian, that is 0x00007f1b5358a800, which points near the top of the userspace portion of the address space on an amd64 Linux machine.<p>In fewer words: It’s most likely a pointer."</i><p>As someone who has not stared at any core dump for more than about 2 seconds, I admire this level of skill.
pc大约 12 年前
I wish there were more posts like this.
评论 #5339308 未加载
评论 #5339320 未加载
lkrubner大约 12 年前
This is slightly off-topic, but I worked on a Ruby project where we did something just like this:<p>"It was easy enough to work around the leak by adding monitoring and restarting the process whenever memory usage grew too large"<p>I was surprised, because I can not think of any other language and/or framework where "just restart the process" is done so often. I mean, this is not a common attitude among Java programmers, I don't think it is common among C programmers, and I don't think it is common among Python programmers. But it does seem to be fairly standard in the Ruby community. David Heinemeier Hansson admitted this used to happen with Basecamp:<p><a href="http://david.heinemeierhansson.com/posts/31-myth-2-rails-is-expected-to-crash-400-timesday" rel="nofollow">http://david.heinemeierhansson.com/posts/31-myth-2-rails-is-...</a><p>Can anyone else tell me of a community where this is done so commonly?
评论 #5341368 未加载
评论 #5343355 未加载
评论 #5341364 未加载
评论 #5341354 未加载
danso大约 12 年前
There are two things I really like about this walkthrough:<p>1. It shows how bugs can be something quite conceptually simple<p>2. It shows the value of logical, detective-like thinking in tracking these bugs.<p>Even as a programmer, I still think of bug-hunting as something requiring an encyclopedia knowledge of the trivial and arcane. Obviously, it looks easier in hindsight, but the OP does a great job of demonstrating how you can discover a much-overlooked flaw with the right deductive thinking (and experience with profiling tools)
评论 #5340219 未加载
gravitronic大约 12 年前
THANK YOU - after looking at the heap perspective, and valgrind perspective, I've exhausted options looking at a memory leak in our production environment. This is another avenue. AWESOME.
pimeys大约 12 年前
I noticed a same kind of leak in one of our apps and this post was more than worth of gold on how to hunt it down.
rurounijones大约 12 年前
Now if there were an online course where you could learn this kind of stuff I would be signed up so fast physicists would be re-evaluating general relativity.<p>We need some online courses dedicated to not-beginners :)
vskr大约 12 年前
Amazing post. How much time did it take to debug and identify the issue
评论 #5341391 未加载