Handling OOM livelocks is exciting, and they have a good explanation for why the current OOM-killer fails:<p>> One usecase is avoiding OOM hangs/livelocks. The reason these happen
is because the OOM killer is triggered by reclaim not being able to
free pages, but with fast flash devices there is <i>always</i> some clean
and uptodate cache to reclaim; the OOM killer never kicks in, even as
tasks spend 90% of the time thrashing the cache pages of their own
executables. There is no situation where this ever makes sense in
practice.
Interesting!<p>After reading it, I realized I was actually hoping for information at a lower level than VM for memory pressure. Actually finding live, actionable information about DRAM bandwidth usage, delays caused by the hardware system of caches including TLBs, L1/2/3 caches and main memory contention, etc. I have not found that existing tools are insufficient in monitoring/dealing with VM swapping - OTOH I usually seek to keep that at zero and leave a little swap just to allow for some chance of alerting and recovery before OOM killer kicks in.
Digging the LKML thread, this appears to be the corresponding userland component for the OOM use-case:<p><a href="https://github.com/facebookincubator/oomd" rel="nofollow">https://github.com/facebookincubator/oomd</a><p>There was also more minimal proof-of-concept example posted by Endless OS guys:<p><a href="https://gist.github.com/dsd/a8988bf0b81a6163475988120fe8d9cd" rel="nofollow">https://gist.github.com/dsd/a8988bf0b81a6163475988120fe8d9cd</a>
Curious that there is no mention of the existing "memory" cgroup. On some desktop Linux, you'll find it here:<p><pre><code> ls -l /sys/fs/cgroup/memory/
</code></pre>
The 000-permission 'pressure_level' file controls asynchronous notifications to apps, advising prompt shedding of load. This is apparently the mechanism alluded to in a Googler's recent blog post, writing from the point of view of Go server coding: <a href="https://news.ycombinator.com/item?id=17551012" rel="nofollow">https://news.ycombinator.com/item?id=17551012</a>