But why would Chrome's setup.exe need a 32MiB working set? If all it does is download and unpack some files in the background, it would seem like something that is doable in 32MiB or less.<p>The Windows BACKGROUND mode is useful for stuff like virus scanning or database log compaction, where you want to make some progress, but only if you can be certain not to hurt any foreground workloads. Back when we had hard drives, this wasn't actually trivial to achieve, because it is very easy to interfere with the foreground workload just by doing a few disk seeks every now and then.<p>I agree the 32MiB working setlimit is somewhat arbitrary and should be documented, but Windows is full of these arbitrary constants, like the 32kiB paging chunk for instance.<p>My recommendation for Chrome would be to stick with the background mode, and fix whatever problem is causing the working set to exceed 32MiB.
I found this paragraph particularly interesting:<p>> Trimming the working set of a process doesn’t actually save memory. It just moves the memory from the working set of the process to the standby list. Then, if the system is under memory pressure the pages in the standby list are eligible to be compressed, or discarded (if unmodified and backed by a file), or written to the page file. But “eligible” is doing a lot of heavy lifting in that sentence. The OS doesn’t immediately do anything with the page, generally speaking. And, if the system has gobs of free and available memory then it may never do anything with the page, making the trimming pointless. The memory isn’t “saved”, it’s just moved from one list to another. It’s the digital equivalent of paper shuffling.<p>I'd always been under the impression that as soon as memory was trimmed from the working set. Perhaps this <i>was</i> the case at some point, and was a reason for the PROCESS_MODE_BACKGROUND_BEGIN priority? As the blog mentions, the SetPriorityClass call has had this behavior since at least 2015, though I wouldn't be surprised if this behavior has existed for much longer.<p>As for why this "bug" hasn't been fixed, my guess is that it's due to a couple of factors:<p>- Windows has become fairly good over the years at keeping the core UI responsive even when the system is under heavy load.<p>- There are plenty of ways to reduce memory/CPU usage that <i>don't</i> involve a call to SetPriorityClass. I'd wager that setting a process's priority class is not the first thing that would come to mind.<p>- As a result of the previous two points, the actual number of programs using that call is quite small. I'd actually be interested in knowing what, if any, parts of Windows use it.<p>(As a side note, if there <i>was</i> a bug in a Windows API function, how would you even report it?
Bruce - any chance you could make your ETW videos available again? I see they used to be available through a Microsoft partnership of some type but that seems to have ended a few years ago.<p>I’d love to learn more about ETW and your videos seem like a good place to start. If anybody else has other recommendations, please share!
Seriously? Windows has had this bizarre “working set” concept effectively forever. I thought it was weird when Windows 2000 was new, and I still think it’s weird. In the land of page replacement algorithms, there are many of various degrees of cleverness, and then there’s “we exceeded an arbitrarily limit — nothing else needed that memory, but we’re going to zap it out of the page tables anyway!”<p>(x86 has had “accessed” tracking for a long long time. What would be wrong with keeping over-the-working-set pages present but not “accessed” and tracking accesses without causing page faults? Or at least only trying to expire pages when there’s some degree of memory pressure.)
Wow, PROCESS_MODE_BACKGROUND_BEGIN is quite the footgun. I wonder if this started out as a hack to reduce the impact of OEM bloatware on benchmarks. Is there an easy way to list all processes that use this priority class?
<i>Trimming the working set of a process doesn’t actually save memory. It just moves the memory from the working set of the process to the standby list. Then, if the system is under memory pressure the pages in the standby list are eligible to be compressed, or discarded (if unmodified and backed by a file), or written to the page file. [… If] the system has gobs of free and available memory then it may never do anything with the page, making the trimming pointless. The memory isn’t “saved”, it’s just moved from one list to another. It’s the digital equivalent of paper shuffling.</i><p>This sounds like it should be basically free in a memory-unconstrained system - is the system really just spending all its time managing these lists? Why is it so expensive?
So he didn't report the bug to Microsoft? His final comments were just to say:<p>"This issue has been known for eight years, on many versions of Windows, and it still hasn’t been corrected or even documented. I hope that changes now."
Haha :) I just checked MSDN docs and vioala:
PROCESS_MODE_BACKGROUND_BEGIN - Not supported under Win2003 or Windows XP.<p>Another improvement from Microsoft without really understanding their own OS.
I wonder if this is some cross-pollination legacy of Windows Phone. On such a device, background processes are still necessary, memory is a much tighter constraint, and enough pages might be mapped directly from device flash for auto-eviction to make sense.
I'm not sure I understand, why not just manage your own memory instead of asking the OS to magically understand your memory use-case and properly manage/limit?<p>If the memory isn't owned by the OS, that's probably the wrong actor to manage it.<p>They say that, instead, they used an undocumented API they obviously didn't understand. Why? If you are concerned about using up too much memory, then <i>do something about it</i>