Run Python Applications Efficiently with malloc_trim

Memory usage is an interesting problem, because the failure mode is so much more painful (crashes and complete lock-ups) than high CPU usage.<p>And even if you return memory to the OS, that doesn&#x27;t actually solve too-high memory usage.<p>Some things you can do:<p>The article mentions `__slots__` for reducing object memory use, and other approaches include just having fewer objects: for example, a dict of lists uses far less memory than a list of dicts with repeating fields. And you can also in many cases switch to a dataframe with Pandas, saving even more memory (<a href="https:&#x2F;&#x2F;pythonspeed.com&#x2F;articles&#x2F;python-object-memory&#x2F;" rel="nofollow">https:&#x2F;&#x2F;pythonspeed.com&#x2F;articles&#x2F;python-object-memory&#x2F;</a> covers all of those).<p>For numeric data, a NumPy array gets rid of the per-integer overhead for Python, so a Python list of numbers use way more memory than an equivalent NumPy array (<a href="https:&#x2F;&#x2F;pythonspeed.com&#x2F;articles&#x2F;python-integers-memory&#x2F;" rel="nofollow">https:&#x2F;&#x2F;pythonspeed.com&#x2F;articles&#x2F;python-integers-memory&#x2F;</a>).

There&#x27;s... a lot of hand waving in this article and no numbers.<p>I&#x27;d be pretty surprised if `malloc_trim` had a significant effect on cpython memory usage as most python memory gets allocated in 256KiB &quot;arenas&quot;, which, what with fragmentation, are unlikely to _ever_ be reclaimed.<p>On the other hand, the article dismisses threads with some vagueries around the GIL, suggesting people need to reach straight for processes if they&#x27;re serious. Really, unless your code has almost no I&#x2F;O or C-accelerated, GIL-less sections, if you&#x27;re not using both threads <i>and</i> processes, you&#x27;re just burning memory unnecessarily.<p>(edit: oh and then there&#x27;s async but I&#x27;m a bit old-fashioned for that)

I discovered a similar trick few years ago: <a href="https:&#x2F;&#x2F;stackoverflow.com&#x2F;questions&#x2F;35660899&#x2F;reduce-memory-fragmentation-with-malloc-mmap-threshold-and-malloc-mmap-max" rel="nofollow">https:&#x2F;&#x2F;stackoverflow.com&#x2F;questions&#x2F;35660899&#x2F;reduce-memory-f...</a><p>The idea is to tweak when `mmap` or `malloc` are used by the Python interpreter. One allows memory to be released to the OS right away, whereas the other is not.<p>It is a useful trick if your application is generating lots of small objects.

From TFA...<p>&gt; For example, GC pauses are notorious in other managed memory languages like Java.<p>(includes link to 2017 article:<p><a href="https:&#x2F;&#x2F;dzone.com&#x2F;articles&#x2F;how-to-reduce-long-gc-pause" rel="nofollow">https:&#x2F;&#x2F;dzone.com&#x2F;articles&#x2F;how-to-reduce-long-gc-pause</a><p>)<p>The author might like to know that since 2017, Java has two of the most sophisticated Garbage Collectors on the planet.<p>1. Red Hat&#x27;s Shenandoah GC<p>2. Oracle&#x27;s ZGC<p>(I could also mention Azul Systems work on its CCCC)<p>Both Shenandoah and ZGC claim to run on multi terabyte heaps with 1 ms max pause times.

Somewhat related, although this is for a huge scale:<p><i>Adaptive process and memory management for Python web servers</i> <a href="https:&#x2F;&#x2F;instagram-engineering.com&#x2F;adaptive-process-and-memory-management-for-python-web-servers-15b0c410a043" rel="nofollow">https:&#x2F;&#x2F;instagram-engineering.com&#x2F;adaptive-process-and-memor...</a><p><i>Previously we used two per-worker thresholds to control respawn: reload-on-rss and evil-reload-on-rss.</i><p>...<p><i>However, since worker respawn is expensive (i.e. there are warm-up costs, LRU cache repopulation, etc..)</i><p>For a smaller scale, I always wondered if FastCGI being the &quot;default&quot; would have saved a lot of headaches. Your workers just get recycled automatically all the time.<p>If you can make your startup fast enough (and I think most apps can), then you can just let the OS do its job. Although it&#x27;s true that Python can be really slow to start if you import many modules...<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=24683304" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=24683304</a>

If the app is long running and forks, you may also want to look into gc.freeze() (added on 3.7) which will save you from copying the imports memory over time and make the GC shorter.

&gt; libc = ctypes.CDLL(&quot;libc.so.6&quot;)<p>glibc SONAME is different on some architectures.<p>You can use ctypes.CDLL(None) instead, which should work everywhere.

if your api surface supports it, a simple and effective solution is to restart the process

It would also make sense to compress large strings in memory as it got tight.