Adding 16 kb page size to Android

In Debian kernel we&#x27;ve very recently enabled building an ARM64 kernel flavour with 16KiB page size and we&#x27;ve discussed adding a 64KiB flavour at some point as is the case for PowerPC64 already.<p>This will likely reveal bugs that need fixing in some of the 70,000+ packages in the Debian archive.<p>That ARM64 16KiB page size is interesting in respect of the Apple M1 where Asahi [0] identified that the DART IOMMU has a minimum page size of 16KiB so using that page size as a minimum for everything is going to be more efficient.<p>[0] <a href="https:&#x2F;&#x2F;asahilinux.org&#x2F;2021&#x2F;10&#x2F;progress-report-september-2021&#x2F;" rel="nofollow">https:&#x2F;&#x2F;asahilinux.org&#x2F;2021&#x2F;10&#x2F;progress-report-september-202...</a>

&gt; The very first 16 KB enabled Android system will be made available on select devices as a developer option. This is so you can use the developer option to test and fix<p>&gt; once an application is fixed to be page size agnostic, the same application binary can run on both 4 KB and 16 KB devices<p>I am curious about this. When could an app NOT be agnostic to this? Like what an app must be doing to cause this to be noticeable?

Seems pretty dubious to do this without adding support for having both 4KB and 16KB processes at once to the Linux kernel, since it means all old binaries break and emulators which emulate normal systems with 4KB pages (Wine, console emulators, etc.) might dramatically lose performance if they need to emulate the MMU.<p>Hopefully they don&#x27;t actually ship a 16KB default before supporting 4KB pages as well in the same kernel.<p>Also it would probably be reasonable, along with making the Linux kernel change, to design CPUs where you can configure a 16KB pagetable entry to map at 4KB granularity and pagefault after the first 4KB or 8KB (requires 3 extra bits per PTE or 2 if coalesced with the invalid bit), so that memory can be saved by allocating 4KB&#x2F;8KB pages when 16KB would have wasted padding.

A little additional background: iOS has used 16KB pages since the 64-bit transition, and ARM Macs have inherited that design.

RHEL tried that in that past with 64KB on AARCH64, it led to MANY bugs all across the software stack, and they eventually reverted it - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27513209">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=27513209</a>.<p>I&#x27;m impressed by the effort on Google&#x27;s side, yet I&#x27;ll be surprised if this effort will pay off.

I wonder how much help they had by asahi doing a lot of the kernel and ecosystem work anablibg 16k pages.<p>RISC-V being fixed to 4k pages seems to be a bit of an oversight as well.

<p><pre><code> iOS has had 16K pages since forever. OSX switched to 16K pages in 2020 with the M1. Windows is stuck on 4K pages, even for AArch64. Linux has various page sizes. Asahi is 16K.</code></pre>

Now I wonder: Does increased page size have any negative impacts on I&#x2F;O performance or flash lifetime, e.g. for writebacks of dirty pages of memory-mapped files where only a small part was changed?<p>Or is the write granularity of modern managed flash devices (such as eMMCs as used in Android smartphones) much larger than either 4 or 16 kB anyway?

Not entirely related (except the block size), but I am considering making and standardizing a system-wide content-based cache with default block size 16KB.<p>The idea is that you&#x27;d have a system-wide (or not) service that can do two or three things:<p>- read 16KB block by its SHA256 (also return length that can be &lt;16KB), if cached<p>- write a block to cache<p>- maybe pin a block (e.g. make it non-evictable)<p>I would be like a block-level file content dedup + eviction to keep the size limited.<p>Should reduce storage used by various things due to dedup functionality, but may require internet for corresponding apps to work properly.<p>With a peer-to-peer sharing system on top of it may significantly reduce storage requirements.<p>The only disadvantage is the same as with shared website caches prior to cache isolation introduction: apps can poke what you have in your cache and deduce some information about you from it.

I see they have measured improvements in the performance of some things. In particular, the camera app starts faster. Small percentage, but still real.<p>Curious if there are any other changes you could do based on some of those learnings? The camera app, in particular, seems like a good one to optimize to start instantly. Especially so with the the shortcut &quot;double power key&quot; that many phones&#x2F;people have setup.<p>Specifically, I would expect you should be able to do something like the lisp norm of &quot;dump image?&quot; Startup should then largely be loading the image, not executing much if any initialization code? (Honestly, I mostly assume this already happens?)

Can someone explain those numbers to me?<p>5-10% performance boost sounds huge. Wouldn&#x27;t we have much larger TLBd if page walk was really this expensive?<p>On the other hand 9% increase in memory usage also sounds huge. How did this affect memory usage that much?

I would expect that this increases the gap between new and old phones &#x2F; makes old phones unusable more quickly: new phones will typically have enough RAM and can live with the 9% less efficient memory use, and will see the 5-10% speedup. Old phones are typically bottlenecked at RAM, now 9% earlier, and reloading pages from disk (or swapping if enabled) will have a much higher overhead than 5-10%.

Good. It&#x27;s about time. 4KB pages come down to us from 32-bit time immemorial. We didn&#x27;t bump the page size when we doubled the sizes of pointers and longs for the 64-bit transition. 4KB has been way too small for ages, and I&#x27;m glad we&#x27;re biting the minor compatibility bullet and adopting a page size more suited to modern computing.

Time to grab some THP popcorn...

No mention of Apple on the page. Apple has been using 16K pages for years now.