Abstract at: <a href="https://www.amazon.science/publications/millions-of-tiny-databases" rel="nofollow">https://www.amazon.science/publications/millions-of-tiny-dat...</a><p>> <i>...Physalia is a transactional key-value store, optimized for use in large-scale cloud control planes, which takes advantage of knowledge of transaction patterns and infrastructure design to offer both high availability and strong consistency to millions of clients. Physalia uses its knowledge of data center topology to place data where it is most likely to be available. Instead of being highly available for all keys to all clients, Physalia focuses on being extremely available for only the keys it knows each client needs, from the perspective of that client.</i><p>> <i>...We believe that the same patterns, and approach to design, are widely applicable to distributed systems problems like control planes,configuration management, and service discovery.</i><p>It'd be interesting to constrast this approach with Route53's or IAM's datastore which need to be globally-replicated with time-bounded eventually-consistent reads, and transactional but verifiable writes.<p>I hope AWS begins publishing about S3, now. One can look at the patents AWS engineers author to get a feel for some of the internals, but they are (intentionally?) hard to read.<p>For instance, patents filed by two of the many S3 founding-engineers: <a href="https://patents.google.com/?inventor=James+Christopher+Sorenson%2c+III,Allan+H.+Vermeulen&oq=inventor:(James+Christopher+Sorenson%2c+III)+or+inventor:(Allan+H.+Vermeulen)" rel="nofollow">https://patents.google.com/?inventor=James+Christopher+Soren...</a><p>Also see:<p><a href="https://aws.amazon.com/builders-library/" rel="nofollow">https://aws.amazon.com/builders-library/</a><p><a href="https://research.google/pubs/" rel="nofollow">https://research.google/pubs/</a>