> <i>And lastly, part three of the dream: it would be great if accessing storage from our services or batch jobs were as easy as accessing in-memory data.</i><p>I'm a believer!<p>The analytics/columnar store folk are well on the way. Voltron for instance,<p>> <i>Voltron Data is building open source standards that support a multi-language language future. Central to this vision is the Apache Arrow project.</i> <a href="https://voltrondata.com/blog/zero-copy-sharing-using-apache-arrow-and-golang" rel="nofollow">https://voltrondata.com/blog/zero-copy-sharing-using-apache-...</a><p>Being able to send data around between different data processing systems is definitely shifting from specially tuned narrow systems to a core piece of infrastructure. First for analytics/columnar loads. Eventually more.<p>There's other pieces ideally slotting into place. Dma-p2p between/across nvme, gpu's, and systems is the target. Is what the object/data stores need, increasingly, to be bringing to the table. Unison is itself an object store, in the fascinating live process kind of way.<p>Im actually struggling to think of good references. Apache Ray's Plasma object store has a thread with plenty of theorycrafting (but no progress) on how to add some mind of RDMA. There's not a ton of other systems dedicated just to holding objects in shared memory, afaik. <a href="https://github.com/ray-project/ray/issues/30094">https://github.com/ray-project/ray/issues/30094</a>