I agree with many of the concerns the author raises, but I'm left with the question:<p>Given all this, what does layering give us?<p>It gives some deduplication, but only a crude form. It gives some reproducibility from building off a well-known base and tag, but not full reproducibility. It gives some security benefit from building off a well-known base, but not as large a benefit as standard package managers provide.<p>I would be excited to see a image distribution system based off of something like casync, maybe with an initial rootfs formed through image-focused distributions like yocto[1]. The embedded device ecosystem has been concerned with reproducibility, image signing, and incremental updates for awhile and I think their approaches are very applicable to container images.<p>[1]: <a href="https://www.yoctoproject.org/" rel="nofollow">https://www.yoctoproject.org/</a>
Hmm thinking about it...a ZFS-filesystem on a file could solve ~all the problems with the exception of a good performing dedup and shrinking the file/device.<p>But really good article, quite interesting tar-history.
Great post, but:<p>> <i>How Do We Get It?</i><p>> <i>I’m afraid to find that out, you’ll need to wait until the next instalment. I hope to get it complete in a few weeks (I was hoping to have a PoC available with the next instalment but that’s just a silly goal at this point).</i><p>> <i>If you want a taste though, the general idea is that we can resolve most of the issues I’ve listed and gain most of the properties we want by creating our own format that is built on top of the OCI content-addressable store, and is a Merkle tree with content-defined chunking of file contents. The basic idea is very similar to backup tools like restic or borgbackup. Since OCI has smart pointers, we can define a few new media-types, and then our new format would be completely transparent to OCI image tools (as opposed to opaque tar archives).</i><p>> <i>But you’ll learn all about that next time. Thanks for reading, and happy hacking!</i><p>Any follow-up on this 3 year old post? This appears to be the final entry on cyphar's blog. Did this ever happen? I've done some hunting but can't find any stated plans for OCIv2 from Cyphar.<p>There was a hack-session in march 2020 on this topic: <a href="https://hackmd.io/@cyphar/ociv2-brainstorm" rel="nofollow">https://hackmd.io/@cyphar/ociv2-brainstorm</a> . Really fascinating document. It has a section for each problem, & does a great job citing & discussing prior work in the area. Was a delight to stumble into this. But still seems very formative. It felt like cyphar was already onto a plan, well before this date. I don't see any evidence of v2 forming in the official repo, <a href="https://github.com/opencontainers/image-spec/tree/main/specs-go" rel="nofollow">https://github.com/opencontainers/image-spec/tree/main/specs...</a> . Cyphar does still seem active in the project.<p>Seems there was a come-together call on the mailing list last June, calling for trying to extend-their-way-forwards rather than do a big revamp/rewrite: <a href="https://groups.google.com/a/opencontainers.org/g/dev/c/6DGtH48fGOg/m/1kUGGhRPBAAJ" rel="nofollow">https://groups.google.com/a/opencontainers.org/g/dev/c/6DGtH...</a>
Since I learned that OCI images are pretty much archives with metadata, I've been thinking about applying that to other stuff.<p>Like, it would be cool to have all my python virtualenvs saved in bundles so they could be reused by other others, without all the container bloat.
Discussed at the time:<p><i>Road to OCIv2 Images: What's Wrong with Tar?</i> - <a href="https://news.ycombinator.com/item?id=18965881" rel="nofollow">https://news.ycombinator.com/item?id=18965881</a> - Jan 2019 (33 comments)
Squashfs instead! There was another post about this recently, replace tar with squashfs solves a bunch of problems. Is arguably harder to create and had a smaller ecosystem than tar, but capability wise it would be great.
The original title, "The Road to OCIv2 Images: What's Wrong with Tar?", is a lot better IMHO. Otherwise, a great history lesson on the tar format, and why it's not a great fit for OCI images.