This is really interesting and something I've been thinking about for a while now. The SEMANTICS[1] doc details what is and isn't supported from a POSIX filesystem API perspective, and this stands out:<p><pre><code> Write operations (write, writev, pwrite, pwritev) are not currently supported. In the future, Mountpoint for Amazon S3 will support sequential writes, but with some limitations:
Writes will only be supported to new files, and must be done sequentially.
Modifying existing files will not be supported.
Truncation will not be supported.
</code></pre>
The sequential requirement for writes is the part that I've been mulling over whether or not it's actually required in S3. Last year I discovered that S3 can do transactional I/O via multipart upload[2] operations combined with the CopyObject[3] operation. This should, in theory, allow for out of order writes, existing partial object re-use, and file appends.<p>[1] <a href="https://github.com/awslabs/mountpoint-s3/blob/main/doc/SEMANTICS.md">https://github.com/awslabs/mountpoint-s3/blob/main/doc/SEMAN...</a><p>[2] <a href="https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html" rel="nofollow">https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuove...</a><p>[3] <a href="https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObject.html" rel="nofollow">https://docs.aws.amazon.com/AmazonS3/latest/API/API_CopyObje...</a>
After teaching customers for years that S3 shouldn't be mounted as a filesystem because of its whole object-or-nothing semantics, and even offering a paid solution named "storage gateway" to prevent issues between FS and S3 semantics, it's rather interesting they'd release a product like this.<p>Amazon should really just fix the underlying issue of semantics by providing a PatchObjectPart API call that overwrites a particular multipart upload chunk with a new chunk uploaded from the client. CopyObjectPart+CompleteMultipartUpload still requires the client to issue CopyObjectPart calls for the entire object.
> This is an alpha release and not yet ready for production use. We're especially interested in early feedback on features, performance, and compatibility. Please send feedback by opening a GitHub issue. See Current status for more limitations.
JungleDisk was backup software I used ~2009 that allowed mounting S3. They were bought by Rackspace and the product wasn't updated. Seems to be called/part of Cyberfortress now.<p>Later I used Panic's Transmit Disk but they removed the feature.<p>Recently I'd been looking at s3fs-fuse to use with gocryptfs but haven't actually installed it yet!<p><a href="https://github.com/s3fs-fuse/s3fs-fuse">https://github.com/s3fs-fuse/s3fs-fuse</a><p><a href="https://github.com/rfjakob/gocryptfs">https://github.com/rfjakob/gocryptfs</a>
There's a similar project under awslabs for using S3 as a FileSystem within the Java JVM: <a href="https://github.com/awslabs/aws-java-nio-spi-for-s3">https://github.com/awslabs/aws-java-nio-spi-for-s3</a>
There's some really confusing use of unsafe going on.<p>For example I'm not sure what they're doing here:<p><a href="https://github.com/awslabs/mountpoint-s3/blob/main/mountpoint-s3-crt/src/io/stream.rs#L111">https://github.com/awslabs/mountpoint-s3/blob/main/mountpoin...</a>
Something similar that I've been using for a while now for an S3 filesystem: Cyberduck[0]<p>[0] <a href="https://cyberduck.io/s3/" rel="nofollow">https://cyberduck.io/s3/</a>
For anyone looking to mount S3 as file system, I will suggest giving rclone a shot. It can mount, copy and do all file operations not just on s3 but on a wide range of cloud providers, you can also declare a remote as encrypted so it does client side encryption
I would suggest changing the title to "Mountpoint-S3 - ..." as that's the project name to avoid confusion with mountpoint(1): <a href="https://man7.org/linux/man-pages/man1/mountpoint.1.html" rel="nofollow">https://man7.org/linux/man-pages/man1/mountpoint.1.html</a>
It would be interesting to see how this compares to other solutions in this space, such as s3fs (the FUSE driver, not the python package), goofys, and the rclone mount feature, among others. This certainly has less features (notably, mounts are read-only!).
If you are looking for something supports atomic rename, you can checkout Blobfuse2[0] + ADLS Gen2[1]<p>Disclaimer: work for MSFT<p>[0] <a href="https://github.com/Azure/azure-storage-fuse">https://github.com/Azure/azure-storage-fuse</a><p>[1] <a href="https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-namespace" rel="nofollow">https://learn.microsoft.com/en-us/azure/storage/blobs/data-l...</a>
Couldn't tell from the README, does this do any sort of cache management or LRU type thing? In other words, does it fetch the underlying S3 object in real time, and then eventually eject them from memory and/or the backing FS when they haven't been used for a while?