Which of these guarantee that data is committed to storage?<p><pre><code> flush()
fsync()
close()
</code></pre>
What about if the storage is on NFS?<p>The reality is that, unless you're heavily reliant on platform-specific behaviour and are certain you're not, for example, dealing with files over NFS, or hardware which happens to relax any guarantees you thought you had, nothing is really guaranteed.<p>The best defense I've seen against the uncertainty of whether files are written is to use the strategy required when writing to a Maildir-format mailbox. This employs a directory used for temporary writes to unique filenames, followed by a move into place in a 'real' directory, where files should be accessible once write is deemed to be complete.<p>Those who've been around for some time may have experienced corrupt mailboxes where the storage format was something mbox-alike - with all messages held in one file (and there's no fancy database-style transaction handling to deal with integrity).<p>Any kind of cross-platform development where files are written to is made more difficult and less performant, thanks to this problem. In many cases, I'm afraid this is poorly understood, or ignored. I don't think there's even a way to measure how often data corruption occurs due to this not being accounted for.
I don’t trust the disk to write data. I trust it to report back when the data is written. If the disk confirms the write, I assume it won’t somehow lose that data if there’s a power loss shortly after confirmation. If the disk doesn’t confirm the write, all bets are off.<p>The problem is that disks are confirming writes before they actually store the write in the non volatile target.
I can't seem to dig this out but I vaguely remember some disks (HDD, not SSD) had this mode when data is:<p><pre><code> * acknowledged & buffered in the cache
* not physically written to the plate (yet)
* but still guaranteed to be written, even when power fails
</code></pre>
Anybody can confirm that?
Related: <a href="https://www.deconstructconf.com/2019/dan-luu-files" rel="nofollow">https://www.deconstructconf.com/2019/dan-luu-files</a> mostly about how the challenges in filesystems in writing data to disk and a bit about disks as well.
> macOS has two low-level ways of flushing disk writes: fsync() and the F_FULLFSYNC command in fcntl().<p>Through the recent spat / flareup (which I expect is what triggered this article), I learned that macOS has an intermediate level called F_BARRIERFSYNC whose purpose is pretty much what it says on the tin.<p>But of course as the article notes that only goes as far as to send the relevant commands to the hardware, whether the hardware responds appropriately is, from what I gathered (from discussions here and elsewhere) a whole other kettle of fish.
Fact of the matter is, when a disk suddenly loses power, all bets are off. You can design intricate and complex systems of buffers all you want, but when the chips are down, at some point bits need to be flipped in the right positions to store the data, and for this you need electricity.<p>The only way around this, would be if disks incorporated some kind of internal backup power system, which kicks in and flushes its internal buffers in case of a power loss.<p>However, even that doesn't really solve the problem, it just moves it somewhere else...because the internal backup power is just another system that can fail.