We shipped a shader cache in the latest release of OBS and quickly had reports come in that the cached data was invalid. After investigating, the cache files were the correct size on disk but the contents were all zero. On a journaled file system this seems like it should be impossible, so the current guess is that some users have SSDs that are ignoring flushes and experience data corruption on crash / power loss.
Misleading headline since after testing eight more drives, none more failed.<p>2/12 is not nearly as dramatic as “half”, and the ones that lost data are the cheap brands as one would expect.
There is a flood of fake SSDs currently, mostly big brands. I've recently purchased counterfeit 1TB. It passes all the tests, performance is ok, it works... except it gets episodes where ioping would be anything between 0.7 ms and 15 seconds, that is under zero load. And these are quality fakes from a physical appearance perspective. The only way I could tell mine was fake is that the official Kingston firmware update tool would not recognize this drive.
Under long term heavy duty, I've routinely seen cheap modern platter outperform cheap brand name NVME.<p>There's some cost cutting somewhere. The NVMEs can't seem to sustain throughput.<p>It's been pretty disappointing to move I/O bound workloads over and not see notable improvements. The magnitude of data I'm talking about is 500-~3000GB<p>I've only got two NVME machines for what I'm doing so I'll gladly accept that it's coincidentally flaky bus hardware on two machines, but I haven't been impressed except for the first few seconds.<p>I know Everyone says otherwise which is why I brought it up. Someone tell me why I'm crazy<p>Edit: no, I'm not crazy. <a href="https://htwingnut.com/2022/03/06/review-leven-2tb-2-5-sata-ssd/" rel="nofollow noreferrer">https://htwingnut.com/2022/03/06/review-leven-2tb-2-5-sata-s...</a> this is similar to what I'm seeing with Crucial and Adata hardware, almost binary performance
Writes are completed to the host when they land on the SSD controller, not when written to Flash. The SSD controller has to accumulate enough data to fill its write unit to Flash (the absolute minimum would be a Flash page, typically 16kB). If it waited for the write to Flash to send a completion, the latency would be unbearable. If it wrote every write to Flash as quickly as possible, it could waste much of the drive's capacity padding Flash pages. If a host tried to flush after every write to force the latter behavior, it would end up with the same problem. Non-consumer drives solve the problem with back-up capacitance. Consumer drives do not have this. Also, if the author repeated this test 10 or 100 times on each drive, I suspect that he would uncover a failure rate for each consumer drive. It's a game of chance.
Does advertising a product as adhering to some standard, but secretly knowing that it doesn't 100%, count as e.g. fraud? I.e., is there any established case law on the matter?<p>I'm thinking of this example, but also more generally USB devices, Bluetooth devices, etc.
Meanwhile I'm over here jamming Micron 7450 pros into my work laptop for better sync write performance.<p>I have very little trust in consumer flash these days after seeing the firmware shortcuts and stealth hardware replacements manufacturers resort to to cut costs.
Losing flushes is obviously bad.<p>I wonder how much perf is on the table in various scenarios when we can give up <i>needing</i> to flush. If you know the drive has some resilience, say, 0.5s of time it can safely writeback during, maybe you can give up flushes (in some cases). How much faster is the app then?<p>It's be neat to see some low-cost improvements here. Obviously in most cases, just get an enterprise drive with supercapa or batteries onboard. But an ATX power rail that has extra resilience from the supply, or an add-in/pass-through 6-pin sata power supercap... that could be useful too.
It'd be nice if there were a database of known bad/known good hardware to reference. I know there's been some spreadsheets and special purpose like the USB-C cables Benson Leung tested.<p>Especially for consumer hardware on Linux--there's a lot of stuff that "works" but is not necessarily stable long term or that required a lot of hacking on the kernel side to work around issues
I am a bit annoyed that everyone here takes this at face value. There's 0 evidence given, not even the vendors and models are named to confirm this.<p>On a related note I tested 4 DDR5 Ram kits from major vendors - half of them corrupt data when exposed to UV light.
This has always been the case? At least it was a course learning when we wrote our own device drivers for minux, even the controllers on spinning metal fib about flush.
Cheap drives don't include large dram caches, lack fast SLC areas, and leave off super-capacitors that allow chips to drain buffers during a power-failure.<p>"Buy cheap, buy twice" as they say... =)
Without any more information this post is just bullshit. For example, it's not documented how the flushing has been done. On Linux, even issuing 'sync' is not enough: <a href="https://unix.stackexchange.com/questions/98568/difference-between-blockdev-flushbufs-and-sync-on-linux" rel="nofollow noreferrer">https://unix.stackexchange.com/questions/98568/difference-be...</a><p>The bottom answer especially states that "blockdev --flushbufs may still be required if there is a large write cache and you're disconnecting the device immediately after"<p>The hpdarm utility has a parameter for syncing and flushing device buffers themselves. Seems like all three should be done for a complete flush at all levels.
Name the offenders please.<p>I am sure it might be easy to see visually - a lack of substantial capacitor on the board would indicate a high likelihood of data loss.