I had written this comment[0] about our pattern here a few months ago and we decided to turn it into a full blog post.<p>It's crazy how many weird bugs you can detect with such a trivial method.<p>For applications like a spreadsheet that have a million overlapping features, it's impossible to manually write unit tests for every combination of features, so randomized tests do a lot of heavy lifting.<p>[0] <a href="https://news.ycombinator.com/item?id=40876726">https://news.ycombinator.com/item?id=40876726</a>
FTA: Sometimes, for the most general tests, the only invariant you can assert is that the program doesn't crash. One of the most valuable tests we ever wrote at Row Zero is the one that just takes the top-level spreadsheet interface, and takes completely random actions on it<p>That strategy was also part of testing the original Mac. See <a href="https://www.folklore.org/Monkey_Lives.html" rel="nofollow">https://www.folklore.org/Monkey_Lives.html</a>.<p>As that article hunts at (“usually it didn't run for more than 20 minutes, even if it didn't crash, because the Monkey would invariably select the quit command”), there’s a black art in determining what “completely random” means.<p>You probably want the ability to change that depending on your testing needs. For example, if you’ve been working on the code for array formulas, the randomizer better create and edit them fairly frequently.
> "Eliminate all local excel files.."<p>Isn't the ability to interact with local environment, half the reason why people use Excel in the first place?<p>Okey maybe if they connect the cloud instance to a shared drive, that could indeed replace it