> <i>Knowing that software is well tested is even more difficult. One widely accepted measure is that of branch coverage, the ratio of basic code blocks that were exercised by some test, to the total number of code blocks in the system under test</i><p>Widely accepted, but still surprisingly easy to game if you want to do it (and have lazy or no code review). Code coverage only tells you that some test has <i>called</i> the code, not that it has actually checked the results it gets. So, if you write a unit test that "exercises" all the various branches of a method and simply ignores (or performs only minimal checks on) the return values, you get 100% code coverage and, as a bonus, a test that's unlikely to ever break! This might still be useful to make sure that the code doesn't throw any exceptions etc., but it's contrary to the spirit of how unit tests should be written...
> One widely accepted measure is that of branch coverage, the ratio of basic code blocks that were exercised by some test, to the total number of code blocks in the system under test<p>I know that statement is true, but WHY is it true? I have never seen anyone look at a coverage report and take any useful action. Instead I see management looking at those reports for a number to rate people on, but no useful action is ever taken. When I've looked at the reports I keep finding trivial code that would be hard to test for little gain.<p>I eliminated the report and nobody misses it. Instead we report to management total number of tests run (test cases, and each assert). Which is easy to game, but since the number naturally increases fast isn't worth the bother.
Thank you for posting! We have a rust application that we compile to wasm and run. (there's no way to run it by compiling to native code.) We have tests that compile our code to wasm and run them in our execution environment. We want to get code coverage set up for our tests, but we had no idea how to do that in our environment.<p>This article suggests a way to have code coverage probes automatically added to various locations in our source. I think I would also need some way to read coverage information from the rust program itself, which seems straightforward. But I wonder if anyone knows if this has already been done? I know about the llvm-based coverage reports, but I don't know if they can easily be adapted to this purpose
I like the term "industrial-strength" transformation system. That's what I'm building, as it happens. I wonder if the existence of a completely free one might not be so good for this company...
related to the "This technology has been used to construct production test
coverage tools for ... PARLANSE (a parallel language used to
implement DMS itself)" part:<p><a href="https://en.wikipedia.org/wiki/DMS_Software_Reengineering_Toolkit" rel="nofollow">https://en.wikipedia.org/wiki/DMS_Software_Reengineering_Too...</a><p><a href="https://www.semanticdesigns.com/Products/Parlanse/examples.html" rel="nofollow">https://www.semanticdesigns.com/Products/Parlanse/examples.h...</a>