Test smarter, not harder (2020)

<i>Don&#x27;t write tests for things that can be more effectively tested in other ways, and lean on other correctness methodologies as much as possible. These include: code review</i><p>I&#x27;ll take tests any day over the luck of getting an individual to pick up flaws in the code review.

&quot;Don&#x27;t listen to Uncle Bob! &quot;<p>+1<p>In general, I&#x27;ve learned to listen to people who actually do something rather than people writing books and giving shows about that something.

I&#x27;ve been on both types of projects.<p>A big side effect of writing tests extensively is that the code is often more thoughtfully written and in smaller chunks. No one wants code that is hard to test or large enough that it requires extensive mocking and faking.<p>Gut feel is that the code quality could be better on more deeply tested code.

&gt; Only write necessary tests — specifically, tests whose estimated value is greater than their estimated cost.<p>Definitely. Ensuring that basic functionality works and that no awful regressions are introduced, using high-level tests (e2e, integration, based on realistic test data) is great. Such tests allow to refactor refactor with sufficient confidence.<p>Having hundreds of tests for trivial functions that might not even be relevant in day to day use for most customers which need to be rewritten or thrown away when refactoring - not so much.

While I think writing tests for the &quot;happy&quot; path builds good momentum, especially for a code base with no existing tests, the real value lies in writing tests that exercise the edge cases. Guy Steele put it best:<p>&quot;I spent a lot of time worrying about edge cases. That’s something I learned from Trenchard More and his array theory for APL. His contention was that if you took care fo the edge cases then the stuff in the middle usually took care of itself.&quot;

Yeah, we&#x27;ve made the mistake of testing too much in our last configuration management. A test suite can be strangling if it starts to restrict every single change to the code base, unless you touch like 3-4 tests per configuration file deployed to a server.<p>At this point, most of our tests are much higher level by default - setup 3 nodes, run the config management against it, check if the cluster formed. If the cluster formed, the setup from the config management can&#x27;t be that wrong.<p>From there, we&#x27;re adding tests in an outage&#x2F;bug-driven way. If it doesn&#x27;t deploy and load a config correctly into the application, that&#x27;s a test to write. Or, we recently almost ran into an outage because some failure earlier on caused the config management to start tearing the cluster apart because of missing metadata. So we added a test that it cancels the rollout in such a case.<p>Those tests are valuable, because we can rely on these issues not cropping up again. Some test breaking on yaml formatting isn&#x27;t as much.

I enjoyed it.<p>I tend to use test harnesses. I write about my approach, here: <a href="https:&#x2F;&#x2F;littlegreenviper.com&#x2F;miscellany&#x2F;testing-harness-vs-unit&#x2F;" rel="nofollow">https:&#x2F;&#x2F;littlegreenviper.com&#x2F;miscellany&#x2F;testing-harness-vs-u...</a><p>That said, my testing code generally dwarfs my implementation code; whether harness or unit. Part of the reason is that I spent 27 years at a company, where Quality was a religion, and it was very important, not to faceplant in public.<p>I learned to write very effective tests, very quickly. Most of my test harnesses are &quot;full-fat&quot; applications, that many people would want to ship. I&#x27;ll often throw them together in a day or so. It helps me to practice for The Rilly Big Shoo.<p>I use almost every line of code I write, in shipping software, so it is a very practical approach.

&gt;Some of the costs associated with tests are:<p>&gt;the time taken to write them.<p>&gt;the time they add to the test suite on every run.<p>&gt;the time to maintain them - understand them, debug them, change them when other things change.<p>To me these are not unit tests in the first place. A unit test is by definition (at least the venerated M. Feathers one):<p>simple<p>fast<p>used either to help understand code behaviour or flag when this changes.<p>If you&#x27;re complaining about unit tests and your unit tests arent unit tests in the first place, then no wonder you won&#x27;t like TDD as a design approach.

I see testing as a partial solution to the more general question - does this thing work. In software product development, someone comes up a problem to which the solution is to build something, someone then build it, someone then test it to make sure it functionally works, then someone test whether they are addressing the right problem. Someone can be same guy or split into different teams. But I duplicated effort in answering the question of &quot;does this thing work&quot;

But I have it on good authority I need 90% unit test coverage for branches, functions, and lines, and this is what makes code &quot;quality&quot;.