The day I started believing in unit tests

I was ambivalent on unit tests until I discovered how much the mere act of writing them was finding bugs.<p>I very vividly remember writing a test for a ~40 loc class of pure functions. I started out thinking the exercise was a waste of time. This class is simple, has no mutable state, and should have no reason to change. Why bother testing it?<p>By the time I was done writing the test I had found three major bugs in that 40 loc, and it was a major aha moment. Truly enlightening.

I started believing in unit tests the day I finished my patch, ran the program and watched it work perfectly. I then grudgingly wrote a test, ran it and immediately observed it fail. One of the test inputs was some garbage input and that exposed a poorly written error handling path. Humbling!<p>I still hate writing them and it grates on my aesthetic sense to structure code with consideration to making it testable, but if we want to call ourselves engineers we need to hold ourselves to engineering standards. Bridge builders do not get to skip tests.

I still like one of the defining characteristics of Unit Tests (paraphrasing Michael Feathers from memory): they are fast and cheap to run. Sure, they might not perfectly simulate production like integration tests, but they also don’t take hours burning cash in cloud infrastructure while risking failure from unrelated races dealing with those dependencies. You can use Unit Tests to get to a place where you’re fairly confident that the integration tests will pass (making that whole expensive affair cheaper to run).

It&#x27;s all degrees. Unit tests are great at finding examples of errors or correct behaviours. However they <i>prove</i> nothing and they definitely do not demonstrate the absence of errors.<p>They are often sufficient for a great deal of projects. If all it takes to convince you it&#x27;s &quot;good enough,&quot; are a handful of examples then that&#x27;s it. As much as you need and no less.<p>However I find we programmers tend to be a dogmatic bunch and many of us out there like to cling to our favoured practices and tools. Unit tests aren&#x27;t the only testing method. Integration tests are fine. Some times testing is not sufficient: you need proof. Static types are great but fast-and-loose reasoning is also useful and so you still need a few tests.<p>What&#x27;s important is that we sit down to think about specifying what it means for our programs to be, &quot;correct.&quot; Because when someone asks, &quot;is it correct?&quot; You need to as, &quot;with respect to what?&quot; If all you have are some hastily written notes from a bunch of meetings and long-lost whiteboard sessions... then you don&#x27;t really have an answer. Any behaviour is, &quot;correct,&quot; if you haven&#x27;t specified what it <i>should</i> be.

Good story.<p>I for one do not believe in Unit Tests and try to get LLM tooling to write them for me as much as possible.<p>Integration Tests however, (which I would argue is what this story is actually praising) are _critical components of professional software. Cypress has been my constant companion and better half these last few years.

I hate unit tests, though I am forced to write them to have my CI process not fail (I need 75% coverage or it won&#x27;t build) - so I have written thousands and thousands of them in the last few years - the problem I have: not a single time that I had a unit test fail that resulted in me finding a bug in my code - all I ever find are bugs in my unit test code - so pretty much seems like a waste of time to me.<p>Either I am writing really good code so there are no bugs, or I am really bad a writing unit testing code to find those bugs.

We can&#x27;t even have a consensus on what &quot;unit&quot; tests really are... Every company i work for has a different meaning for it. Some places consider a test &quot;unit&quot; when all the dependencies are mocked, some places consider a whole feature a &quot;unit&quot;.

So at work we would run tons of tests against the real service with a real database, seeding thousands of schemas to allow for parallel testing of tests that change state.<p>This takes 3 minutes, 1 if you use tmpfs. It only takes &lt;10 seconds if you dont run writing tests.<p>These actually cover most real world use cases for a query-engine we maintain.<p>Unit tests have their place for pieces of code that run based on a well defined spec, but all in all this integration or component-level testing is really what brings me the most value always.

The gem of this story is the author is not running unit test in what most folks understand a unit test is. As he also pointed out, he is executing the tests on target so it is more of an integration tests rather than unit tests. In the test that he is doing, it brings in new categories of potential faults. ie scheduling issues, memory constraints, interrupts servicing,

It is a little sad to see so many be so dismissive of unit tests. They aren&#x27;t a universal solution, which seems to be why they are written off in many cases, but they make your life so much easier in so many cases.<p>If you need to mock out 80% of a system to make your unit test work, then yes, it&#x27;s potentially pointless. In that case I&#x27;d argue that you should consider rewriting the code so that it&#x27;s more testable in isolation, that will also help you debug more easily.<p>What I like to do is write tests for anything that&#x27;s just remotely complex, because it make writing the actual code easier. I can continuously find mistakes by just typing &quot;tox&quot; (or whatever tool you use). Or perhaps the thing I&#x27;m trying to write functionality for is buried fairly deep in an application, then it&#x27;s nice to be reasonably sure about the functionality before testing it in the UI. Unit tests just makes the feedback loop much shorter.<p>Unlike others I&#x27;d argue that MOST projects are suited for unit testing, but there might be some edge cases where they&#x27;d provide no value at all.<p>On caveat is that some developers write pretty nasty unit tests. Their production code is nice and readable, but then they just went nuts in the unit tests and created a horrible unmaintainable mess, I don&#x27;t get why you&#x27;d do that.

unfortunately legitimate use cases for unit tests (like this) are pretty rare<p>in corporate codebases, overwhelmingly, unit tests are just mocked tests that enforce a certain implementation at the class or even individual method&#x2F;function level and pretend that it works, making it impossible to refactor anything or even fix bugs without breaking tests<p>such tests are not just useless, they&#x27;re positively harmful<p><a href="https:&#x2F;&#x2F;gist.github.com&#x2F;androidfred&#x2F;501d276c7dc26a5db09e893bb7c9f3ee" rel="nofollow noreferrer">https:&#x2F;&#x2F;gist.github.com&#x2F;androidfred&#x2F;501d276c7dc26a5db09e893b...</a>

I barely ever have unit tests flagging real issues. It&#x27;s always a chore to update them. Feature&#x2F;end to end tests though... Plenty of real issue flagged.

I have an unrelated (and most likely dumb) question about the article. When they talk about the inheritance relationship between &#x27;Thread&#x27; and &#x27;MyThread&#x27; in the example code in reference to the destructor methods, particularly here:<p>&gt; <i>Now, what happens when MyThread::singlepassThreadWork() uses a member variable of MyThread like foobar and we delete the MyThread object while the thread is still running? The destruction sequence is such that MyThread is deleted first and after that, the destructor of its parent object Thread runs and the thread is joined. Thus, there is a race condition: We risk accessing the vector foobar in singlepassThreadWork() after it was already deleted. We can fix the user code by explicitly stopping the thread in its destructor</i><p>What does it mean when they say &#x27;the destructor of its *parent* object Thread runs&#x27;? I&#x27;ve always thought that when you inherit from one class to another and then instantiate an object of said class, they&#x27;re just one object, so what do they mean when they make the distinction between &#x27;parent&#x27; and &#x27;child&#x27; object? When you have inheritance of say two classes, those would be two distinct objects instantiated in memory? Is there something I&#x27;m missing?

I think about unit tests being useful for getting more confidence that some deterministic, pure (mathematically speaking) and stateless piece of code that&#x27;s data in data out actually works, particularly when you change it.<p>If any of those conditions doesn&#x27;t hold the cost&#x2F;benefit certainly and even sometimes the absolute utility goes way down.<p>If I have to mock anything, in particular, or more generally care at all about any implementation details (ie side effects) then I just think might as well make this a full on automated functional test then.<p>As soon as fake code is introduced into the test its utility rapidly decays in time as the things it fakes themselves change.

How do you all feel about the need to rewrite a unit test when code gets refactored or business logic changes, isn’t that like a huge pita?

&gt;It is a little sad to see so many be so dismissive of unit tests.<p>You&#x27;re preaching to the choir. The overwhelming majority of people worship unit tests like dogma. There&#x27;s almost no point in saying the above. It&#x27;s like saying it&#x27;s a little sad to see some people who are so dismissive about eating and breathing to stay alive.<p>Your next part is the one that&#x27;s interesting. Mocking 80 percent of a system to get unit tests to work. I&#x27;ve seen so much of this from developers who don&#x27;t even realize the pointlessness of what theyre doing that it&#x27;s nuts. They worship test so much that they can&#x27;t see the nuance and the downside.<p>Take this article. This article is literally presenting evidence for why unit tests are bad. He literally created an error that would not have existed in the first place we&#x27;re it not for his tests. Yet he has to spin it in such a strange way to make it support the existing dogma of test test test.

To me an interesting distinction is not between unit and integration tests, but between tests that are run quickly as part of a gate on commits in CI, vs. tests that are run more asynchronously searching for bugs.<p>The former must run quickly, and it&#x27;s ok if the exact same test is run over and over. The latter need not run quickly, but benefits if new tests can be created and run, or if the tests incorporate randomness so they don&#x27;t do the same thing each time they are run.<p>Here, it seems he was using tests intended for the first purpose for the second purpose instead. That can work, as it did here, but I don&#x27;t think it&#x27;s optimal. Better to have more exploratory, randomized, property-based tests chugging away in the background to find weird new ways the code can fail.

I like pasting code into ChatGPT, then saying &quot;Write unit test(s) that demonstrate the bug(s) in this code&quot;. I have pre-instructions that say &quot;Show code only. Be concise&quot; to keep it simple. This has resulted in many learnings for me.

I believe in them. But unit tests are useless around useless humans. And there’s lots of those. A fine example is that time I wrote a test suite for a domain specific language parser. Someone wanted to break the language so they deleted the tests. New stuff was added without tests.<p>They confidently broke everything historically and looking forward. Then blamed it on me because it was my test suite that didn’t catch it. The language should not have been broken.<p>Everything only works if you understand what you are doing so every argument should be posed as both sides.

Most software doesn&#x27;t work the moment you stray from the expected path.<p>Whether that&#x27;s because most software isn&#x27;t tested competently or because software testing practices don&#x27;t deliver robust software is not yet clear.<p>I suspect that unit tests, and tests in general, will be considered a historical artifact from the time before we worked out how to write software properly.<p>For example, we don&#x27;t generally unit test things that a static type system checks for us. Maybe good enough type systems will remove the rest of them.

I don&#x27;t see how you can either believe or not believe, in a unit test. A unit test is what it is. It&#x27;s a real thing. It exists. Use it, or don&#x27;t.<p>How this topic can sometimes be about belief is beyond me. It&#x27;s like if a person found a screw driver and says, I now believe in screw drivers.<p>The topic of how people believe in unit tests, to me is proof that the world is screwed. We&#x27;re all screwed and everything is a screw driver.

Unit tests are great, but the way we often do unit tests is often as simple change detectors, which don’t say so much about correctness as much as they do about the code still doing what the programmer thinks it is doing (and tests often need to change if the code changes).<p>It would be nice if unit tests were more like interlocking evidence of system correctness, but right now we just have integration tests with poorer coverage for that.

Similar experience to this guy - didn&#x27;t believe in them initially, but now I&#x27;m a believer.<p>For what it&#x27;s worth, I find Copilot to be quite an exceptional help in writing unit tests ! A real game changer for me. Not only it takes care on most boilerplate code, but also kind of &#x27;guesses&#x27; what case I&#x27;m about to write - and sometimes even point me in a direction I would miss otherwise.

Unit tests is like buying insurance but you don’t know how much insurance has paid you if things go wrong. You spend a lot of time and effort to make your code testable, figure out what the useful test is and change the unit test when you do refactors in hope it speeds up your project, except you cannot really know if there was a net gain in speed&#x2F; reliability vs proper QA and other techniques

I like them because they help me to partition my code into units that are easier to write and test. Once they&#x27;re working, assembling the parts generally leads to a working project.<p>It also motivates me to get small pieces working and tested before I get to the finish line. Each successful test is a victory!

Said it before and will say it again. There is no replacement for unit test - it is the only thing that will give you flawless deployments. Not MIT degrees, not process, not managers - tests are the literally the only thing I&#x27;ve seen consistently produce flawless production deployments. It&#x27;s not a discussion.

Another thing unit tests are is focused. If you change just a small part of your code, you should only need to run a small fraction of your unit tests. Your unit test framework should support this selective execution of tests.

Unit tests saved me many times.<p>I&#x27;m happy that this article praises unit tests without forcing a TDD perspective to the reader. It presents it like a tool, not a religion, and that&#x27;s very refreshing.

I am troubled by the word belief, not just in the title, but in the comments here. Unit tests should not be doctrine, there is a time and a place. And, I feel that more often than not they are warranted.<p>We can argue about what granularity they should be, talk about functional programming, debate whether they should hit the database or not, but IMO all of those things miss the point. For me, in order of priority, unit tests provide the following benefits:<p>1) Make me write better, more decoupled code<p>2) Serve as documentation as to the intent of the code, and provide some expected use cases<p>3) Validate the code works as expected, (especially when &quot;refactoring&quot;, which is basically how I write all my code even from the start)<p>4) Help you when deleting code by exposing unexpected dependencies<p>You can argue against all of those points, and I often will, myself. It depends on the scale, importance, and lifetime of the project as to whether I will write unit tests. But, as soon as I think someone else will work on the code, I will almost always provide unit tests. In that scenario, they:<p>- Provide a way to quickly validate setup and installation was correct and the application functions<p>- Signal that the code was &quot;curated&quot; in some way. Someone cared enough to setup the test environment and write some tests, and that gives me a certain comfort in proceeding to work on the code.<p>- Provide a gateway into understanding why the application exists, and what some of the implementation details are.<p>So, thinking about the advantages I&#x27;ve outlined above, for me it would be very hard to say I don&#x27;t &quot;believe&quot; in unit tests. I just don&#x27;t always use them.

I don&#x27;t believe in unit tests as they are practiced. This, unfortunately, is the kind of thing that can work in principle, but the realities make it unusable.<p>There are multiple problems with unit tests, as they are implemented in the industry. And to make the unit tests usable and productive you need to make them so productive that it can offset those problems.<p>First of all, for unit tests to work <i>everybody</i> has to contribute quality unit tests. One team member writing unit tests well for his part of functionality is not going to move the needle -- everybody has to do this.<p>Unfortunately, it is rarely the case that all team members are able to write quality code this is the case for unit tests.<p>Usually, the reality is that given deadlines and scope, some developers will deprioritize focusing on writing good unit tests to instead deliver what business people do really care about -- functionality. Give it enough time and unit tests can no longer be trusted to perform its job.<p>Second, it is my opinion that refactoring is extremely important. Being able to take some imperfect code from somebody else and improve it should be an important tool in preventing code rot.<p>Unfortunately, unit tests tend to calcify existing code making it more expensive to change the functionality. Yes, more, not less expensive. To move a lot of stuff around, change APIs, etc. you will usually invalidate all of the unit tests that work around this code. And fixing those unit tests in my experience takes more effort than refactoring the code itself.<p>Unit tests are good for catching errors <i>AFTER</i> you have made the error. But my personal workflow is to prevent the errors in the first place. This means reading the code diligently, understanding what it does, figuring out how to refactor code without breaking it. Over the years I invested a lot of effort into this ability to the point where I am not scared to edit large swaths of code without ever running it, and then have everything work correctly on the first try. Unit tests are usually standing in the way.<p>I think where unit tests shine is small library code, utilities, where things are not really supposed to change much. But on the other hand, if they are not really supposed to change much there also isn&#x27;t much need to have unit tests...<p>The most paradoxical thing about unit tests is that teams that can write unit tests well can usually produce code of good enough quality that they have relatively little use of unit tests in the first place.<p>What I do instead of unit tests? I do unit tests. Yes, you read that correctly.<p>The trouble with unit tests is that everybody gets the part of what unit is wrong. Unit does not have to mean &quot;a class&quot;. Units can be modules or even whole services.<p>What I do is I test a functionality that matters to the client -- things I would have to renegotiate with the client anyway if I was to ever change it. These tests make sense because once they are written -- they do not need to change even as the functionality behind them is being completely rewritten. These test for what clients really care about and for this they bring a lot of bang for the buck.

The classic test rookie mindset is to test the functionality of the whole system, because that&#x27;s what really matters.<p>But in reality, unit testing every single function and method is where the vast majority of the benefit lies. Details <i>really</i> matter.<p>It took me some time to learn this, even after being told. It&#x27;s the same for most people. This little post will probably convince no one.<p>But maybe remember it when you finally get there yourself :)

I think he should have credited Tom Van Vleck with the &quot;three questions&quot; idea. It was published in ACM SIGSOFT Software Engineering Notes, vol 14 no 5 July 1989, pages 62-63, and you can read the whole thing here:<p><a href="https:&#x2F;&#x2F;multicians.org&#x2F;thvv&#x2F;threeq.html" rel="nofollow noreferrer">https:&#x2F;&#x2F;multicians.org&#x2F;thvv&#x2F;threeq.html</a><p>I hope he got permission to reproduce the comic.

Unit tests are not even well defined. What is a unit?