I can't agree with this. In his example (Transactions), he assumes that there is perfect knowledge of the underlying data and perfect knowledge on the assumptions made by the teams in the corner cases of each module, which is not the case in practice.<p>Very very often, (one of the most common source of errors in programming large systems), two parts of a program are doing the "right" thing, within their assumption, but when combined together you erroneous results.<p>Unit tests will not help you find them, as they cover the "is this gear/part working", but not the whole machine itself, e.g. is the engine actually running well...<p>In my experience, higher level system tests have a much higher ROI and real work value then simple unit tests.<p>Unit tests just test for made up "hypothetical" simplistic scenarios, and usually are written by the same dev. that did the functionality, and they will not expose blind spots on the implementation, while higher level system/integration tests are more likely to test on real world scenarios.
IMHO integration tests are not there to validate the functionality that you're building - Kent is right here, if you rely on integration tests to find bugs, either your modules are either, or your design is bad (e.g. cross-module code paths that break encapsulation, etc).<p>Integration tests are a god send when refactoring something that used to work in the first place. I'm currently switching 3rd party API providers in a system developed by somebody else long long time ago - all gritty corner cases need to continue working. The documentation is way outdated, especially the design docs. But, thankfully, the system has really good integration tests; so I'm super comfortable replacing modules one at a time, and knowing that functionality is preserved.<p>Integration tests have their role, and, while they shouldn't be there to validate your building blocks, they should be there to validate that the customer is getting the functionality they're paying for.
The “benefit” of unit tests telling you exactly where the problem lies is not compelling. All tests should always be passing on master, therefore the problem lies in “git diff master.” If you can’t tell where the problem is that’s breaking an integration test, your changeset is too large or there are too many layers of indirection.<p>Approximately all of the bugs I encounter in real life would have been prevented by integration tests but not unit tests. Unfortunately my organization only believes in unit testing, so 95% of our test code mostly just exercises its own mocks.
Networked resources have listeners. Listeners have ears. Ears have wax.<p>A unit test verifies that your Q-tip is made of cotton, and that the cotton is soft and small enough to fit inside an ear to a depth that will fetch wax. Another unit test might confirm that a swabbed substance is actually ear wax, by reporting the qualities of a verified sample of earwax, and maybe also a sample of candle wax as a true negative.<p>An integration test verifies whether you are allowed to swab a specific person's ear, that they still <i>HAVE</i> at least one ear (but check for both, YES. <i>EVERY SINGLE TIME</i>.), that the ear is healthy enough to tolerate a cotton swab, that the person will hold still, and wait for you to finish swabbing (and emergency procedures for what happens if they violently react to suddenly getting stung by a bee in the middle of being swabbed), and that the ear in fact <i>HAS</i> wax to swab, before swabbing.<p>One preliminary unit test tells you that you're not holding a knife. The integration tests do almost everything else. You need to authenticate (ask first, maybe this alone tells you they have ears... <i>OR NOT</i>), connect to the network resource (reach your arm out with Q-tip in hand, and approach the ear), start the transaction (apply pressure to someone else's ear), complete the transaction (extract a sample of ear wax), and check the response for your request (inspect the earwax specimen). One last unit test to make sure you got back wax, and not blood.<p>Sounds disgusting, right? It is. And you can keep a rubber ear in the cabinet, as a stub test target, sure. We all understand the textbook definition of the noun (earwax, integers, money).<p>Real ears still need to be cleaned.
I'd have to register my vote on this not clearing things up. All it did was talk about relatively simple sections of code with well understood boundaries.<p>Instead, I often find people that try to make "units" of code not "Money" or "Transaction" but crap like "WellsFargoTransaction" or "EuroToUSDollarTransaction" complete with separate classes for each of the things that those imply.<p>And the amusing dig against mutable state is... well, unnecessary? It didn't even really do anything other than signal to a certain crowd that he is on their side.<p>Integration tests may be a symptom of poor design. I don't see it, though. Integration tests should exist to test the assumptions that your unit tests rely on. Because, by necessity, your unit tests will be making assumptions about the rest of the systems they interact with. To claim that some "pure" design can get you away from needing to test those assumptions is interesting debate starting rhetoric, but a dangerous goal to put in people's minds.
<i>GIVEN a function that returns one or zero, which is fully unit tested.<p>GIVEN a function that divides one number by another, which is fully unit tested.<p>WHEN I integrate both functions together.<p>THEN I should get back a number.</i><p>This is why you need integration tests. Interaction of fully tested units does not guarantee that they can be integrated together in arbitrary ways without the possibility of bugs being introduced from the integration.<p>The above example integ test would reveal a DivideByZero error. This is a super simplistic integration of two simple functions, now imagine a massive enterprise system and how many bugs could be due to the integration of its parts alone, even if individually each part is one hundred percent correct and bug free.
I can feel him searching for the difference, and think he is so close but I have a slightly different take on it. He says over time you become confident enough in something that you don’t bother mocking it. Others have said they don’t bother mocking primitives. I would say you don’t bother mocking values. Objects with mutation, methods that cause side effects, these things need to be mocked, pure values never do. The system clock has been around a long time, and I trust it, but it has mutation, so I will mock it. If I make a new point class that is just an immutable wrapper for two immutable integers I won’t mock it. Pure functions never need to be mocked. Immutable values never need to be mocked. These don’t need time to become trusted, they are created trustworthy.
Kent is not arguing against integration tests.<p>> I take this as a challenge. I’m happy to write integration tests. I insist on it. But a part of me registers the need for integration tests as a failure or limitation of my design skills. I’ll put that frontier in the back of my mind and a month or a year or a decade later bing I’ll figure out how to raise the level of abstraction, put more of the system in that happy state where I have complete confidence in it, and I’ll have new tools for thinking. Yippee!<p>He is trying to figure out which design abstractions are yet to be invented. It is a challenge not a criticism.
A link (via Internet Archive) in case, like me, you can't access Facebook wherever you are:<p><a href="https://web.archive.org/web/20171206200104/https://www.facebook.com/notes/kent-beck/unit-tests/1726369154062608/" rel="nofollow">https://web.archive.org/web/20171206200104/https://www.faceb...</a>
No non-trivial design is ever perfect, and tests are meant to <i>test</i> where the empirical reality (implementation) diverges from the design (expectation). It isn't clear at all how these two things are related in the way suggested, much less to the extent that they can be considered unified under an (as yet undiscovered) abstraction.<p>Rather, I think the essay accidentally supports something else entirely: developers easily get distracted testing the wrong things and thinking about testing as being easier, more valuable, or less fallible than it is.
Eh...if you have a REST API for instance, If I'm an ardent user of your API I'd sure feel a hell of a lot better if you were testing at the REST layer. I'm sure your mocks for your internal helper classes are really swell, but being a dev It really doesn't make me believe the thing is actually working.<p><a href="https://twitter.com/kentcdodds/status/628658648001048577" rel="nofollow">https://twitter.com/kentcdodds/status/628658648001048577</a>
Integration tests are meant to exercise your external dependencies, ie. Make sure that the external API you depend on still replies that 2+2=4 as it did before. Unit tests prove that your own code is behaving consistently, and integration tests prove that your external dependencies are still behaving the same too ... The point is to assure that nothing unexpected changed, it's not a direct measurement of the quality of the design.
Why is testing so confusing?<p>All automated tests (unit, integration, functional ui) are regression tests. They are not there to find problems when you write code; they are there to surface problems when you change code.<p>The distinction between the types of test help you find at what level the problem lies.<p>One of the best skills of the troubleshooter is to break down interactions into layers, and then troubleshoot each layer.<p>Organizing tests in a similar fashion aids in troubleshooting.
I don't see the need for integration testing as being an indication of poor design, I think it is just what happens to testing as you put together your well-understood abstractions to solve larger and more complex problems.<p>I think we are all agreed that the way to deal with complex problems is to break them down into smaller, independently-solvable problems. An inevitable consequence of this is that there will be at least one level of abstraction that is concerned with the interaction of little-problem solvers, and things can go wrong even if all the components are working as you think they should. Complex systems generally have emergent properties (that's what we make them for), and some of those emergent properties might be bugs.<p>In fact, the more you practice divide-and-conquer, the more your time will be spent on assembling units and dealing with the issues of their interaction, rather than in making them.<p>If some of the assemblages have broad applicability, we make them units - for example, we can use matrices to simplify a broad range of mathematical processes that would otherwise have to be done as a mass of bespoke scalar operations - but the integration testing must be done (at least once) before they can be used as units.<p>A system is never going to be abstract generalities all the way up, however. At some point, towards the top of your abstraction hierarchy, you will be solving a unique problem - using matrices in a control system for a specific vehicle, for example - and that is going to need integration testing.<p>Insofar as integration testing is a consequence of divide-and-conquer, we might say that the inability to create unit tests would be a symptom of poor design, as it would indicate that the system is not composed of self-contained elements having well-defined interfaces.
I liked this bit of perspective from the comments:<p><i>When people start arguing about "unit" vs "integration" tests, I think about Smalltalk, the home of the original xUnit testing framework: In Smalltalk, everything is an object. The constant integer value zero is an instance of a class. And your application may define new methods on that class. So in Smalltalk, you literally can't do anything without using some number of classes, many of which may have methods defined by your application. Also, in assembly language, the only abstractions handed to you are bytes, words, and various CPU status flags. The "units" you're working with are extremely low level.<p>So I conclude that the dividing line between "unit" and "integration" is largely arbitrary, and subject to reinterpretation, depending on the tools and libraries we happen to be using, and possibly how familiar and comfortable we are with them.</i>
I think the problem I have with this is the idea that the design is static. In practice, if you’ve got five layers, their exact behaviour can be subject to subtle change. Integration tests (not necessarily whole system tests) are invaluable for finding where changed assumptions actually break required behaviour. This, in turn, can inform a better design.<p>Never thought I’d be disagreeing with Kent Beck about anything in his bailiwick, but there you go.
Unit tests are great if you need to drill down and verify a very specific piece of logic. Integration tests on the other hand are very useful if you need to test the overall system, especially if testing smaller parts is not possible or does not make sense.<p>For instance, in my company we have developed an internal tool that allows external translators to directly translate resource strings in our database and VCS. Parts of this system are in unit tests (like the parts recognizing what the language of a file is), but the interop with the VCS is not something that makes sense to isolate. Therefore the only tests for that parts are through the integration tests.
Making Transaction mutable usually means its internal state is not 100% controlled by its implementation. The consumer of a Transaction object might be able to manipulate its state in an expected way (e.g. calling mutation methods in an unexpected ordering) and makes it invalid. So the unit test suite you write for Transaction will never cover all cases. You need to test Transaction in the context where it's used as well, e.g. the Account example. That means when you're writing tests for Account, you are actually testing Transaction at the same time. That's probably why it feels more like an integration test than a unit test.
Different people tend to have completely different ideas about what an integration test is.<p>At one extreme, some people think it means that you have your entire system running and then you have some headless bot simulate clicks and evaluate the system's behavior based on the UI...<p>From my point of view, any test that traverses through the logic of more than one kind of object is an integration test. Primitive types don't count.<p>I actually find integration tests much more useful than unit tests. It's unusual for me to come across a unit of code that is so complex on its own that it needs to be tested in isolation. I think that simple units of code is actually a sign of good de
The presumption here is that you are a team of Kent Beck's. Mine isn't.<p>Yes, with a lot of work and time, we can come up with very good abstractions so that all code is easy to understand and follow and unit tests are all we need. But we don't have that time, nor often the skill needed. What we need is a running product, today, for our customers.<p>Integration tests are a crutch. Some of us need crutches. And I don't feel like I should be ashamed to need one sometimes.
Is there really such a hard boundary between integration tests, unit tests and other types of tests? If you do something that crunches only numbers and returns numbers you can probably live purely with simple unit tests. But once you get more complex systems you slowly creep into integration tests territory. I feel like it's more of a continuum.
Disclaimer: I mean no offense.<p>I am yet to see one production-ready real-world code example from such gurus of design of the past. Kent Beck, Bjarne Stroustrup, Scott Meyers.<p>I think that in 2017 you shouldn't be entitled for your opinion about design if you have nothing to back up your cases. Sadly, this also covers me with this throw-away account, but bear with me for a while.<p>When they started their journey the world was different and chains of thought that led them to their current points of view are long obsolete.<p>We are living in a different world with different values. My own experience with integration tests led me to believe that I cannot afford to skip on them. There are far more moving parts nowadays and it's better to know that something introduced breaking changes from tests than from angry customers. Something along long chain of dependencies that you did not even consciously knew you are depending on.<p>Now, some guy who is known only for some books he wrote in times of yore comes in and tells me I am doing it wrong, only on basis of his non-existent experience and unfounded authority? We already know what comes out of unsoliticed advice like that. We wasted decades on OOP modeling, UML and tons of other things that never came off.<p>There are now widespread ideas that he either conceived or popularized, but even these mutated to the point of unrecognizability.<p>How about we will move on?