The elephant in the room here is the bugs that crop up in the interfaces between units. Depending on your problem domain, it may be easier or harder to create interfaces between all these TDD'd, isolated modules. There's certainly value in a spec as isolated component documentation, but I'd argue there's more value in specs for regressions, particularly when refactoring. This is especially the case in dynamic languages like Ruby which require tests to avoid the most basic runtime errors.<p>The answer to this conundrum is often: that's the job of the integration tests. Okay, but integration tests are slow, so you'll never test all the permutations of the interactions of the components.<p>DHH said during the keynote, "it's easy to make your tests fast when they don't test anything". Of course that's hyperbole, but there's a kernel of truth to be investigated there. When working with something as complex as an ORM like ActiveRecord, isolating the business logic and using the ORM strictly for persistence may allow for fast tests, but you <i>still</i> run the risk of bugs creeping in on the interface because of some assumption about or change in the way ActiveRecord works between versions or whatever.<p>That's why, as ugly and slow as Rails unit tests are, they are a simplifying compromise that strikes a balance between the theoretical ideal of a unit test and an integration test. ActiveRecord itself is this way too, in that often times the business logic just isn't complex enough to warrant the complexity of separating persistence from domain logic. As much as DHH may be talking out his ass without really having ever grokked TDD, I don't think his complaints are completely without merit.
Gary,<p>I've been hoping that you would write an intelligent response to DHH's latest articles and talk ever since I saw them, thank you for doing it.<p>I suspect that his claims of TDD isolation damaging the integrity of designs are mostly red-herrings too. I know they are for me personally -- in fact, the opposite is true. I'd love to see you address this issue as well. In particular, your thoughts as they relate the technique of moving logic completely out of controllers (as you do in raptor). This technique seems to infuriate DHH in the context of rails, while to me it seems a far superior design.
All of these arguments seem like they are overly prescriptive. Individuals should use what works best for their needs. TDD may work wonders for me today on one project, and then it could be horrible tomorrow on the next project. My co-worker might have the inverse.<p>Stop arguing over how other people create software; Ship Code instead.
> That file is going to be a plain old Ruby class: not a Rails model, controller, etc. Isolating those would lead to pain, as anyone who's tried to do it knows. I keep models very simple and allow them to integrate with the database; I keep controllers very simple and generally don't unit test them at all, although I do integration test them.<p>This is what I was thinking of when I read DHH's post. Mock objects and indirection and all that certainly add complexity, but if your "business logic" is all in pure functions, or at least self-contained source files that don't do IO (and don't use "frameworks"), you don't <i>need</i> mocks to test it.<p>Better to keep your glue code simple and do integration tests, while unit testing your actual logic, than to shoehorn extra indirection for mocking into complicated glue code.<p>I gather that DHH complained because he had been doing the latter, and found that it sucked.
Is it me, or there are other people who feel stomach ache when reading this sort of stuff? Why don't just learn algorithms and other computer science "direct" approach by analysing problem and solving it in most officiant way? 100 lines of tests for 50 lines of code for a _catalog_? Really? I'm sorry just reading this brings bad feelings, like those which traders describe when they feel something's wrong with the market...
> "Second, that sentence is false. Isolation from the database, or anything else, is generally done with mocks, but mocks didn't even exist when TDD was rediscovered by Kent Beck in 1994-1995."<p>See, that's not how I remember it. Stubs implementing the Interfaces you wrote (because Composition over Inheritance) was the common solution. AR's idea of a Unit Test has always been <i>wrong</i> to my mind. They aren't Unit tests. They're <i>Integration</i> tests.<p>This only really became an issue with the rise of Rails and the fact that you couldn't really write Unit tests for an ActiveRecord.<p>At least that's how I remember it.
I think there is a good point here, in that a very tight TDD loop requires fast tests. Nothing's untrue about that, and if you've worked with a set of speedy tests I'm sure you know it can be quite efficient, and indeed it can be quite a different way of working.<p>Ultimately I don't really think that this is particularly in conflict with what DHH was talking about. A tight TDD loop might work for some developers, but if you get to the stage where you are making substantial architectural changes to enable this, then it looks like a problem. And I've seen a lot of this first hand.<p>FWIW, I take your point that things like Spring aren't ideal solutions. But with a bit of care they can help offer the best of both worlds; I've got Guard and Spring working together on the Rails project I currently have open. Looking at an equivalent model (100 LOC, 200 test LOC), it runs <i>without database isolation</i> in less than 400ms, which is less time than it takes me to switch to my terminal. It's really great, and I don't have to worry about isolation.<p>TLDR; there are compromise solutions possible. Maybe not suited for everyone, but in some cases better than going too far with "design for testability."
The critical flaw in this post is that Gary is not refuting what DHH said.<p>Gary makes this claim:<p>> You finally get to see what's really going on. David's tests run in a few minutes, and he's fine with that.<p>> I'm not fine with that. A lot of other people are not fine with that.<p>But what DHH actually said is this:<p>> You might think, well, that's pretty fast for a whole suite, but I still wouldn't want to wait 80 seconds every time I make a single change to my model, and want to test that. Of course not! Why on earth would you run your entire test harness for every single line change in a particular model? If you have so little confidence in the locality of your changes, the tests are indeed telling you that the system has overly high coupling.<p>and this:<p>> These days I can run the entire test suite for our Person model — 52 cases, 111 assertions — in just under 4 seconds from start to finish. Plenty fast enough for a great feedback cycle!<p>Using a workflow like Gary's, there's an argument to be made that 4 seconds is not acceptable, and this is why we want single files that can run in a few milliseconds.<p>However, that's not the only possible way of running tests, and the difference between 4 seconds and 300ms for the feedback you're actually interested is massively different than 300ms vs "a few minutes".<p>For a post that calls DHH out on a strawman, this is in itself a great example of one.
On the one hand this article does at least provide citations for some of the timelines involved. On the other hand it makes me throw my hands up when I see some of the examples provided. 100 lines of code to test 50 lines of code, which implement a catalog is pretty par for the course it seems in TDD advocacy. This could honestly be me venting my current frustrations, but I get tired of this type of advocacy, also often seen in the functional world, of proving how great something is by using the most trivial possible example, in a domain squarely in your technique's wheelhouse. 100 lines of code runs in 0.24s? What is this, I don't even. Your functional algorithm is incredibly elegant at implementing the fibonacci sequence? Awesome.<p>But are these things honestly the bulk of what people do? Is testing that you can put objects in a catalog, access them, and remove them most of what people do and need assurances on? Am I the only person who has spent most of their career working on software with codebases measured in the millions of lines, with significant user interfaces as well as significant technical domain knowledge embedded in them? I know that TDD says if our objects have many collaborators we are "doing it wrong" but honestly how far do you break down something without it blowing up into a million classes and a ton of code? How do you get around the fact that a button press can set off a numerical simulation (actually these are super testable and I support that), several trips to the database, multiple changes to the user interface, all in the face of dozens of constraints at all levels of the program to ultimately end up with the graphical representation of chemical pressure that the user wants? Especially when every moving piece in that calculation is supposed to be tested apparently in independent units?<p>I would estimate that the bulk of code that I have experienced in my career relied on multiple other objects to be effective. Do I just mock these out and pay the price of keeping those mocks in sync with the classes they are imitating? Do I then have my test be tightly coupled to the internals of the function being tested, making sure this mock is called in this way, this many times? Do I rewrite things such that every function takes its collaborators as arguments and returns the results I am looking for? What happens when each of those collaborators is itself a giant series of other stateful objects or at least provides access to a highly complex piece of state which is a pain in the ass to setup?<p>I ask these questions seriously, because on the one hand TDD advocates keep telling me I am not professional if I don't follow their practices, but on the other the details of how to do so in the face of real-world constraints (not 50 line data-containers) are always somehow left out of the discussion. I want to believe, I do, it just gets hard to sometimes.
I have great respect for Gary Bernhardt. I think he is missing DHH point here, which as I understand it, that:<p>tests can be an end in itself<p>I think this is reasonable argument on DHH part that we want to spend more time writing actual code, instead of going through movements of TDD. That doesn't mean TDD is not valuable.
> Classical TDD does not involve mocking or other forms of synthetic isolation by definition. We even use the term "classical TDD" to mean "TDD without isolation".<p>I wish there was a source on this paragraph. According to this talk that often refers to Kent Beck's book ( the talk: <a href="http://vimeo.com/68375232" rel="nofollow">http://vimeo.com/68375232</a> / a good summary of the talk, in text: <a href="https://groups.google.com/forum/#!topic/growing-object-oriented-software/Hxp8cVfE4gI" rel="nofollow">https://groups.google.com/forum/#!topic/growing-object-orien...</a> ), you are supposed to isolate from the database.
Of course creating mocks or circumventing the database may create more bugs (or hide more) than just wait the couple of minutes for <i>the real thing</i><p>Adding another failure point is exactly what the name says: another possibility of error or bug<p>Sure, we can talk about the "one true way" of doing tests, how to make hundreds of tests run in a short time, etc (the fact that TDD insists in creating one test for each tiny thing goes against it, btw) but yeah, I prefer spending more time solving the problem, and not testing around it.
This comports quite well with my personal experience. In late 1999 I happened to pick up XP Explained and set about to try this testing thing. At the time, there wasn't any "Unit Tests == Fast Tests" thing. What we did on our project was have two different types of tests: "Fast" tests and "SlowAndExpensive" tests, that we ran separately. But there wasn't much some fundamental distinction between them. That came a lot later.
Fast tests are awesome, but hard to achieve - at least for me. TDD advocates: prove DHH wrong with easy to adopt frameworks and working software, not blog posts or books.
I'd believe the same endorsements from someone who makes their living primarily off working code, rather than selling edu. material promoting test-heavy coding.<p>That said, rapid "all's still well" feedback is awesome when you can get it.
> "I want my feedback to be so fast that I can't think before it shows up. If I can think, then I'll sometimes lose attention, and I don't want to lose attention."<p>Don't you want to think while programming? I feel like that's practically all I do -- I spend most of my time thinking about a problem and very little actually writing code.
This banter is really getting old and hogging a lot of space.<p>If you don't like TDD, don't do it. If your employer or organization forces you to do it, leave.<p>If you do like it and think it matters, do it.<p>It's really that simple.