Where unit testing fails

Whut?<p>Listen kids, testing is a tool for HELPING THE DEVELOPER, not for using to engage in a "more pious than thou" dick-swinging my Cucumber is bigger than yours idiocy.<p>Testing is about giving YOU THE DEVELOPER useful and quick feedback about if you're on the right path, and if you've broken something, and for warning people who come after you if they've broken something. It's not an arcane methodology that somehow has some magical "making your code better" side-effect...<p>The whole concept of "test driven development" is hocus, and I speak this as someone who writes a lot of tests, and who charges a lot of money for fixing test suites. Instead: developer-driven testing. Give your developers useful tools for solving problems and supporting themselves, rather than disappearing in to some testing hell where you're doing it a certain way because you're supposed to.

The exact Sudoku URLs mentioned in the article, just checked/updated:<p>Ravi's article: <a href="http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-solvers.html" rel="nofollow">http://ravimohan.blogspot.com/2007/04/learning-from-sudoku-s...</a><p>Peter Norvig's solver: <a href="http://norvig.com/sudoku.html" rel="nofollow">http://norvig.com/sudoku.html</a><p>Ron Jeffries' attempts:<p><a href="http://xprogramming.com/articles/sudokumusings/" rel="nofollow">http://xprogramming.com/articles/sudokumusings/</a><p><a href="http://xprogramming.com/articles/oksudoku/" rel="nofollow">http://xprogramming.com/articles/oksudoku/</a><p><a href="http://xprogramming.com/articles/sudoku2" rel="nofollow">http://xprogramming.com/articles/sudoku2</a><p><a href="http://xprogramming.com/articles/sudoku4" rel="nofollow">http://xprogramming.com/articles/sudoku4</a><p><a href="http://xprogramming.com/articles/sudoku5" rel="nofollow">http://xprogramming.com/articles/sudoku5</a><p>And, as dessert (Ron is very frank about his failures):<p><a href="http://xprogramming.com/articles/roroncemore/" rel="nofollow">http://xprogramming.com/articles/roroncemore/</a><p><i>"This is surely the most ignominious debacle of a project listed on my site, even though others have also not shipped. (Sudoku did not ship and will not. Shotgun will go forward if the Customer wants to.)"</i>

There are a couple tricks to effective unit testing. Ultimately though, the goal is to write non-brittle tests. That is, tests that won't break due to unrelated changes. Get this wrong and the cost of maintaining your tests will outweigh the benefits.<p>Achieving this, in my experience, comes down to disciplined used of mocks and stubs. I've seen people lean too heavily on them, or not heavily enough. The biggest problem I've seen, particularly common in the Java world due to jMock's datedness, is over specifying expectations on mocks. Stubs which can provide canned responses (whether through a framework or manually rolled) are absolutely and totally the most underused yet useful tool in your testing arsenal.

Maybe I'm in a mad mood or somth (it's Monday, after all) but all this "let's write a parallel world, called TDD, to which the actual production code has to comply and praise and give sacrifices"-mantra has gotten a little over the top.<p>Don't get me wrong, I'm sure that all these guys and chicks that live and die by TDD are smart (probably smarter than me), but what they are building is starting to get more and more out of touch with real-world requirements (I wanted to say somth about how this resembles a schizophrenic world, but for the moment I'll stop short of that).<p>My father was a civil engineer, at a time when they didn't have AutoCAD available and all that fancy computer stuff. As he elegantly put it: "if I mess things up in my work people will die, either in 2 weeks or in 20 years, when the next big earthquake will hit us". Well, I never saw him "testing" building apartment-blocks, or industrial buildings, or roads, he made sure that what was on the sheet of papers where the construction plans were drawn would get built as accurate as possible in real life. And keep in mind that he was managing construction workers, many of them close to illiterate, even former convicts, and not CS-graduates. That's the job of all civil engineers from all over the world. So, if they can do it, why in the name of God do we programmers write tests for mundane stuff like sudoku-solvers? Something is not right.

Ok, TDD didn't help with designing an algorithm for Lychrel number - because you just don't get around implementing the loop that tests if a number is a lychrel number, and it's very simple.<p>But to me that's not the whole purpose of unit tests - the most important part for me is that it tells you very quickly when you have bugs in your implementation. Somehow the author neglects to address this very central point.

First write the high level test. Then start implmenting and when you writte a helper function, write a little test for that function too.<p>Don't just write all the test for every little function in the beginning thats stupid, How do you even know what helper function you need to implment.

Here's my trouble with testing as espoused by proponents of TDD: I bought into the Agile Manifesto. I believe in working code over specifications. How then, am I supposed to write a bunch of tests that will in turn tell me what code to write?<p>It is not a coincidence that one of the reasons TDD'ers love rspec so much is that the "code reads like documentation" - it's even called r <i>spec</i>.<p>Perhaps that is ok for mature projects. But I often hear TDD and agile used together as if they weren't in conflict and I get confused. Then again, I've only ever worked on new projects.<p>Maybe I'm missing something?

I can't help but appreciate the irony of a link titled "Where unit testing fails" leading to a WordPress database connection error.

What are some cases that you would and wouldn't use unit tests? Igal is doing a webinar <a href="https://www2.gotomeeting.com/register/545851563" rel="nofollow">https://www2.gotomeeting.com/register/545851563</a>

I like to think of it this way:<p>When writing code it is imperative to to get great feedback, as often as possible. In a number of cases, TDD helps you do that. In cases where it don't, well, it doesn't, so don't use it.

I haven't had time to read them all yet so I was wondering if someone could give me a summary of why the TDD based sudoku approach failed?