Where TDD Fails

I've seen a lot of silver bullet fads come &#38; go. I'll be glad to see the TDD fad go. Testing sure is a useful tool in the toolbox, but that's it.<p>It's funny this article quotes the asymptotic sort behavior, because that's almost identical to a real-world example I have used to debunk over-reliance on unit tests (AKA "false sense of security"). Back in the days when we still wrote our own sort functions, we were working on various QuickSort implementations. A popular optimization is to end with an insertion sort after a certain point. So your QuickSort can be completely broken, and the insertion sort at the end will still clean up the mess. Unit tests will typically just validate that the list is sorted correctly and completely miss the fact that the QuickSort itself is broken internally. Nobody has seen fit to comment on this example, so far.<p>Another fad I'd like to see gone is "no comments". My mind still boggles from a recent code review where I had to defend using comments. Sure, we've all seen worthless comments, and misleading comments, but let's not throw the baby out with the bathwater and ban comments altogether! Self-documenting code is an ideal that should be aimed for whenever possible. But there's still a place for block comments in certain areas to explain key design and implementation decisions, tell a future maintainer what they might want to consider before changing or enhancing something, etc. It's not always possible to capture all that in code.

TDD always seemed like a impossible thing to a developer like me.<p>The problem is something like this, the moment I face a programming task- If the task is a little complex I do some work on the notebook else I directly fire up the editor and write some code. Now unless I get some code/prototype up and running I can't proceed further. That is how my mind works.<p>I don't know how anyone can write tests <i>about the idea/task</i> for which they haven't written a line of code. I've found it impossible. For me, its always:<p><pre><code> step 0. Basic version of Idea. step 1. Write code. step 2. Making it nice(optimize/document/clean up). step 3. Make it run faster. step 4. extend idea and repeat steps 1 to 4. </code></pre> I'm jealous of all those people who can have a 'step -1' where they can write test cases 'for the code' they would end up with in 'step 4', even while they have a very blurry version of the idea.<p>To me this has always looked like premature optimization. Like putting the cart before the horse.

I will start doing TDD all the time when:<p>1. It is faster than developing without it.<p>2. It doesn't result in a ton of brittle tests that can't survive an upgrade or massive change in the API that is already enough trouble to manage on the implementation-side- even though there may be no functional changes!<p>Some other thoughts:<p>Unit tests that test trivial methods are evil because the LOC count goes up (maintenance anyone?) and the number of entry points and NPE possibilities or checks goes up (bugs anyone?) -&#62; TDD promotes testing trivial methods -&#62; TDD promotes evil<p>TDD increases the chance that more people will mock, and mocking can lead to brittle tests -&#62; TDD increases the chance many of these brittle tests will be written -&#62; Lots of brittle tests means that you throw them away later or rewrite the whole app (with more tests!)<p>TDD promotes 100% test coverage of the code you write -&#62; Very, very few successful companies have 100% test coverage -&#62; Code with 100% test coverage has brittle tests (period) -&#62; TDD promotes things that are not best-of-breed practices in a quest for the false god of 100% test coverage.<p>I was a firm believer in what Kent Beck, Ron Jeffries, et al were pushing in the early part of the last decade. But since then I think most of the rest of the world already knows that TDD practiced religiously will lead to huge amounts of code that slow... down... development... and... make... it... easier... to... buffer... estimates... because when absolutely required you can stop TDD and just hack a spike into production. And... when the application needs to be rewritten because it is too crazy complicated to change all of those tests- you just rewrite it, and we all love greenfield development!

My company's website code is probably about half Haskell (using Yesod, for all the backend code) and half Javascript (for the single-page app frontend). Despite the nuances described by the author, I really think there is a definite substitution the robust type system vs. unit testing. On the Haskell side, I don't have unit tests, and don't have many tests at all. The number of bugs I discover in the Haskell code is very small in spite of this. I know people have said it before, but when something passes the type checker, most of the time it literally just works.<p>The contrast with the Javascript side couldn't be more stark. The Javascript side is much more challenging to get to a bug-free state, and some of the remaining bugs defy replication, etc. I think to get a system that has a fraction of the reliability afforded by the type system, you really need extensive unit tests.<p>I'm not contradicting the blog author, but I think it's important to keep in mind that in broad strokes, those two aspects really do tradeoff.

Author here. Sadly our WordPress didn't have caching enabled so here's a Gist of the content:<p><a href="https://gist.github.com/pufuwozu/5095510" rel="nofollow">https://gist.github.com/pufuwozu/5095510</a>

This isn't really a new thing though, is it. There is the (in)famous example of Ron Jeffries trying to solve Sudoku through TDD (and failing miserably). <a href="http://xprogramming.com/articles/oksudoku/" rel="nofollow">http://xprogramming.com/articles/oksudoku/</a><p>Compare and contrast his impotent flailing with Peter Norvig's masterful approach. <a href="http://norvig.com/sudoku.html" rel="nofollow">http://norvig.com/sudoku.html</a><p>Of course for the lulz you had people like Spolsky and Martin all use it to set out their view of software engineering with essays like Duct Tape Programming and Echoes from the Stone Age.<p>Anyway ... The conclusion from it all was that TDD works best as a design tool for interface discovery. That you get a regression suite, of sorts, is a side effect.<p>It doesn't work as a specific optimization tool or algorithm discovery tool. Which isn't really surprising as these are two of the few remaining 'non-mechanical' aspects of programming.

Could someone link me to a consumer web app with standard features (generating HTML, talking to a database, making requests to 3rd party services) that uses one of these typed functional languages (Haskell, Clojure, whatever)?<p>I'm genuinely curious what this code would look like - every example I've ever seen goes something like "Types are the best thing ever! Here's an example: sorting a list! making a stack/queue! See it's GUARANTEED to work! Math and stuff!". The closest I've seen is the Gilded Rose kata, but that doesn't seem to handle UI/database.<p>And, please, don't say "just use an I/O monad!" - that isn't helpful to me.

I always liked Fogus's post "Not Enough" on this topic:<p><a href="http://blog.fogus.me/2012/06/20/not-enough/" rel="nofollow">http://blog.fogus.me/2012/06/20/not-enough/</a>

Looks like precog.com doesn't have much interest in load testing either.

Often times, the goal of TDD is often confused with spec testing. I think 37signals is right when they say TDD development is like TSA's security check process...

I've played around with roy. Good to see it getting some press!<p>More on topic, what about Contract DD as an alternative to Test DD and Type DD? e.g. <a href="https://github.com/disnet/contracts.coffee" rel="nofollow">https://github.com/disnet/contracts.coffee</a>

&#62; Sadly, we can't write a test or a type to satisfy our specification. We need to actually perform some (asymptotic) analysis to derive our code, instead of relying on tests!<p>Actually, this test can be written without too much trouble. Take some different sized data sets and time the sort operations. Do some analysis of your data to see that the data points fit acceptably close to an asymptotic curve.

&#62;Neither TDD nor types are the single answer to writing code that does what we want.<p>This is an oddly persistent strawman. Is there any seasoned developer who believes there is a single answer to this problem?

"Where our website fails at serving pages" ... can we stop these 'authoritative' posts that can't even serve a static webpage? Ugh.

TFA begins by explaining that it's going to explain why why do we need static types although we have TDD.<p>However these are two orthogonal concepts and the first paragraph of TFA makes no sense.<p>Also TFA seems to imply that using static types + TDD means we can have programs provably bug-free.<p>Not so fast there: <i>any</i> bug in production in a TDD+static types is a bug that "escaped" the type system and that didn't get caught by TDD.<p>And there are <i>many</i> such bugs in production for TDD + static types projects.<p>I'd say TFA is very poorly trying to make its point (not that the author doesn't have one: but the 'talk' is not nearly anywhere near close the 'code' in TFA).