The Start-Up Trap

(tl;dr: it depends)<p>It's a hallmark of "experienced" non-dogmatic product people (UI/UX/Dev) that can use their intuition to know what are the happy paths that they need to test, what interfaces are likely to not change (e.g. a user is probabilistically always going to be a member of an organization, so unit tests around that are likely not waste), and what level of quality to introduce for a given feature relative to the probability the feature is going to exist in perpetuity.<p>You can concretize this by calling it "spike driven development" if you want (that's what we do) but the point isn't that TDD is faster or slower but that high coverage might be inappropriate (TDD isn't binary - degree of test coverage goes from 0-100%) at different phases of the build/learn/refine cycle.<p>For example, if we're building a speculative feature that we think has value but hasn't been proven yet, we want to spend the least amount of effort (code/time/money/pick some) possible to prove that it's viable.<p>Small bugs aren't really that interesting (provided the thing "works") and so we're explicitly willing to live with degraded quality to prove the feature is worth further investment. We're also explicitly not building a "large" feature (so it's very likely to get coded in a day or two) so the surface area for major showstopper bugs is minimized.<p>Often the feature will be thrown away or majorly refactored into something that is completely different.<p>In this case, full-bore 9x% coverage TDD is probably waste as the feature existed for a short period of time, the interface changed dramatically, or otherwise was proven invalid. High test coverage makes <i>major</i> interface refactors really expensive and you really don't need that level of quality for speculative features.<p>After you prove this feature is a "thing" you want to keep around (and you've nailed down the interface), then it's a perfect time to re-write it with a much higher degree of test coverage.

Wait, since when TDD become a best practice?<p>I know this author is advocating it, but has 0 (zero) solid data or evidence to back this assertion. He is just trying to sell another religion. What I have noticed, TDD is being pedled heavy by RoR contracting shops that just care about billable hours and not vested if the startup or company is going to make it on the long run. I see often that it is the usually youngish and guidable engineers that fall in the trap of eating it up. TDD is something that makes younger and less experienced developers feel good (more locs) and a false sense that the code is correct while there is solid data that even 100% unit testing coverage finds only 20% of defects, at best.<p>Having shipped many world class products, and none of the processes that build those products had any resemble of TDD in them. So, to all the pragmatists out there feel free to ignore the advice on the article. I'd yet to hear a super successful startup that used TDD.<p>Having Tested Code (which is a good goal) != TDD. TDD to me is putting the carriage in front of the horse. Anybody that have created large systems from scratch, TDD is a major slowdown on the refining process. Your job is to SHIP a working product. Testing is only a small part of it, and its positives should be considered alongside its negatives and side effects on the shipping timelines.<p>Having worked for over 10 years in the industry I have to meet any great programmer that preferred TDD. I hate to attack the author, but from his bio he seems a bit like a process peddler. I don't see him have worked on any successful startups, or anything that had a good exit. So take his claims with a grain of salt. <a href="http://www.8thlight.com/our-team/robert-martin" rel="nofollow">http://www.8thlight.com/our-team/robert-martin</a><p>He is trying to sell a religion, so buyer beware.

Great, more TDD evangelism.<p>I would love to see an empirical study proving the claims made about TDD in this article and numerous others.<p>It seems to me TDD is a huge waste of time when prototyping a minimum viable product. You want me to spend 2 hours writing tests for a feature that someone's going to tell me to rip out 15 minutes later? No thanks.<p>It's really easy to tell others to use TDD, and even admonish them for not using it. But unless you are the one in their shoes, you will not know the whole of their reality.<p>In a perfect world I'd write tests for everything. In the real world, I write tests for almost nothing. Most of the work I do is on stuff that might be gone tomorrow.

I like to talk about TDD using a finance analogy.<p>One of the first concepts you learn in introductory finance is that, in a mature market (ie the stock market -- basically something with a bunch of people and a bunch of information) you have to be compensated for <i>risk</i>. This is why a riskier stock -- like tech stocks or penny stocks -- can boast incredible amounts of risk but generally will give you higher returns over the aggregate than something like pork commodities or a CD. When you choose to purchase a tech stock -- or any stock at all -- you're saying "okay, I recognize that this is riskier, but I think I'm being fairly compensated for the risk, too."<p>Choosing to eschew TDD is like purchasing stocks. By definition, going through TDD is going to be a safe route, but it's rare that TDD (at least in my and my friends'/peers' experiences) is actually going to make you get from Point A to Point B any faster. TDD isn't, by default, a superior or inferior approach to anything: it's a tradeoff -- do you want risk or do you want return?<p>Sometimes, you want to minimize risk, and that's probably smart. Sometimes, you just want to produce an MVP -- and that's okay, too.

I thought this was going to be an article about overwork, or about charging for what you build as a way of proving your market, or about how employee stock options are often a bad bet. Instead, it's about TDD. What a disappointment.

TDD is like those best-practices books that everyone pretends to read, but hardly anyone ever really reads them (I'm looking at you Code Complete). The Folks that do read these books know the dirty secret that no one else really reads them either, but they feel that endorsing the books somehow gives them credibility.<p>TDD has its place, as most all main-stream methodologies do. But, lets just admit that the people that use this methodology are in the minority. The rest of us are working on smallish projects that are struggling to be worthy of the time-budget that they've been granted, and we're more worried about shipping than caring about how not having TDD in place will slow us down in phase III. Assuming phase III ever happens. We're using all the best-practices we can, but TDD doesn't rise above the bar most of the time.

My feeling is that the point of this article isn't strictly to promote TDD or any single engineering tool that startups tend to eschew. Instead, the author seeks to make the broader point that cutting corners in engineering is not a good idea <i>even in a startup</i>. While it can make startup engineers feel like they are moving quickly relative to the competition, the engineering debt incurred is non-negligible.<p>I strongly agree with this point, and don't buy the blanket rationale that speed trumps everything in a startup. Yes, speed is quite important when you are bringing something new to market; it lets you get your foot in the door and rapidly find the right fit for your product. But if you cut corners then your product will suffer, and you'll end up treating your customers poorly by it.<p>There's a fine balance to all these things, of course. We all have to find the right balance between keeping engineering standards high and heavy-handed structures.

Man, I've been back and forth on tests from one extreme to the other. I have to say that I do agree with this but with one caveat - Test Driven Development is a bit miserable.<p>Test "Driven" Development is a real invitation to write too many tests. A small, good set of tests gives you freedom to work fast, to refactor and to have multiple developers pushing simultaneously. Not having them really isn't sustainable past a certain point but equally to many tests will drag you to the ground.<p>Tests are overhead, they cost time to write, maintain and run and TDD tends to drag you into the deep end. Avoiding them altogether ends up being a false economy (if only in that it makes you too nervous to push often).<p>To those who fear the slippery slope, a nice self-annealing approach to get into testing is to only write tests for something that has failed. That way you waste no time writing tests for things that are actually pretty robust but equally you avoid addressing (and fearing) the same issue twice.

Over the years, I've grown less and less likely to write lots of tests, after being a very large test zealot during the peak of the TDD wave. There are a few reasons.<p>First, yes, TDD slows you down. The reason this matters is because a lot of our time as developers is spent exploring ideas, and it's pretty well understood that the faster you can get feedback on your ideas the easier it is to creatively develop them. In fact, the final result of your creative process can be completely <i>different</i> depending on the feedback cycle you have. From the micro to the macro perspective, religious TDD introduces an constant factor slowdown for small projects that yes, ends up being a net win in the long run if you live with the code for long, but is a net loss in the short run and also can <i>prevent</i> you from finding a solution to a problem since your creativity is stifled by the slow speed of idea development.<p>Second, when building web applications, building tests turns out to be fairly overrated. First, people will tolerate most bugs. (Write good tests around the parts they won't.) Second, if you have a lot of traffic, bugs will be surfaced nearly instantly in logs and very quickly via support tickets/forum posts. (What if there are bugs that don't make it into the logs and don't affect people? If a tree falls in a forest...) Much more important than mean time between failures is mean time to recovery. I'd rather have ten bugs that I fix within 5 minutes of them affecting someone than one bug that festers for a month. Not only because this is healthier for the code base in terms of building robustness, but also because human behavior is such that many fast bug fixes make everyone feel good but few lingering bugs make everyone miserable. People want to feel like you care, and are much more likely to feel cared for when you fix their problems quickly, and are not often interested in just how few bugs you ship if the one that you do has been affecting them for a month.<p>This isn't theoretical nonsense, it's a very real phenomenon where you use tests and manual testing to get up to a basic working prototype and then just throw it over the fence to flush out the bugs. It's the only way to do it anyway. (This only really works if you have traffic and can deploy changes quickly. To paraphrase a colleague, production traffic is the blood of continuous deployment.) Obsession over deploying bug-free software (an oxymoron anyway) is usually coming from people who haven't gotten over the fact that we don't need to ship software once a month or once a year but can do it every 10 minutes in certain domains. Instead of focusing on not shipping bugs, focus on shipping faster.

Imagine test driven development for visual art. How would one write a test for visual art, beyond viewing and comprehending the partially completed work? If one could write a test in advance, one would have made a significant advance: distilled to a verbal form a description of what good art is.<p>Instead, many artists -- or software authors, or chefs, find a way to repeatedly sample and appraise their work as it develops over time.<p>I think this is far more important than TDD, because the truly important problems of software engineering are not in making software that can achieve simple correctness, but in making something that people want.

"Of course one of the disciplines I'm talking about is TDD. Anybody who thinks they can go faster by not writing tests is smoking some pretty serious shit."<p>Why do so many TDD people think that people who don't use TDD don't write tests?

TDD is often a waste of time. I worked at some place that had taken the TDD philosophy to the extreme: every getter and setter had a test.<p>On the opposite side of the spectrum, I worked on software that processed millions of dollars in daily transactions and there were maybe 5 "tests" in the whole system. This was about 500,000 lines of C and C++ code in the early 2000's.<p>My personal philosophy is to write unit tests where I think it's important, not test everything.

&#62;<i>Of course one of the disciplines I'm talking about is TDD. Anybody who thinks they can go faster by not writing tests is smoking some pretty serious shit.</i><p>That's some serious strawman. In between 0 tests and TDD, there's a lot of room to maneuver.

TDD is but one method of writing code; one that I've never found particularly useful. There's plenty of other things you can do to perform your job well.

Skipping testing amounts to risk. The winners in a brutal race take every risk to win.<p>So yes, its foolish to skip testing. And yes, the race goes to the foolish.

"as if any bug is acceptable!"<p>This is the point at which I stopped reading.<p>Yes, some bugs are acceptable. The point of writing software is not to create bug-free code; it's to create value. The marginal returns on eliminating the last bug are much lower than implementing new functionality.<p>As for the gist of the post, from my experience, TDD is OK; but it is unit tests that are essential. They are far more important in dynamic languages, because you otherwise have very little feedback when you make mistakes as simple as a typo. They're also important in static languages, but fairly large programs can successfully be written with without anything near the volume of testing needed with a dynamic language.

It is easy to hate TDD.<p>In a Rails project, the Rails boot-up time makes TDD a painful experience if you are not using tools like Zeus or Spork. Even in the presence of such tools, you need a powerful machine to not hate the slowness of the whole thing and worse still, break your flow.<p>My recommendation for someone who hates TDD for the wrong reason, aka: breaking the flow, would be to get a fast machine, fix the code to make TDD as painless as possible and to use the right tools.<p>But once you start using TDD as a tool to organize your thoughts and model your domain, you might end up becoming too dependant on it and find it hard to work any other way. This is anecdotal experience.<p>Also, learning how to wield TDD properly takes a lot of time, error and practice. Good things don't come easy. There are obviously places where TDD isn't a good fit - a spike, requirement that is known in advance to change soon, and exploratory programming are all candidates. However, good practice dictates that you refactor your code once a spike calcifies into production code. At this point, TDD becomes just unit-testing.<p>Most of the arguments against TDD in this thread seems to be against unit testing in general. But we know unit tests are important. Doing it before the fact increases the value of the unit tests manifold and also ensures that you do have coverage (though that is not at a primary objective).

Whilst not exclusive to TDD, I think DHH said it best:<p>"The road to programming hell is paved with “best practices” applied too early."

Largely agree with everything here but in my opinion many times there can be a very strong current that forces you to do it the fast, stupid way. In my opinion if you dont have any of these problems, you work in a unique place...<p>- Agile - Agile IMHO largely screws you in making sound technical decisions. Its not necessarily because agile is flawed - its usually because business/management uses agile as an excuse to randomly take a hard left turn every other week making it much harder to make long term architectural choices that are beneficial.<p>- No agreed upon standards or unification amongst software professionals - The accountant in this situation has a set of standards and expectations that allow/force him or her to do things in a moderately set way. This allows for the accountant to usually fend off management from pressuring things to be done in a shoddy or just get it done way. On the other hand its much more difficult for software professionals to say "we as a group do not condone writing shitty software" (because ironically a large number of us do ... "move fast, break things" has done more good than bad from my perspective)<p>- Ageism - Also known as experience doesn't really matter. Some will say thats because technology changes so much - but it actually doesn't. Just because you've been grappling with software problems and design patterns in Java doesn't mean that when you switch over to Python that really anything changes. Same shit, same problems, different words - but in all honesty we seem to be pretty bad at building on the experience of our elders because they are over the hill at 35... what could they possibly teach us.<p>For me it at times has been frustrating because coming out of school I really enjoyed the fun of designing systems and code that are sustainable, performant, etc etc but there seemingly is typically more reward for just throwing quality out the window in startups. Just my personal experience.

In my mind, TDD is a defense against complexity. Complexity is always the enemy. If you can simplify your code through structure or clean design you can minimize or remove testing.<p>The moment you can't hold the whole thing in your head with ease is the moment you should have done TDD a while ago.

Although I've been a hardcore TDD advocate from time to time, I find myself writing fewer and fewer tests in the early stages these days.<p>In the very earliest days of a startup, you can easily hold the entirety of the problem space in your head. Refactoring is simple, tests or not. I can destroy and rewrite a feature from scratch in hours or days. A non-trivial number of the features I'm writing will not exist <i>in any form</i> a week or two later. This is not a problem with process or planning; this is the nature of a startup.<p>I think that TDD is immensely valuable as a team and product grows, but claiming that your startup will fail because you aren't applying engineering "best practices" from day 1 is counter to most successful startups I've worked with.

I disagree with this almost entirely.<p>Startups _are_ different:<p>1. What you are doing starts at 0 value. 2. You can (usually) break it and it's ok. 3. You need to change stuff a LOT to figure out something that someone wants to use, rewriting tests makes that slower.<p>Every test case you write has a cost: It verifies that something of unknown value works. What if the value of that code is 0? Well, you just doubled down on something useless.<p>There are no absolutes in software development, but how we did it was have uncomfortably too few test cases when something is new, change the product for a while until someone actually wants to use it, then add a bunch of tests until we know it works from there on out.<p>And I hesitate to add: Also we work in Java so a bunch of testing comes for free, so there's that.

The way I do my TDD is by writing the unit tests to specifically test the functionality of the code.<p>For example, if I'm writing a module that does validation for a specific type (for example, in JS) I tend to test the functionality while writing the tests. I am unable to speak amongst others, but you have to test your code eventually, and it's either going to be in the classes that are using the code, the REPL, or it's going to be in your unit test/integration framework. I usually pick the test framework first because it serves 2 benefits:<p>1) I can prove my code works. (I must prove this to my self, or the people using the code at some point.)<p>2) I can reliably change the code later.<p>Complexity is also mentioned in a couple of comments here. TDD helps simplify complexity by writing code that is testable. It also makes you think about your API as you're writing the different possibilities the module/code can be used in.<p>Further, I think there are multiple issues with tests, and depending on the type of test your doing, different problems can arrise. You have your unit tests, and then you have your integration tests.<p>Unit tests are relatively easy to write (if your code is split into individual chunks, that do a specific task.) and they should always be written regardless of the time you have.<p>I think the major issues arrise in terms of time are the integration tests. When you're testing the functionality between complicated modules that require databases, i/o writes, network communication, it is sometimes hard to write the tests, and it may not even be worth it.<p>The majority of people who disregard TDD, usually disregard a specific sub-problem of TDD. I think TDD has its benefits depending on what type of tests you're doing, and how easily the problems can be solved with those types of tests. You can always pick to write unit tests, and disregard integration tests, etc.<p>I think there are a lot more issues to expand upon, like the language you're using, the platform you're using and how often your code is required to change, but TDD has large benefits, and code rot is very real and TDD can help mitigate it.

I'm not sure if the author has any idea what is it like to bootstrap a web product and watch the money go down the drain day after day, with zero income to compensate, and with the certainty that you won't make it for another 6 months if the product is not out and making revenue. I'll deal with the technical debt later, thanks. I'm pretty sure Facebook and Google didn't do anything remotely close to TDD when they first shipped, and yet they survived.

To those of you who asked for "links" to studies that prove that TDD works; google around, you'll find plenty. Some are positive, some are negative -- what else is new. Now, please show me the studies that show that _your_ discipline works.<p>To those of you who consider TDD a religion; you are being silly. TDD is a discipline, like double-entry bookkeeping, or sterile procedure. Like those disciplines it has aspects that are dogmatic; but that aim at a purpose. That purpose is to help you write software fast and well.<p>To those of you who think I'm a process peddler. You're right; I am. I make a very good living teaching people how to do TDD. I have been doing so for over ten years now. I hope to continue for some years to come. My goal is to help raise our industry to a higher level of professional behavior; because at the moment our behavior is pretty poor.<p>To those of you who wonder whether I've ever worked at a real start-up. I've worked at several over the years. My first was in 1976; and it was very successful. Another was in 1989, and it didn't do so well. I've recently founded cleancoders.com a startup in video training for software developers. The website is done entirely with TDD. And this doesn't count the rather large number of startups I have consulted for in the last 10 years. So I've got a _lot_ of experience with startups.<p>Folks, I am 60 years old. I got my first job as a programmer when I was 18. I wrote my first program when I was 12 (on a plastic 3-bit computer). I started using TDD in 1999, after I'd been programming for thirty years. I continue to write code using TDD today. I've seen the startup trap. I've lived the startup trap. I strongly advise you to avoid that trap.

I want to vomit when I see titles like Master Craftsman.

So the key to meeting your short term goals is... blow them off and focus on the long term goals. Doesn't make sense to me.

Hacking up a quick demo the non-TDD way definitely has its upsides but if you don't force yourself to review and rewrite your early code you would have been better off doing the right thing from the start.<p>Just preaching TDD is not helpful, we need a coding process that allows clean separation of early and production code.

Know the rules before you break the rules. The accountant analogy is dead on - that and the best quote of the day:<p><pre><code> You'd fire his ass! You'd fire it so fast that the rest of his worthless carcass would be left outside the door wondering where his ass went! </code></pre> Brilliant.

Statistically speaking, a web startup can safely not invest in TDD. For precisely same reason that a daily Russian Roulette player can safely not invest in health insurance.<p>But for the minority who miraculously dodge the bullet, the extra investment will eventually become a lifesaver.

I find that making sure the foundation of an application is solid (I'm an API first kind of person) let's you get away with a lot less testing in the upper layers. This lets you move faster and experiment far more quickly than if you were built on quicksand.

I see this as an argument for testing, but not necessarily TDD. Why have the two become so intertwined?

"Oh, I know you are a warrior-god."<p>Well, at least someone noticed!

A useful reminder

This "trap" is Deadline Culture and it doesn't pertain only to code quality, but to other matters like product direction and personnel decisions, which are made quickly and often badly. It's sloppy and destructive. Companies with Deadline Culture often make it easy for a total asshole to gain power just by citing existential risks that either do not exist or are of extremely minor importance. Deadline Culture companies are <i>obsessed</i> with competitors, even though it's their own internal corrosion that does them in.<p>Sometimes, having deadlines is unavoidable. They might be set externally and exist for regulatory reasons. Deadline Culture is when the company starts encouraging the formation of unreasonable deadlines that triumph over professional integrity and intellectual honesty.<p>VC-istan seems to encourage it with the whole "if we get bored with you, we'll fund your competitors" arrangement.<p>Deadline Culture is, however, great for the true fascist. Fascists love (real or imaginary) existential threats, especially vague ones they can continually adapt to their needs, but that come with time pressure.