For startup founders looking for market fit, or up to Series A/bootstrapped and growing, how often do you (or your team) write automated testing?<p>How do you go about it? Full coverage? End to end only? Integration only? How much dev time does it end up taking you?
The answer is so dependant on what you're doing, and your industry.<p>The answers here are all correct for the use case of the person answering it. We don't know your use case so we can't give you a good answer.<p>For example a seed funded health tech company needs 100% test coverage to be compliant with regulations.<p>A bootstrapped online collaboration tool with no users doesn't need any tests.<p>A seed funded online collaboration tool needs some tests so the developers can work together without breaking each others code.
Not a founder, but a serial side-project'er. Initial prototype typically has no tests, at least for the bits that are wired. Specific bits of complexity will have unit tests. At some stage might add integration tests using something like docker compose.<p>Use TDD for bits that have very limited scope and function. For instance, an order book class or similar. Otherwise code first, test bugs if needed. For smaller projects (that aren't critical espc.) overtesting can kill your momentum and motivation, which is an often overlooked resource.
While working on my startup as a solo-founder I used to follow a TDD approach supported mostly by Integration tests. I mostly focused on the user happy paths with limited coverage outside the Integration tests.<p>The goal is to have the major functionality of the platform tested automatically so we can be confident that new features don’t introduce new bugs.
Coverage is the best example of Goodhart's Law, never focus on it.<p>Integration tests are typically best bang for buck: <a href="https://kentcdodds.com/blog/write-tests" rel="nofollow">https://kentcdodds.com/blog/write-tests</a><p>Once manual testing starts causing noticeable pain, you can start automating some of it. This may very well be beyond $10-100M in valuation.<p>A lot of people say unit tests give them confidence, but that's a delusion. Losing confidence in your code means you have outgrown your architecture. However, you can <i>temporarily</i> address it with more tests, because addressing architecture issues takes some time. The secret is to write <i>as little tests as possible</i> to bring confidence back to acceptable level.<p>Don't spend more than 10% time on tests.<p>Don't let junior engineers write tests.
Almost Never test-first. It impedes my concentration and flow and inspiration. A new idea to me is like a fresh slab of marble you have to sculpt. Over time, you play with the code and api ergonomics datamodel and abstractions and UI until it “feels” right and takes on a more defined form. Working on tests at that point would be like continuously stopping to check the foundation of the marble.<p>But for self contained code that is guaranteed to have edge cases (working with dates, date math, unstructured user input like SMS responses, almost always yes, but after some of the code is written and is getting cumbersome to test different cases manually.
Not a founder, but I work at a very large fortune 500 firm.
The vast majority of our custom apps have little to no tests.
However our INFRASTRUCTURE is loaded with tests, heartbeat monitors, etc.
You name it, we have it.
On front end, almost none. Apps are very test-hostile, so much so that just setting up the framework can increase complexity by ^3.<p>Well, unit tests are fine and often we do it for things like testing that email parsers and unit conversion works. As long as it's easier than manual testing.<p>It's also easier to do tests than debug sometimes.<p>Back end is a different story - the ROI on tests increases drastically because tests are easier to write and failure is more expensive.
Code is a liability. Founder or not I write as few tests as I feel necessary for things I don't trust in my code.<p>I generally write as little code as possible overall, and try to sell it as fast as possible.<p>I hate source code, it never ever is the result for me. So if i do not have to write code than I get to not write tests.<p>My goal is always to write as little code as absolutely possible. That's it
The best practice is to write the test by knowing all the scenarios.<p>Usually this is the practice I have seen successful in all the big projects<p>- unit test for each function<p>- Functional Integration test to test the building blocks and functions together (for example rest API with business layer and database<p>- BDD Behavior driven development where you write automated test to test your business scenarios by initializing the seed data in the test systems<p>- Performance testing<p>If you have freelancer who charge per hour or if you have consultants which are expensive - they will write too many test and there code will be so complex in the end that when you want to hire permanent employees or other freelancers they will say "I would like to rewrite it"<p>The developers now a days like to do clean coding and for simple things they discuss too much even for a program of 2 + 2. there will be endless discussions and in some companies (diverse or multi culture) developers are always under mining each other with what they like and don't like
It depends.<p>I've seen so many absolutist opinions on testing that I'm distrustful of the wisdom of. When I see people say that there's only 1 way to test or not test things, and you always have to do it in a certain way, and fully test everything, I'm not sure that they're right. I think your need for code testing, and how much testing you do, matters more based on the context and risk you see through experience and good judgement.<p>> Making software for nuclear power plants? Sure, go slow and test the shit out of it.<p>> Making software for cars or space ships? Sure, go slow and test the shit out of it.<p>> Making large scale software for a big financial company where millions of dollars are at stake in the case of a fuckup? Sure, test the shit out of it. You can afford to throw 8 full teams of developers on every continent into nothing but testing, and then testing the tests, and then testing the tests for the tests.<p>> Building an API that multiple teams and customer are going to depend on? Yeah, testing that sounds like a good idea to ensure everything is working as expected.<p>> Running a profitable SaaS? Sure, at a certain point where you're not in wartime mode struggling for survival, fully testing and formalizing a lot of things makes some sense.<p>> But trying to get a tiny 1-human startup off of the ground that no lives depend on? I don't know what works for you best, but would you rather write code to make you money, or spend lots of time writing tests on stuff that's probably going to change a lot later anyway or that you don't even know if it really is a realistic business?<p>> Building a shitty app for some dumb marketing firm or whatever? I've literally never seen anybody in many types of businesses demand formal code testing. The only thing that matters to them is user-acceptance: the client and/or their QA team will manually test all of the functionality and tell you whether or not it's acceptable to them.
When working with Regex and PyParsing, I find it imperative to maintain unit tests on everything small because I am dealing with over 2,400 abstract keyword combination built out of some 423 EBNF syntax of ISC Bind9 named.conf configuration file and all in Python.<p>Tiny mistakes that I make are caught early on, and development gets a solid foundation in which to build the proper abstract syntax tree.<p>Oh, test code is about 1/4th the size of the original source code, line-wise, but it has 100% coverage on syntax linkage (and not necessarily 100% data value validation due to dependency on other field settings).<p>source: <a href="https://github.com/egberts/bind9_parser">https://github.com/egberts/bind9_parser</a>
The things that keep breaking get automated tests. The things that can't possibly break or it will cost me significant revenue or time get tests. Everything else gets to wait until it falls into one of the above two buckets.
This is something I have been considering as I work on firestarter [0] which is a going to be an open source starting point for building a SaaS.<p>I decided tests are a good thing to have, and especially when already set up and you can easily add more by following the pattern.<p>I prefer unit tests that are narrow in scope and avoid IO interactions (for example databases)<p>Of course anyone using my framework can ignore the tests if they desire. But my gut feeling is tests save you time in the short run as well as long run.<p>[0] <a href="https://github.com/mcapodici/firestarter">https://github.com/mcapodici/firestarter</a>
In our backend team we almost exclusively write integration tests.<p>For the frontend we do automated ui tests with a service called Autify.<p>This is not by any means extensive, BUT with very little time investment we get catch most of our bugs.
I write tests as I go along. It helps me challenge my assumptions about what I wrote, and it helps me write more robust code. I usually end up with 80-90% coverage.
Last company, no tests for the first 6 months. Current company after 2 months, have some unit tests for complicated parts in place, will start writing tests once bugs are resolved.<p>I guess it really depends on what your company is doing, experience of the team and the language/tech you are using.
I only write tests in two situations:<p>1) mission critical parts of my system that will cause damage if they break<p>2) small functions that I can't wrap my head around (eg some complex regex extraction)<p>Everything else can be fixed fast with minimal damage as it comes up. My test coverage is less than 1%.
Depends. For web-app and business logic, we have about 95% coverage. All our models and back-end processes are well-tested with error cases. Our front-end & E2E is much lighter and we focus on happy cases.<p>Firmware is mostly tested manually.
E2E for the core & commonly used features. But I plan to add more E2E tests for less commonly used features that I’d still consider core (these are manually tested for now due to time constraints).
Around 15-20% test coverage sadly enough. As a solo founder, small team, not enough funds in the bank, the most important thing is shipping the next change and iterating from there
The second you "think" a piece of software/a feature works, you should be verifying that this is the case automatically. It's often just as easy to write an automated test as test manually, just do it and that test will be there forever saving you from your own idiocy. You don't need to write the tests before you know how the feature is going to shake down to its implementation, but you'd sure do better to test before pushing to prod. You save yourself so much hassle in the long run with automated CI.
I only write tests for important stuff that I must continually work on. Although in the past 12 months, my tests have been written by either chatgtp or playground codex models before that. Which makes me lose 0-time testing.