TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Test coverage only matters if it's at 100%

23 点作者 charlax超过 5 年前

24 条评论

cbanek超过 5 年前
As someone who worked somewhere where 100% code coverage was practically required, while it does point out flaws in your testing where you may have &quot;missed a spot.&quot; It still doesn&#x27;t prove that you&#x27;re testing everything. You never know if a library function you are calling beneath your code has a different path it can also follow.<p>&gt; I believe the only level of test coverage that&#x27;s worth it is 100% of the lines you want to be covered.<p>This statement I vehemently disagree with. It seems like the kind of thing someone would hit you with as an excuse for why testing isn&#x27;t useful at all because it&#x27;s not perfect. I have found in my experience that simple end to end tests give me the best bang for my buck, and even a test suite that covers 80% of the code is pretty good.<p>To paraphrase from the Google code reviewer guide: No code is perfect, just try to make it better than it was.<p>100% test coverage doesn&#x27;t tell you if you have race conditions, security problems, or networking issues. 100% test coverage doesn&#x27;t matter much in distributed systems, where you can individually test each component to 100% but still have the system not work as a whole. 100% coverage doesn&#x27;t mean you&#x27;re checking that all your returned results are verified and what they should be.<p>&gt; The problem with having a goal that is not 100% is that it leaves room for interpretation and negotiation.<p>By handing out edicts like these, you&#x27;re taking away tools like negotiation and common sense and replacing them with blanket rules. Time is a finite resource and test coverage is only one small part of testing. Going from 95% to 100% might take more time and provide less value than other kinds of testing and other concepts (like UX, or market fit, where you test that what you&#x27;re building is usable and useful).<p>Just because you answer all the questions right on the test doesn&#x27;t mean you have perfect knowledge of the situation. Thinking such is hubris and leads to errors&#x2F;defects.<p>By excluding lines and saying only make sure you cover what you&#x27;re testing you&#x27;re intentionally making blind spots. Just let the coverage number be what it is. Look at your coverage reports. Check the coverage of key components.<p>&gt; I think it is a perfectly fine decision to exclude some code from coverage if the tests are too expensive to write (because of the required setup for instance).<p>No, no, no, no, no x a million. This is the fallacy that if you unit test each component, then when you put it together it&#x27;s perfect because each part is perfect. You still have to do integration tests and complicated setup tests. If a test takes a long time, try to make it shorter, don&#x27;t try to avoid testing it. Anything you avoid testing is where bugs will nest and code will churn.<p>To sum up, most of the important, complicated, and hairy bugs are not found by unit testing. It is the simple bugs that are found by unit testing. These are important to get out of the way because a system with a thousand cuts can still kill you, but in no way is 100% coverage and tests passing mean anything other than 100% coverage and tests passing.
评论 #20898474 未加载
评论 #20915845 未加载
评论 #20898409 未加载
aluminum96超过 5 年前
This is totally ridiculous.<p>Test coverage clearly matters even if it&#x27;s not 100%, as anyone who has ever worked on an older codebase knows.<p>As long as you&#x27;re not causing coverage regressions, it is perfectly acceptable to leave untested code that &quot;should&quot; be tested, but hasn&#x27;t been changed in years.<p>Test it when you change it; don&#x27;t test for the sake of achieving a coverage benchmark.
评论 #20898036 未加载
aeternum超过 5 年前
This is the type of article I&#x27;d expect from a junior engineer.<p>Test coverage matters even if it is not 100%, especially if it covers the most important use-cases. Cost and time always needs to be considered. 100% coverage may not be worth it. Coverage based on LOC is also an arbitrary metric in some sense, the author himself has examples of that. From a business POV, use-case coverage is more important.
lvturner超过 5 年前
We had some mild debates at an old company on this matter (I don&#x27;t think anyone was really ever pro 100% coverage, but we were under pressure to increase coverage in order to provide momentum and to emphasise that automated testing should be routine)<p>It resulted in a mug being made in the same style as our company mugs but with the words &quot;Fuck Coverage&quot; instead of the company name.<p>Eyebrows were raised when it was mistaken for a &#x27;real&#x27; mug and taken in to a board meeting by a manager however..
kempbellt超过 5 年前
So, don&#x27;t bother writing any tests unless you can test everything?<p>At a previous shop, we had a simple test that ensured user could log in for a while, until we created more coverage. We didn&#x27;t test every aspect of the code. This simple sanity check saved us once or twice.<p>Should we have not written this test just because we couldn&#x27;t test every conceivable piece of code?<p>Don&#x27;t let perfect be the enemy of good...
评论 #20898072 未加载
josepot超过 5 年前
Test coverage is a very dangerous metric. IMO it will only be useful the day that we find a way to not take into account those tests that are testing implementation details. Every time that I see a high number being enforced, the end result is always a disaster: lots of poor quality tests that are only there in order to bump up that metric. Those tests are poison: they don&#x27;t aim to test correctness, they make refactors&#x2F;improvements a PITA, they just lock you down to a concrete implementation and they don&#x27;t even give you any guarantees that your code works correctly.
评论 #20898287 未加载
amflare超过 5 年前
The title is disingenuous. A better title would be &quot;100% test coverage should be the goal&quot;. Though that is laughably obvious and would generate no clicks, so I can see why the author made the choice they did.<p>That said, if you have a single test, that better than no tests. And 90% is better than 80%. Yes, 100% is ideal, but if you let perfect be the enemy of good, than you waste time when low hanging fruit breaks.
评论 #20897928 未加载
t-writescode超过 5 年前
The irony to me is that the examples he provides for things not to cover are exactly things I think should be covered. Short-circuit logic is ripe for bugs and the code that causes the short-circuit should definitely be tested!
评论 #20898259 未加载
rraghur超过 5 年前
- Don&#x27;t write unit tests for testing code, use them to guide your design.<p>While that statement is itself is somewhat questionable, it&#x27;s a little better and IMO reflects the purpose of unit tests better. Usually, if I&#x27;m finding it hard to write or setup a test, that&#x27;s a signal that something&#x27;s off. Also, UTs are absolutely lovely when you&#x27;re refactoring code - as long as code tests invariants and publicly visible behavior instead of implementation. So even if the part you&#x27;re refactoring has say 60%, you&#x27;d still be in a much more confident of making changes.<p>Code coverage under anything other than unit tests can be very misleading - for ex I&#x27;ve seen places where system &amp; integration tests are run on instrumented code (and little to no unit tests) and since a huge swathe of code will be hit (mostly happy path), there&#x27;s a false sense of code quality and safety which is probably more dangerous than no tests.<p>Overall, &quot;When a measure becomes a target, it ceases to be a good measure.&quot; applies most often to test coverage.
everyone超过 5 年前
The author is either crazy, or else maybe works in a very niche area where this makes sense. I work in games, and some modules are very conducive to automated testing.<p>Eg. complex math functions, it&#x27;d be very easy for them to be not quite right and you would never notice without tests, and also they will probably never change.<p>An area that would be stupid to test (with automated tests) for example, would be a UI menu.<p>* The test (ie. replicating a user pressing buttons via code) is probably gonna be more complex and less maintainable than the code its testing.<p>* If there is a bug it will be obvious in human acceptance testing.<p>* UI menus are gonna change drastically during development.<p>Anyway, I could see a misguided requirement for 100% code coverage massively hampering a project and potentially killing it.
评论 #20898173 未加载
dbcurtis超过 5 年前
100% of what? Oh... line coverage, in Python.<p>Well, with Python, the compiler doesn&#x27;t do type checking, so 100% coverage (or close) is very important. One of his cases of &quot;what do you not need to check&quot; is error-handling. Yet, having something in production that should raise an exception that gets handled versus having the exception blow up and crash the system is a critical system stability defect. And in fact, in my personal code, since I tend to try to create exception messages that are as specific as possible, I must admit with chagrin that screwing up a &#x27;&#x27;.join() to create the exception message is one of my most common errors. I at least touch-test my exception code, without exception.<p>And then... line coverage? Oh my, that is just the beginning. I pretty much always use the hypothesis package so that everything get exercised with a large variety of inputs. Inputs that I didn&#x27;t have to think up, and avoid my pre-existing biases of what might get thrown at the function.<p>Line coverage does do much to really ensure that you have reached &quot;interesting conditions&quot;. I wish Python had a good tool for that kind of instrumentation. Line coverage is really only a &quot;touch-test&quot; and mainly useful as a check-in gate, not as production testing.
评论 #20898605 未加载
pacaro超过 5 年前
For me the absence of coverage, or low coverage, is a useful piece of information. It&#x27;s not the whole story but it is a smell, and it should be taken into consideration with the other smells&#x2F;signals<p>Conversely perfect or high coverage is not a positive signal <i>per se</i>, but rather just the absence of a negative signal
alephnan超过 5 年前
Edgy, but wrong.<p>Is there a difference between 79.9% and 80%? Maybe not. My team is increasing test coverage for a legacy code base. 60% to 80% DOES matter. And it’s not just the fact that the tests flags regression, but the very process of increasing test coverage uncovers bugs &#x2F; dead code.
nrook超过 5 年前
There&#x27;s an insight here I haven&#x27;t seen elsewhere. Everyone but the most blind ideologue knows 100% test coverage isn&#x27;t a good use of your time. But it&#x27;s hard to care about increasing coverage from 89% to 90%, or to feel justified in objecting during a code review because the change under review would decrease coverage from 90% to 89%.<p>The solution to this dilemma is to make it easy to mark code as &quot;OK not to be tested&quot;. It&#x27;s annoying to have test annotations in code, but I&#x27;d expect mostly entire modules will be exempted, so it&#x27;s not that big a deal. And this way, the coder has to either write tests or make a specific decision not to.
choward超过 5 年前
&gt; With this setup, test coverage will be at 100% even though we did not test the case where the toaster is not plugged because it is too simple. Writing a test for this &quot;functionality&quot; would not be worth it. Therefore we explicitly exclude it from coverage. You should revisit this decision if it becomes more complex and needs to be tested.<p>This is ridiculous. He writes unreachable code and says he doesn&#x27;t test it because it&#x27;s &quot;too simple&quot;. No, you&#x27;re not testing it because you can&#x27;t test it, because it&#x27;s code that will NEVER run. Instead of deleting the pointless code he marks it as not needing test coverage.
rbongers超过 5 年前
Not my experience at all... Tests need to be intelligently written for the project at hand, which sometimes means testing something to freeze the API, writing tests for small functions that are easier to test in a suite than anywhere else, and especially catching a problem where your program keeps failing, like testing with an older version of a library or runtime, or even testing something that you know could be fragile.
ihaveajob超过 5 年前
Seems like the Intelligent Design nuts arguing &quot;what good is half an eye?&quot;. Well, it&#x27;s half as good as a full eye, a lot better than no eye.
the_narrator超过 5 年前
I think what really matters is that your test coverage percentage increases over time and never goes down. A goal of 100% might be ideal, but is rarely realistic.
mabbo超过 5 年前
This is such bad advice that&#x27;s so well written I can only assume the author is trolling HN-types with it.<p>I&#x27;m serious. I think we&#x27;re being trolled here everyone.
al2o3cr超过 5 年前
<p><pre><code> Some lines of code should be tested twice to ensure they&#x27;re correct. </code></pre> Translation: &quot;100% coverage&quot; is a vanity metric
m4r35n357超过 5 年前
Not only that, you should subject your automated tests to mutation testing. No link coz wikipedia seems to be down for me.
matt2000超过 5 年前
Given unlimited time, money and importance of the code in question - go for 100% coverage. Otherwise, probably not.
tracker1超过 5 年前
100% coverage only ensures that you&#x27;ve looked at everything twice. Nothing more, nothing less.
mpweiher超过 5 年前
There is actual empirical evidence that this is false.
评论 #20897774 未加载
评论 #20897570 未加载
评论 #20897771 未加载