TE
科技回声
首页24小时热榜最新最佳问答展示工作
GitHubTwitter
首页

科技回声

基于 Next.js 构建的科技新闻平台,提供全球科技新闻和讨论内容。

GitHubTwitter

首页

首页最新最佳问答展示工作

资源链接

HackerNews API原版 HackerNewsNext.js

© 2025 科技回声. 版权所有。

Who Will Test the Tests Themselves? (2017)

20 点作者 vp超过 4 年前

2 条评论

sitkack超过 4 年前
Mutation Testing [1] is a generalized form of fuzzing. It is also analogous to Sensitivity Analysis [2]. As part of closing the feedback loop between the code and the tests, if one had a repeatable way to break the code and measure the selectivity in the test result, one could ensure that they are testing the same thing as the code evolves.<p>Automatic program repair [3] tries to find patches to broken code, maybe goal directed program breaking (possibly using DNNs) could be used to infer properties for code so that better invariants could be discovered.<p>[1] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Mutation_testing" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Mutation_testing</a><p>[2] <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Sensitivity_analysis" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Sensitivity_analysis</a><p>[3] <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1807.00515" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;1807.00515</a>
评论 #24402997 未加载
hansvm超过 4 年前
Derailing the conversation a bit, what other strategies beyond mutation testing do you use for validating your tests? I&#x27;ve caught test bugs with a few techniques, but none of them are comprehensive, and I&#x27;d love to hear more thoughts. Here are a few examples:<p>(1) Validate assumptions in your tests -- if you think you&#x27;ve initialized a non-empty collection and the test being correct depends on it being non-empty, then add another assert to check that property (within reason; nobody needs to check that a new int[5] has length 0).<p>(2) Write tests in pairs to test the testing logic -- if your test works by verifying that the results for some optimized code match those from a simple oracle, verify that they don&#x27;t match those from a broken oracle.<p>(3) If you&#x27;re testing that some property holds, find multiple semantically different ways to test it. If the tests in a given group don&#x27;t agree then at least one of the tests is broken.
评论 #24403102 未加载
评论 #24403368 未加载