Check out the Delta Debugging paper. There has been a ton of follow-on work (1000+ direct citations), and lots of tools exist.<p>Here's the citation, abstract and link to a PDF copy:<p>Simplifying and isolating failure-inducing input
Andreas Zeller, Ralf Hildebrandt
IEEE Transactions on Software Engineering 28 (2), 183-200, 2002<p><i>Given some test case, a program fails. Which circumstances of the test case are responsible for the particular failure? The delta debugging algorithm generalizes and simplifies the failing test case to a minimal test case that still produces the failure. It also isolates the difference between a passing and a failing test case. In a case study, the Mozilla Web browser crashed after 95 user actions. Our prototype implementation automatically simplified the input to three relevant user actions. Likewise, it simplified 896 lines of HTML to the single line that caused the failure. The case study required 139 automated test runs or 35 minutes on a 500 MHz PC.</i><p><a href="http://cs.purdue.edu/homes/xyzhang/fall07/Papers/delta-debugging.pdf" rel="nofollow">http://cs.purdue.edu/homes/xyzhang/fall07/Papers/delta-debug...</a><p>Edit: adjusted tone
An even more fine-grained binary search technique for isolating compiler bugs is optimization fuel:<p><a href="http://blog.ezyang.com/2011/06/debugging-compilers-with-optimization-fuel/" rel="nofollow">http://blog.ezyang.com/2011/06/debugging-compilers-with-opti...</a>
<a href="https://dlang.org/blog/2020/04/13/dustmite-the-general-purpose-data-reduction-tool/" rel="nofollow">https://dlang.org/blog/2020/04/13/dustmite-the-general-purpo...</a><p>Dustmite (say thank you to cybershadow) is the D community's beloved test case reduction tool
JReduce (<a href="https://dl.acm.org/doi/abs/10.1145/3453483.3454091" rel="nofollow">https://dl.acm.org/doi/abs/10.1145/3453483.3454091</a>) uses similar technique but with semantics for debugging such problems with Java decompiler and bytecodes.<p>The problem is we don't know if compiling with these functions follows the monotonicity of binary searching: what if a problem is caused by a special combination of two functions? You have to arrange these two functions in the right order in order to find the problem.
Binary search is also helpful in code that doesn’t reliably produce useful stack traces (<i>side eyes Promise.then</i>). Comment out half of the potentially implicated code. If you get the same error, you very likely haven’t found a stack frame. If you get a different error or no error at all, keep halving until you either isolate the thing or know where to look.
D also has Dustmite (<a href="https://dlang.org/blog/2020/04/13/dustmite-the-general-purpose-data-reduction-tool/" rel="nofollow">https://dlang.org/blog/2020/04/13/dustmite-the-general-purpo...</a>), same idea.
> You may have heard of bisecting from geometry or from git bisect1. Those are the two places I heard about it<p>I heard of it from when I first started programming and I cut the code in half (maybe 3rds or other units, close enough) until I found which part was breaking the program.
bugpoint is the equivalent for llvm: <a href="https://www.llvm.org/docs/CommandGuide/bugpoint.html" rel="nofollow">https://www.llvm.org/docs/CommandGuide/bugpoint.html</a>
I have wondered how to extend this to combinations of elements.<p>What if the bad outcome only happens when two or more elements are enabled?<p>Is there a generalization of binary search for powersets?<p>Can this be done efficiently?
cvise is another tool for test case reduction, similar to creduce mentioned in the article:<p><a href="https://github.com/marxin/cvise" rel="nofollow">https://github.com/marxin/cvise</a>