I haven't tried afl-fuzz myself, although it sounds like world-class awesome software, but I'm a real believer in testing things with David MacIver's Hypothesis, which invokes your functions with random inputs, and then does similar canonicalization and minimization kinds of things. I like Hypothesis so much that when I wrote Dumpulse <a href="http://github.com/kragen/dumpulse" rel="nofollow">http://github.com/kragen/dumpulse</a> I added a Python interface to it <i>purely so I could test it with Hypothesis</i>. Which found bugs, of course, even though Dumpulse is around 100 instructions when compiled.
> When I got started, I had just learned how to use yacc (really Bison) and lex (really flex)<p>I've dug into parsers a few times, but everything I encountered seemed to think that once I had a parsed tree of commands it was obvious how to consume it...and it wasn't (for me). I've never had the free time to dedicate to experimenting that abstractly, so anytime I'm tempted to write a DSL or similar for a current problem I punt and do something else.<p>Does anyone have advice on how to find nice ways to use parsers/lexers in a practical way that doesn't involve a massive investment of time or assume I have a lot of abstract comp sci background? Every thing I've seen has been more of a compilers course, which seems overkill.
For anyone interested on the JVM side, I wrote a fuzzer last year using afl-like logic: <a href="https://github.com/cretz/javan-warty-pig" rel="nofollow">https://github.com/cretz/javan-warty-pig</a>
<i>> I also combed through the outputs to see what sorts of inputs were succeeding, what was failing, and observe how my program handled various edge cases. It was rejecting some inputs I thought should be valid, accepting some I thought should be invalid, and interpreting some in ways I hadn’t intended. So even after I fixed the crashing inputs, I still made tweaks to the parser to fix each of these troublesome inputs.</i><p>I love just looking through the absurd stuff AFL comes up with, even if it's not causing a crash or incorrect behavior. Like this bit of art it caused my parser generator to produce: <a href="https://i.imgur.com/VoV7cU9.png" rel="nofollow">https://i.imgur.com/VoV7cU9.png</a>
This is a great introduction to fuzzing, I love the section about minimising the corpus of fuzzed inputs and turning that into a test suite. That's something that should be done with all parsing software!
Fuzzing is super powerful, but can be a bit complicated to set up - that's why I'm working on a fuzzing-as-a-service platform[1] that automates a bunch of the steps described here.<p>If you're interested in trying out fuzzing without having to learn the intricacies of AFL or set things up manually, let me know[2] and I can get you set up with an account to play around with.<p>Happy to answer any questions about AFL/Fuzzbuzz!<p>[1] <a href="https://fuzzbuzz.io" rel="nofollow">https://fuzzbuzz.io</a><p>[2] everest [at] fuzzbuzz [dot] io
Someome made afl work with rust here: <a href="https://github.com/rust-fuzz/afl.rs" rel="nofollow">https://github.com/rust-fuzz/afl.rs</a>
The test files are coupled tightly to the implementation. He says it himself that when he wants to restructure things new tests have to be generated. It seems clumsy, but I don't have strong negative feelings on it.
The fuzzing is interesting but whats the deal with storing config files as binaries in 2019? Paradox has tons of text configs and doesn't even bother minifying them.<p>Am I underestimating the quantity or smth?