Boosting Compiler Testing by Injecting Real-World Code

This is a critically important line of research. Every time a new orthogonal testing approach is explored, it always finds a bunch of latent problems in existing codebases. Great contribution by the authors.

I did a very simpleminded form of this for testing the compiler in Steel Bank Common Lisp. The COMPILE function in SBCL should never signal an error condition, even on invalid code, so the test just involved splicing together fragments of code from existing Common Lisp code bases (packages in Quicklisp as well as other publicly available CL programs I could find) and trying to cause the compiler to error. It found a significant number of bugs.<p>This is a lot simpler that what these people were doing, as they wanted to produce code that was actually meaningful and could be run.