I don't understand the wording:
"Our method, SILO, superoptimizes programs an expected 6.2% of our test set when compared with the gcc version 10.3 compiler's aggressive optimization level -O3."<p>Does that mean that the runtime is only 6.2% of the runtime after gcc optimization, or that there was a 6.2% reduction in runtime compared to gcc optimization?
Okay, had to register just because of the positive comments so far. I know nearly nothing about machine learning and may be a bit biased, but has anybody actually read the paper?<p>As I understand it, the model is trained on what compilers do anyway (the training data consists of compiler output from -O0 and -O3), so it won't come up with the kind of clever tricks that a human might.<p>One of their cherry-picked examples (fig5, prstree_empty) is incorrect: in one branch it is supposed to return true if the value at rdi+0x10 is a null pointer, but in the optimized version it falls through from "sete %al" to the next instruction, which overwrites that register so it will always return true.<p>It would be easy to fix by inserting another return instruction, but this clearly shows the neural net doesn't "understand" how to generate correct code. Besides some peephole optimizations, a lot of what it learned seems to be how to fool automated checks, and in this case even the authors who didn't spot the bug.<p>See also the section about verifier exploits, which are embarassingly simple: a completely empty "if statement" which depends on memory causes a function to pass when it clearly does nothing more than always return false.<p>The 6.2% figure is after "human verification" (again, one of the examples in the paper is broken!), down from 8.3% using only the automated verifier.<p>Not impressed.