Elliot Glazer seems to have been caught in a contradiction: <a href="https://xcancel.com/ElliotGlazer/status/1880809468616950187" rel="nofollow">https://xcancel.com/ElliotGlazer/status/1880809468616950187</a><p>Here he says that Epoch is "developing" a private test set that OpenAI doesn't have access to, but elsewhere Epoch strongly implied that this already existed. This kind of makes me lean towards "Epoch AI lied" instead of "Epoch AI got played." (Even the coauthors weren't informed about the funding, so Epoch does not deserve a presumption of good faith.)<p>I guess the real question: o3 was able to solve 25% of Frontier problems, so were these the problems whose solutions OpenAI had access to? If so, then that score is meaningless and dishonest.