Really unfortunate timing with Deepseek-R1 and the distills coming out at basically the same time. Hard for people to pay attention to, and plus open source > API, even if the results are a bit lower.
The set of math/logic problems behind AIME 2024 appears to be...
<a href="https://artofproblemsolving.com/wiki/index.php/2024_AIME_I_Problems" rel="nofollow">https://artofproblemsolving.com/wiki/index.php/2024_AIME_I_P...</a><p>Impressive stuff! But unclear to me if it's literally just these 15 or if there's a large problem set...
I really, really dislike when companies use GitHub to promote their product by posting a "research paper" and a code sample.<p>It's not even an SDK, library, etc., it's just advertising.<p>I've noticed a number of China-based labs do this; they will often post a really cool demo, some images, and then either an API or just nothing except advertising for their company (e.g. model may not even exist). Often they will also promise in some GitHub issue that they will release the weights, and never do.<p>I'd love to see some sort of study here, I wonder what % of "omg really cool AI model!!!" hype papers [1] never provide an API, [2] cannot be reproduced at all, and/or [3] promise but never provide weights. If this was any other field, academics would be up in arms about likely fraud, false advertising, etc.