Find it funny how Stephen Wolfram probably used prompt engineering and fiddled a magic temperature variable to get the right answer there at the very end. My model can do anything, let me just flip a bunch of switches, and run it 100 times until if gives the answer I want.
Somehow, once LLMs emerged the crisis of people evaluating model performance hit an all time low. I appreciate what Knuth is doing here, and we need more of it. There's a lot of reporting that these tools are immaculate and screenshots show great results. Then when someone shows a bad one the community answer is "it's fake!" Or "you did something wrong". Have interacted with people lately who truly believe LLMs can violate mathematical laws... It's really annoying how people treat personable blackboxes to try to be first on the wagon.