From
1.criticism of how inadequate the transformer architecture is
2.depletion of training data
3.improvements to the attention mechanism
4. AGI supposedly being miles away<p>where does the new results put the field and what avenues need more focus and what others need less focus to the point of complete cutoff?
It isn't as impressive as they'd like you to believe.<p>They fine tuned it for that test.<p>What we are seeing is a marketing trick to keep markets and investors excited about AI. It's a trillion dollar industry for NVidia and other players. Fake it until you make it.<p>If you look deeper, there's very little change since GPT-3.5 and Anthropic have catched up with everything OpenAI has built so far.<p>Sora was a huge fluke with other companies clearly ahead of it. Also mostly useless.<p>The numbers don't add up.