I remain to be convinced that encoding statistical information about syntactical manipulation alone will somehow magically convert to semantic knowledge and agency if you just try really hard and do it a lot.
To me GPT-3 excitement is equivalent to when people get hyped about "defeating aging" after seeing some resveratrol trial or something.<p>Language is only part of it. And you can't get complete understanding without integrating spatial information. Take a look at Josh Tenenbaum's work for explanation of why.
It talks about how you can go from a language model that simply generates text to an agent that is capable of performing actions in the real world<p>Essentially, the missing pieces in the picture come down to input and output modules. "How do you formulate any given problem into a form that a language model can answer?".