LLMs having capabilities they were not trained for – explained

&quot;After crossing an apparent critical value (between 1010 and 1011 effective parameters), the model seems to be catapulted into a new regime in which it can complete this (fairly complicated) task with drastically increased accuracy.&quot;<p>&quot;The important part about this phenomenon is that we, a priori, do not know that this will happen in advance or even at what scale it might happen. &quot;