At first I thought this had something to do with the classic "breadth vs. depth" notion on learning stuff -- if you're preparing for the MCAT it is better to have breadth that covers all the topics than depth in one or two particulars for the exam, but this is actually just about the dimensions of the neural network used to create representations. Naturally, one would expect a "sweet spot" or series of "sweet spots."<p>From the paper at <a href="https://arxiv.org/pdf/2010.15327.pdf" rel="nofollow">https://arxiv.org/pdf/2010.15327.pdf</a><p>> As the model gets wider or deeper, we seethe emergence of a distinctive block structure— a considerable range of hidden layers that have very high representation similarity (seen as a yellow square on the heatmap). This block structure mostly appears in the later layers (the last two stages) of the network.<p>I wonder if we could do similar analysis on the human brain and find "high representational similarity" for people who do the same task over and over again, such as play chess.<p>Also, I don't really know what sort of data they are analyzing or looking at with these NN, maybe someone with better scansion can let me know?