Thank you @amitness for this wonderful website with graphical explanations :)<p>The PIRL technique in question here seems useless to me, because its loss deliberately trains it to behave differently from what a human would do. But that overview page <a href="https://amitness.com/2020/02/illustrated-self-supervised-learning/" rel="nofollow">https://amitness.com/2020/02/illustrated-self-supervised-lea...</a> is gold.
This looks a lot to me like learned image descriptors from the computer vision community. Some examples are
Discriminative learning of local image descriptors (<a href="http://matthewalunbrown.com/papers/pami2010.pdf" rel="nofollow">http://matthewalunbrown.com/papers/pami2010.pdf</a>)
DeepDesc (<a href="https://icwww.epfl.ch/~trulls/pdf/iccv-2015-deepdesc.pdf" rel="nofollow">https://icwww.epfl.ch/~trulls/pdf/iccv-2015-deepdesc.pdf</a>)
L2-net (<a href="http://www.nlpr.ia.ac.cn/fanbin/pub/L2-Net_CVPR17.pdf" rel="nofollow">http://www.nlpr.ia.ac.cn/fanbin/pub/L2-Net_CVPR17.pdf</a>)
The issue with this is that you (most of the time) don't want your image representations to be immune to geometric transformation.<p>A rotated p should be recognized as a d. In nature pictures you want to recognize blue at the top as sky and blue at the bottom as water.