Self-Supervised Learning from Images with JEPA (2023)

It’s not new and only superior in a very narrow set of categories.

Needs a (2023) tag. But definitely the release of ARC2 and image outputs from 4o got me thinking about the JEPA family too.<p>I don&#x27;t know if it&#x27;s right (and I&#x27;m sure JEPA has lots of performance issues) but seems good to have a fully latent space representation, ideally across all modalities, so that the concept &quot;an apple a day keeps the doctor away&quot; becoming image&#x2F;audio&#x2F;text is a choice of decoder rather than dedicated token ranges being chosen even before the actual creation process in the model begins.

GPTs are in the “exploit” phase of the “explore-exploit” trade-off.<p>JEPA is still in the explore phase, it’s good to read the paper and have an understanding of the architecture to gain an alternative perspective.

Not new, not notable right now, not sure why it&#x27;s getting upvoted (just kidding, it&#x27;s because people see YLC and upvote based on names)

JEPA is presumably superior to Transformers. Can any expert enlighten us on the implications of this paper?