Formal Algorithms for Transformers

I find the distinction introduced in this paper into encoder-decoder Transformers, encoder-only Transformers and decoder-only Transformers very useful for my informal understanding of the different architectures. Thank you for this clear clarification.

I like how this seems to actually be self-contained. They even have a list of notations in the end.

This is a fantastic resource. It's the missing piece of many machine learning articles.

Zero diagrams, but maybe they wouldn’t be helpful to clarify the concept? Guess it depends on the types of learners, I’m not sure.

I can't tell who this paper is aimed at. It isn't formal. It isn't mathematical. It isn't a good description and doesn't have good coverage. I can only assume it is for citations.

I was assuming electrical transformers.

familiar with basic ML terminology might be an understatement