just as ML engineers of 5 years ago generally knew nothing about SVMs, Gaussian Mixture Models, MLE vs MAP ...
ML engineers of today know little about VAEs, GANs, LSTMs, Reinforcement Learning.<p>Cold topics of yesteryear quickly fall off the map.<p>I would go further and claim that a 2024 ML Engineer may well be unaware of 2019 BERTology and GPT internals as well as 2020 fast Attention kernels and be today focused solely on:
a) Huggingface pipeline SFT, Diffusion, and Langchain RAG chunk GenAI orchestration.
b) the ever-prevalent mind-numbing drudgery of data munging that occupies 50% of a data scientist's time.<p>I mean, all that effort put into learning PyTorch / Tensorflow implementations of Transformer architectures - is it relevant without 10,000 TPU year resources, when big tech GPU hoarders expose simple GenAI REST APIs for mass consumption ?
An analogy: learning how CPUs are optimally engineered, when you dont have access to a fab, is completely irrelevant to app dev.
a complete bifurcation of skills between the laymen and the high priests.
OOTOH, if AGI ever takes off, it may be advantageous to have some idea of how it works.