Being one of those lucky few at Google Brain who switched into ML early enough to catch the wave... I fielded my fair share of questions from academic friends about how they could get one of those sweet gigs paying $$$ where they could do whatever they wanted. I wrote these two essays basically to answer this class of questions.<p>Why did Brain Exist? <a href="https://www.moderndescartes.com/essays/why_brain/" rel="nofollow">https://www.moderndescartes.com/essays/why_brain/</a>
Who pays you? And why? <a href="https://www.moderndescartes.com/essays/who_pays_you/" rel="nofollow">https://www.moderndescartes.com/essays/who_pays_you/</a><p>Free lunch briefly existed for a small lucky few in 2017-2021, but today there is definitely no more free lunch.
I appreciated the humility in this paragraph in particular<p>> once the first generation of lucky PhD’s (including me!) who were there not out of career prospects but mostly out of luck (or unluck), we started to have a series of much more brilliant and purpose-driven PhD’s working on deep learning. because these people were extremely motivated and were selected not by luck but by their merits and zeal, they started to make a much faster and more visible progress. soon afterward, this progress started to show up as actual products.<p>I have to say, though, that AI and robotics are going through similar transitions as were highlighted in TFA. Robotics has been basically defined by self driving cars for a long time, and we're starting to see the closure of some very big programs. Already the exorbitant salaries are much lower based on what I've seen, and demand is flat lining. My hope is that the senior level engineers with good PhD backgrounds move out into the broader field and bring their experience and research zeal with them as a force multiplier. I expect the diaspora of talent to reinvigorate industry innovation in robotics.<p>So it will be with LLM-focused researchers in industry in the next phase after we pass peak hype. But the things those battle-scarred researchers will do for the adjacent fields that were not hype-cycled to death will probably be amazing.<p>Unless they succeed in replacing themselves with general AI. Then all bets are off.
This is probably the best example of what it feels like to be someone dealing with cognitive dissonance of Elite Overproduction in AI.<p><a href="https://en.m.wikipedia.org/wiki/Elite_overproduction" rel="nofollow">https://en.m.wikipedia.org/wiki/Elite_overproduction</a><p>Since 2012 (Alexnet winning Imagenet with DL) “AI” has been dominated by corporations for one simple and obvious reason that Suttom has pointed out over and over: The group with the best data wins<p>That wasn’t always true. It used to be the case that the government had better data or academia had better data, but it’s not even close at this point to the point where both of those groups are nearly irrelevant in either applied AI or AI research.<p>I’ve been applying “AI” at every possible chance I have since 1998 (unreal engine + A* + knn …) and the field has foundationally changed three or four times since.<p>When I started, people looked at those of us that stated out loud that we wanted to work on AGI as totally insane.<p>Bengio spoke at the AGI 2014 conference, and at a lunch between myself, Ben G, Josha Bach and Bengio we all agreed DL was going to central to AGI - but it was unclear who would own the data. Someone argued that AGI would most likely come out of an open source project and my argument was there are no structured incentives to allow for the type of organization necessary at the open source level for that.<p>My position was that DL isn’t sufficient - we need a forward looking RL streaming reward system that is baked into the infrastructure between planning and actuation. I still think that’s true and why AGI (robots doing things instead of people at greater than human level) is probably only a decade away.<p>It’s pretty wild that stating out loud that your goal is to build superhuman artificial intelligence is still some kind crazy idea - even though it’s so obviously the trajectory we’re on.<p>As much as I dislike OpenAI and the rest of the corporate AI companies, I respect the fact that they’ve been very vocal about trying to build general intelligence and superhuman intelligence.
<i>"There are much fewer opportunities if they do not want to work on productizing large-scale language models, and these positions are disappearing quickly."</i><p>Yes. In 1900, the thing to be was an expert electrician. I had a friend with a PhD in bio who ended up managing a coffee shop because she'd picked the wrong branch of bio.<p>LLMs may be able to help with productizing LLMs, further reducing the need for people in that area.
Trained as a physicist I became acutely aware of what mismatches in the academic job market look like and, particularly, how smoking hot fields can become burned out fields in just about <i>the time it takes to complete a PhD.</i><p>(Some physicists told me about how quickly Ken Wilson's application of RG to phase transitions went from the the next big thing to old hat, for instance.)<p>When I can bear to read editorials in CACM I see the CS profession has long been bothered by whipsawing demand for undergraduate CS degrees. I've never heard about serious employment problems for CS PhDs and maybe I never will because they have a path to industry that saves face better than the paths for physics.<p>Maybe we will hear about a bust this time. As a cog in the social sciences department, I used to have a view of a baseball diamond out my office window but now there is a construction site for a new building to house the computer science, information science and "statistics and data science" departments which are bulging in undergraduate enrollment.<p>Will there finally be a bust?
As someone who did a PhD ending 4y ago and joined a industry research lab, I can relate to this a lot, I think this is very much spot on<p>> a lot of these PhD’s hired back then were therefore asked to and free to do research; that is, they chose what they want to work on and they publish what they want to publish. it was just like an academic research position however with 2-5x better compensation as well as external visibility and without teaching duties,<p>exactly!<p>> such process standardization is however antithetical to scientific research. we do not need a constant and frequent stream of creative and disruptive innovations but incremental and stable improvements based on standardized processes.<p>A lot of the early wave AI folks struggle with this. They wanna keep pushing wild research ideas but the industry needs slow incremental stuff, focused on serving
If someone believed they will earn 2-5x better than in academia, with full freedom to work on whatever interests them, and no need to deliver value to the employer... Well, let's say "ok", we have all been young and naive, but if their advisors have not adjusted their expectations, they are at fault, maybe even fraudulent.<p>Even being in elite research groups at the most prestigious companies you are evaluated on product and company Impact, which has nothing to do with how groundbreaking your research is, how many awards it gets, or how many cite it. I had colleagues at Google Research bitter that I was getting promoted (doing research addressing product needs - and later publishing it, "systems" papers that are frowned upon by "true" researchers), while with their highly cited theoretical papers they would get a "meet expectations" type of perf eval and never a promotion.
As a consultant data scientist who attended NeurIPS this year I was surprised at how few of the talks and posters covered innovations practically applicable outside of niche domains. Those that did (DL for tabular data, joint LLM training with RAG, time series foundational models) were swamped with attendees. It's not that there aren't jobs for innovators, it's just that there needs to be some amount of applicability of your research outside the theory of learning.
This story plays out so often that here should be a law about it: Supply lags demand, prices soar, everybody hears about it, everybody pours in, supply surges, demand normalizes, supply overshoots demand, prices collapse. Already happened to software engineering, data science. Keeps happening to hardware production every few years. Sounds like AI research is headed that way too.
"betrayed", "promised" - these are quite telling about the sense of entitlement these people have (rightly or wrongly).<p>The world doesn't owe you anything, even if you did a PhD in AI.<p>Sorry if you picked the wrong thing, but it's the same for anything at degree level.
It's kinda funny, the general dread regarding AI in tech circles is mostly due to fears that all jobs will be automated away, while the concerns expressed by the researchers according to the author are ironically a lot narrower in the sense that they haven't learned the currently cool topic.
I think the author is twisting things a bit to suit their own hiring needs.<p>Researchers at big tech companies always struggled to get promoted without showing some sort of impact.<p>What changed is before they could just publish papers (with 0 reproducibility due to “ai ethics”) and still coast by with a decent salary.<p>ChatGPT ended the whole “the paper is the product” phenomenon, and the end of ZIRP ended the smooth run for slackers in big tech overall.<p>AI phds are still in an amazing position compared to any other new grad software engineer.
In my lab, we try to focus on the science and less on applicative deep learning, of course it has its uses and we develop 'fashionable' deep learning with some students, but I think the value of a PhD remains in the path taken.<p>It's all that you learn doing it, it is not just knowledge, it's the scientific method, social skills, learning to communicate science.<p>A PhD is a personal path, an experience in life that changes it profoundly, I hope that those students will be able to handle any kind of future issues, not just a very niche applicative aspect of deep learning.
Big conferences suck in just about every area. Back when I was going to the joint math meetings (a couple few thousand math people), the dominant smell was adrenaline, as departments tried to streamline their hiring process into the conference setting... Phds seeking postdoc positions were therefore trying to talk up their work publicly and rushing around to interviews on the side. It sucks as a conference, though: you get lots of heat, and little light.<p>I get the same feeling going to big ML conferences now. The incentives are all about job prospects and hype, rather than doing cool stuff.<p>The sweet spot is the 200 person conferences, imo. You have a much better focus on the science, can literally meet everyone if you put your mind to it. And the spaces tend to be more about the community and the research directions, rather than job prospects.
There's an underlying sentiment through the article that there isn't more significant research to do to further improve ai and generate more revenue. This is a large gap and myopic.<p>For example, the amount of compute needed to obtain state of the art on a benchmark is only obtainable by big labs. How can we improve the learning efficiency of deep learning so that one can obtain similar performance with 100x less compute.
> some of them probably feel betrayed, as the gap between what they were promised earlier and what they see now is growing rapidly.<p>When you enter a PhD program, you are promised nothing else but hard work (hopefully within a great research group). I don’t know what the author is referring to here.
I'm reading the author's PhD thesis at the moment, which he seems to have successfully applied to some modelling projects.<p>I'd imagine the most significant aspect of this betrayal of PhD students, as far as it exists, is to polarise the domain along research lines that never very clearly mapped to the perennial objectives of modelling or mere curve-fitting. Had a thread of 'modelling science' been retained then it would always be useful.<p>Businesses may, at the moment, appear to have been confused into the value of curve-fitting --- the generic skills of algorithm design, scientific modelling, and the like, will survive the next wave of business scam.
This is probably anecdotal evidence that the field is maturing rather rapidly.<p>Phd hires will continue ofcourse, its the volume and selection criteria that get adjusted. When a technical domain gets standardized its natural that the ratio of PhD's to Masters in a team lowers.<p>Mostly teams with a mandate to innovate (or at least signaling that intention) will have more of the former. A succesful Phd will always be a strong filter for certain types of abilities and dispositions. Just don't expect stampedes to last forever.
I guess I'm a little bit optimistic, but looking at the title I was thinking that the AI models had reached the point where <i>they</i> were the ones expressing anxiety and frustration :)<p>Never mind . . .
This is the best time in history to research AI and get paid for it.<p>I respect your perspective and acknowledge I might be missing key context since I didn’t attend the conference. That said, one interpretation is that you’re grappling with the rapid acceleration of innovation—a pace you’ve helped shape. Challenges that once seemed distant now feel imminent, creating a mix of excitement and unease, like the moment a rollercoaster drop starts to terrify.
When I started my MS in 2018, my focus and interests were in complex systems and computer vision. I remember back then being more interested in Hinton’s capsule nets when the paper “attention is all you need” was new. I am looking forward to when this LLM fever passes. The most interesting thing to come from all of this is IMO, the semi-recent uses of RAG and vector search. More recently, having had some exposure to a CS undergrad group of ~120 seniors, anxiety and frustration (related to the job market) were exactly how I would have described the zeitgeist as well… as another post mentioned, I suppose this is all part and parcel of software engineering - and now applications of machine learning (and PhD degrees in any/all fields for that matter) - increasingly becoming ever more commoditized. Such is life in a world of capitalism I guess?
Absolutely no chance am I reading an article that doesn’t even give the respect of capitalising sentences. Honestly despicable and I feel personally offended
Now imagine how 80% of humanities grads feel when they didn’t get the job they thought they were promised. (With 10x the debt a CS PhD will finish with)
I know this comment doesn't add much to the discussion, but why, god, why does the author feel the need to write completely in lowercase? It seems completely unnecessary. Impress with your words, not with a weird formatting.