To me, academia has always been about developing proof-of-concepts for techniques that industry adopts years later. These proof-of-concepts are intricate but small, so they require a lot of ingenuity but not much resources or man-hours (a company can also hire far more employees than a professor).<p>AI is no exception. All evidence suggests that current LLMs don't scale (EDIT: they do scale, but at a certain point somewhere around GPT4 the scaling slows down very quickly), so we need new techniques to fix their (many) flaws. A proof-of-concept for a new technique doesn't need hundreds of millions of dollars to demonstrate huge potential: such a demonstration only needs a small model using the new technique, a small model using the current SOTA, and some evidence that it scales (like slightly larger models that show the new model isn't slowing down its scaling vs SOTA).<p>Academia is also about creating better explanations for things we already have. So researchers don't even need to develop new models, they can simply create small existing models and demonstrate some new property or explanation to get a worthwhile result. We probably need better explanations for how and why current LLMs work in order to create the better models mentioned above.<p>EDIT: At least how it's supposed to work, you don't even need to show success either. Academics merely need to show a new technique with the <i>potential</i> to improve the SOTA. Even if the technique is a huge failure, the work still advances the future of AI, by contributing to our understanding of models (what works and what doesn't) as well as removing one potential option.