Gradients are not all you need

Seems to me like the whole history of neural nets is basically crafting models with well-behaved gradients to make gradient descent work well. That, and models that can achieve high utilization of available hardware. The surprising thing is that models exist where the gradients are <i>so</i> well-behaved that we can learn GPT-4 level stuff.

&gt; Gradients Are Not All You Need<p>Sometimes you need to peek at the Hessian.<p>Seriously though, what is intelligence if not creative unrolling of the first few terms of the Taylor expansion?

Global optimization techniques which don&#x27;t rely on gradients seems theoretically superior in all instances, except that we haven&#x27;t found super fast ways to run these kinds of optimizers.<p>The cartpoll demo famously tripped up derivative based reinforcement learning for awhile.

Interesting paper, thanks for bringing this up! I have been working on methods for trajectory optimization using both, analytic gradient computations and black box stochastic gradient approximations (proximal policy optimization).<p>I was always wondering about a question that is touched in the paper: despite the analytic gradient computation being intuitively more efficient and mathematically correct, it is much harder to learn a policy with it than with the “brute force trial-and-error” black box methods.<p>This paper brings many new perspectives on why.

&gt; chaos based failure mode<p>I studied this in undergrad, but it’s not the same thing the paper is talking about

My one wish is that machine learning papers would use paper titles that actually described what the paper was about. I suppose there is a certain &#x27;evolutionary pressure&#x27; where clever titles &#x27;outcompete&#x27; dryer, more descriptive titles (or it seems that way). But i don&#x27;t like it.