Understanding Aesthetics with Deep Learning

Jurgen Schmidhuber has an interesting perspective on the rewarding aspects of art and music.<p>To paraphrase, &quot;Artists (and observers of art) get rewarded for making (and observing) novel patterns: data that is neither arbitrary (like incompressible random white noise) nor regular in an already known way, but regular in way that is new with respect to the observer&#x27;s current knowledge, yet learnable (that is, after learning fewer computational resources are needed to encode the data)&quot;.<p>In other words, enjoyment of art is about learning (easy) patterns. Schmidhuber likens &quot;fun&quot; to the improvement of an observer&#x27;s ability to compress a scene.<p>The source is worth the read: <a href="http:&#x2F;&#x2F;people.idsia.ch&#x2F;~juergen&#x2F;creativity.html" rel="nofollow">http:&#x2F;&#x2F;people.idsia.ch&#x2F;~juergen&#x2F;creativity.html</a>

I wonder how much our sense of aesthetics has to do with the perceived scarcity or effort of creation needed. I remember the first HDR photos looked absolutely mind-blowing to me, but now, as the process has been automated and is ubiquitous, it just looks tacky.

A better title would have been &quot;Using ML to rank and classify images according to aesthetics.&quot; Which in itself is quite impressive. But nowhere do we see indications that these algorithms &quot;understand&quot; the images, in any meaningful sense.

Aesthetics is a game of cat and mouse. Artists create some new things. Then critics and theorists observe the patterns of composition, color, proportion, etc., that are popular. These rules are canonized in books. Then artists challenge the rules.<p>The comparisons to music theory in this thread are apt. Music theory is always behind music production.<p>How can you understand aesthetics without understanding creativity?

I believe that there are factors that go beyond visible composition. Just a though experiment, I imagine that the brain would evaluate easthetics of two similar images differently depending on whether it is an image of an object it recognizes or not - when evaluating the image with an object other qualities of the object (that are not necessarily visible in the image) will be taken into account.

I don&#x27;t think computers would be able to understand aesthetics. It is a really high-level concept. Plus, I think deep-learning is a marketing mambo-jambo and does not perform much better than a linear SVM.