Using Deep Learning to model personal visual aesthetics

Woah, figures 7 and 8 blew my mind. This reminds me of the various Pinterest boards I&#x27;ve curated in the past. I would set a theme and try to collect visual items in that theme.<p>Sometimes I could not describe the theme in mind clearly, for example &quot;foresty-earthy suburban adolescent feelings with little-to-no ruggedness but with a bit of a punk edge&quot;. Of course, no single image could fulfill the entirety of that theme (probably), so it&#x27;s fascinating to wonder how aesthetic preferences emerge in the mind, though it&#x27;s possible that with a description like that another person could filter images to match that description.<p>Are we combining various specific preferences (the color green, for example), or are we driven by the emotional flavor of a whole aesthetic object (a haze-covered mountain range evoking nostalgia for childhood hikes with siblings leading to the specific preference for pine trees leading to the specific preference for the color green, etc), basically top-down, bottom-up or a combo? Just some thoughts...

Man, this is crazy awesome stuff. I wonder if you could use a deep dream type thing to make an image more like your own style - that&#x27;d be next level.

Total side note, did anyone check out the EyeEm website (the apparent authors of the article)?<p>Their curation algorithms are doing a pretty good job! Their &quot;selected&quot; Galaxy photos look amazing. <a href="https:&#x2F;&#x2F;www.eyeem.com&#x2F;en&#x2F;pictures&#x2F;galaxy" rel="nofollow">https:&#x2F;&#x2F;www.eyeem.com&#x2F;en&#x2F;pictures&#x2F;galaxy</a>

Summary of approach: they embed the photos (convert photo-&gt;vector of numbers using T-SNE or CNNs [the details are actually here <a href="https:&#x2F;&#x2F;devblogs.nvidia.com&#x2F;parallelforall&#x2F;understanding-aesthetics-deep-learning&#x2F;]" rel="nofollow">https:&#x2F;&#x2F;devblogs.nvidia.com&#x2F;parallelforall&#x2F;understanding-aes...</a>) and then train a small-ish classifier (three fully connected layers) on top of it to capture a user&#x27;s preference. A pretty obvious approach, basic version should be doable in a hackathon, but cool result nonetheless.<p>&quot;We chose a three-layer multilayer perceptron (MLP) network as a good ranker, since it is able to capture the inherent non-linear shift in distribution between the user’s choices and the original training set. Notably, an MLP can be trained rapidly by leveraging GPU computation to obtain near-real-time results. This is important because it enables us to build interactive interface, as we’ll explain. We typically precompute a set of negative features (about 40,000 negative samples) and extract the positive features from the user-provided input.&quot;