I wonder if they included sexy images in their negative training sets -- many videos accrue millions of views (and ad dollars) by having a few frames of cleavage interspersed with other (often derivative) footage.<p>It would be great if their algorithm picked a thumbnail that reflected the <i>entire</i> video, not just a few frames specifically chosen to game people's compulsive clicking.
It looks like they prefer images with a few large faces near the center of the frame. That's probably the right answer for social media. (Plus a cat recognizer.) Used on news footage, you probably get the talking head rather than the news event.
There's an outside company that was working on this: Neon Labs (<a href="https://www.neon-lab.com/" rel="nofollow">https://www.neon-lab.com/</a>).<p>Their insight is that not only are there images that are "high-quality", but also images that are positive. Positive images get more clicks, over just a decent image. I wonder if that information is encoded in the RNN in some way.<p>(This is where I'd normally rant about RNNs and other ML techniques hiding this information from their creators by locking it up inside the black box, but I'll save that for another day.)
They've got to be training on more inputs than mentioned. For example, is one or a close set of times in the video linked externally and generating traffic? Grab the entire set of frames from that time period and run it through the quality classifier, there might be iconic frames from that section that people are looking for.<p>Are people re-watching a small segment of the video? Try classifying individual frames from that segment or just before. Of course, those are often action moments that result in smeared motion and artifacts and may not result in a quality thumbnail.<p>These ideas also only come into play when a video has been live for a while, after the uploader has initially picked a thumbnail. Maybe a "We have some new thumbnail suggestions for you, take a look" alert or message?
So, in an article about image processing, why not include nice big beautiful images, that get even bigger when you click on them?<p>I click on the low detail inline images, and they stay the same disappointing size and reveal no further detail.<p>They're all, like 600px X 200px? Am I being greedy for want of gigantic images, upwards of 3000px wide?<p>I suppose it <i>is</i> an article about thumbnails, after all, so maybe I shouldn't be so surprised.
Seeing this run through an equivalent of the deep dream visualizer could be really interesting -- what _are_ people looking for in thumbnails? I'm having difficulty imagining what features would even be relevant in such a situation.
Meanwhile, I still can't edit a playlist while playing it.<p>edit: constructively put - there's simpler stuff to fix UX and match user patterns still isn't there?