PixelNN – Example-Based Image Synthesis

I used to roll my eyes at crime television shows, whenever they said &quot;Enhance&quot; for a low quality image.<p>Now it seems the possibility of that becoming realistic are increasing with a steady clip, based on this paper and other enhancement techniques I&#x27;ve seen posted here.

To paraphrase Google Brain&#x27;s Vincent Vanhoucke, this appears to be another example where using context prediction from neighboring values outperforms an autoencoder approach.<p>If 2017 was the year of GANs, 2018 will be the year context prediction.

I hope some day this will generalize to video. I don&#x27;t care about the exact shape of background trees in an action movie - with this approach, they could be compressed to just a few bytes, regardless of resolution.

I wonder if this could be applied to &quot;incomplete&quot; 3D models and the work shifted to the GPU!?

I don&#x27;t understand how the edges-to-faces can possibly work. The inputs seem to be black &amp; white, and yet the output pictures have light skin tones.<p>How can their algorithm work out the skin tone from a colourless image. Perhaps their training data only had white people in it?

Seems to have a thing for beards.

I have a large collection of images, many being accessible through google image search.<p>I wonder if there could be a way to &quot;index&quot; those images so I can find them back without storing the whole image, using some type of clever image histogram or hashing-kind function.<p>I wonder if that thing already exist, since there are many images, and since most images have a lot of difference in their data, could it be possible to create some kind of function that describe an image in a way that entering such histogram redirects to (or the closest) the image it indexed? I guess I&#x27;m lacking the math, but it sounds like some &quot;averaging&quot; hashing function.

Is anyone in the FX business playing with this stuff? I&#x27;m thinking generational backdrops with groups of people&#x2F;stuff&#x2F;animals in them without a lot of modelling input.

So is there an analagous process that would apply to audio I wonder?

This is amazing. I especially like how the result can somewhat be interpreted by showing from what image the part of the generated image is copied (see Figure 5).

Apparently you grow a beard after using their nn model?

No code available.

Very clever. I wonder if something like this could be used for other forms of sensor data as well?

A pair of the inputs in the edge-to-edge faces are swapped. I have nagged an author.

All those examples are fairly low-resolution. Does this approach scale or can it be applied in some tiled fashion? Or would the artifacts get worse for larger images?

Does anything like this exist on the web would like to send a blurry license plate picture through this and see what it comes up with..

OMG, now the &quot;enhance&quot; they say in investigative TV series and movies will actually be reality! :p

This is cool, but in the comparison with Pix-to-Pix, it seems like Pix-to-Pix is the clear winner.

&quot;Enhance&quot; is real. When will this stuff trickle into lower level law enforcement?

Can we use this to identify the leprechaun and find where da gold at?

Is there a GitHub repository link?

I found the title somewhat misleading. I was expecting some clever application of the nearest-neighbor interpolation. But this seems to involve neural nets and appears far from &quot;simple&quot; to me (I&#x27;m not in the image processing field though).

It almost looks like they mixed training and testing data in some of the examples. The bottom-left sample in the normals-to-faces is extremely suspicions.

Enhance.

Hollywood got it right!!!