So, basically, this is the thing in a crime detective movie where the forensic analyst is looking at a terrible pixelated surveillance camera still and says "enhance," and the computer magically increases the resolution to reveal the culprit's face.<p>Just another entry on the "things that are supposed to be impossible that convolutional nets can do now."
I think it's always problematic to compare to images upscaled via nearest-neighbor. The big pixels are hard to parse for our brain, we detect all the blocky edges.<p>A good content unaware upscaling would be nice (one of the default photoshop algos)<p>I also wonder what they used for the downscaling. I see 4x4 pixel blocks, but also some with 3px or 7px lengths.<p>This looks pixely and is supposed to be a source file?: <a href="https://raw.githubusercontent.com/Tetrachrome/subpixel/d2e28518d2ce75a1ae7dde7727603b8652d683cb/images/lowres_input.png" rel="nofollow">https://raw.githubusercontent.com/Tetrachrome/subpixel/d2e28...</a>
Interesting image "upscale" algorithm.<p>I'm not familiar enough with the field to understand how the "neutral net" part feeds in, other than to do parallel computation on the x-pos, y-pos, (RGB) color-type-intensity tensor interpolated/weighted into a larger/finer tensor.<p>(linear algebra speak for upscaling my old DVD to HD, that sort of thing)<p>At the risk of exposing my ignorance, this has nothing to do with "AI", right? It's "just" parallel computation?
It seems that this subpixel convolution layer is equivalent to what is known in the neural net community as the "deconvolution layer" but it is much more memory and computation efficient. The interlacing rainbow picture was a bit hard to understand until I read this <a href="https://export.arxiv.org/ftp/arxiv/papers/1609/1609.07009.pdf" rel="nofollow">https://export.arxiv.org/ftp/arxiv/papers/1609/1609.07009.pd...</a>
Interesting. They should post more examples (not with just faces), or make an online demo, like waifu2x [1]<p>[1] <a href="http://waifu2x.udp.jp/" rel="nofollow">http://waifu2x.udp.jp/</a>
I'm not sure, but there seems to be something wonky in the input images. They are very blocky, so I thought that they would be just pixel doubled (or quadrupled) from low-res pictures, but the blockiness lacks the regularity I'd expect from pixel-doubled images.<p>How were the input images prepared?
The explanation in the README of the github project is excellent and well-written! Here's a really great set of animations by Vincent Dumoulin on how various conv operators work: <a href="https://github.com/vdumoulin/conv_arithmetic" rel="nofollow">https://github.com/vdumoulin/conv_arithmetic</a>
This is impressive! But, I'll be really impressed once this 'new thing' brings us roto masks in motion. That is, isolating objects from background on a movie with pixel-perfect accuracy. It will also make a lot of people out of job and a lot of people happy at the same time.
The problem with subpixel images is that there are RBG and GBR monitors. Not only that, there are horizontal and vertical variations. And there's no way to tell which one the user is using on the web. And that's not even counting all the mobile number like pentile.<p>It's still useful though, browsers, for instance, could use it for displaying downscaled images.