I've been dabbling in using Fourier analysis in deep learning lately, and I'm surprised it that I haven't turned up very much research in this area (Fourier Neural Operators being what seems to be the biggest exception). Fourier analysis is such a ubiquitous tool, intuitively I'd think it would work great for deep learning. My suspicion has been that complex numbers are difficult to work with, and maybe I'm just bad at surfacing the relevant research, but I'd be interested to hear from those better informed. (My naive approach has been to simply concatenate the real and complex components together into an n+1 dimensional tensor, but surely there's a way that better respects the structure of complex numbers.)
Might be useful to use gabor filters as the basis function, since just 2d cosine filters doesn't produce particularly sparse output for angled features. The additional overcompleteness would probably be helpful for the NN learning.
I don't know why but I get this uncanny feeling when looking at the restored images. Maybe it's because I know it is restored, I wonder if I'd feel the same way if I find it in the wilds.
Wait, all my eye-rolling at the TV/film trope of "Computer, Enhance!" de-blurring is now redundant, and that stuff <i>is real</i>?!<p>This looks incredibly impressive as a result, but I'm wary of the use of metrics like FID to evaluate performance. I can take a high-res image, downsample it, then use the method and measure performance very easily: what percentage of pixels were correctly restored? Instead they're using metrics like FID which - while useful for purely generative techniques - seem a little vague for this purpose.