Ray Tracing Denoising

It&#x27;s only briefly mentioned, but NVIDIA&#x27;s recurrent denoising autoencoder is pretty amazing.<p><a href="https:&#x2F;&#x2F;research.nvidia.com&#x2F;publication&#x2F;interactive-reconstruction-monte-carlo-image-sequences-using-recurrent-denoising" rel="nofollow">https:&#x2F;&#x2F;research.nvidia.com&#x2F;publication&#x2F;interactive-reconstr...</a><p>Two Minute Papers did a nice explanation of it:<p><a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=YjjTPV2pXY0" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=YjjTPV2pXY0</a>

Aw this didn&#x27;t mention orthogonal array sampling, another sampling technique aimed specifically at ensuring good distribution in higher dimensions: <a href="https:&#x2F;&#x2F;cs.dartmouth.edu&#x2F;~wjarosz&#x2F;publications&#x2F;jarosz19orthogonal.html" rel="nofollow">https:&#x2F;&#x2F;cs.dartmouth.edu&#x2F;~wjarosz&#x2F;publications&#x2F;jarosz19ortho...</a>

Can&#x27;t wait until ray tracing is more mainstream. I know NVIDIA has their share of uncool things but I really appreciate them making the RTX series, hopefully we&#x27;ll see more and more of this.

At one time, writing a ray tracer was a right of passage when getting into computer graphics. I never did it, but this week I was trying to figure out how to find the intersection point of a ray and an oblique cone and all of the resources I found were related to ray tracers.<p>I&#x27;m thinking that maybe I should actually write a ray tracer. Is that still a worthwhile thing to do, or has the world moved on?<p>FWIW, I never solved the ray-oblique cone problem...

For a practical demonstration of what denoising can theoretically do I recommend Quake II RTX. There is a dev mode with a bunch of rendering settings, many more so than in an average game. They were added to it because this game is more of a tech demo of RTX at this point.<p>There is a switch where you can turn the denoising of the ray-traced output on&#x2F;off: it shows tremendous difference, to a point where it&#x27;s hard to even imagine looking at the noisy image that it is even possible to extract the denoised version.

Something I&#x27;ve wondered is whether technology like this could eventually be self-defeating for hardware manufacturers. Rather than the evolution of graphics deriving from improved accuracy of the optical simulation fuelled by advances in computational power, it may instead derive from optimising subjective video quality, similarly to video codecs.<p>While accurately simulating optics is needfully computationally expensive and gives special-purpose graphics hardware an advantage, it&#x27;s not clear that psychologically subjective high quality graphics (i.e. generating visuals which are inaccurate but convincing to humans) has such a need.

So far, what I&#x27;ve read about denoising is always the context of doing image post-processing, but it seems to me that some of these techniques could be used just as well to identify areas of the image that the denoiser is most uncertain about, so that you can trace more rays in those directions.

What&#x27;s fascinating to me about this is that it sounds like future renderers may end up working very much like we think the brain does. There is a virtual world, but very little raw data about that world is used directly, just a small sample, and the rest of the image is filled in by a neural model that is able to infer how the whole scene should look based its a priori understanding of how things like light and depth work.

Great post but disappointingly few images. It would have been really interesting to see these techniques applied on a standard scene with before&#x2F;after comparisons.

Shirley published a paper in 1991 showing that low discrepancy samplers worked well in a ray-tracing context. So I wouldn&#x27;t say that&#x27;s particularly new.