Differentiable Signed Distance Function Rendering

This reminds me of phase-problem from protein crystallograpy (PX). I&#x27;m 15+ years away from that space, so I have no idea what the modern solution space looks like, but it seems like there&#x27;s a corollary between reverse-AD-over-SDFs (in this work) and the MLE methods for inverse space in PX. It <i>feels like</i> we should be able to take an initial estimate of some SDF over the direct space (2d object) as an ellipsoid, flip to inverse space, do an MLE for phase improvement, and then just do the regular ping-pong on that. The MLE (and successor?) methods are <i>really</i> robust.

Very cool. Presumably the advantage of this is that it can solve big flat areas like the back of the chair that traditional methods never work well with.<p>But am I understanding correctly that it needs known lighting conditions? Presumably that&#x27;s why they don&#x27;t demo it on real images...

I&#x27;ve noticed a lot of interest in differential rendering&#x2F;reverse rendering recently<p>Does anyone know what the end goal of this kind of research is or why there is so much interest?<p>Its definitely cool but is the idea just to make photogrammetry cheaper&#x2F;easier?<p>Or are there other use cases I&#x27;m missing

It&#x27;s funny how this totally could have been implemented in like the late 90s except I guess no one did and yet we now see how it&#x27;s totally able to solve lots of &#x27;inverse problems&#x27; that 3d reconstruction algorithms have tried to solve for decades.

Why use SDFs instead of NeRFs? Those were designed to be differentiable. Then you could turn the NeRF to an SDF later. Related: <a href="https:&#x2F;&#x2F;nvlabs.github.io&#x2F;instant-ngp&#x2F;" rel="nofollow">https:&#x2F;&#x2F;nvlabs.github.io&#x2F;instant-ngp&#x2F;</a>

I wonder if anyone is working on combining these techniques with traditional structured light field or lidar geometry capture pipelines. Seems like a fantastic way of both speeding up and improving capture quality.

I wonder what pcwalton and raphlinus thinks about this. Pathfinder was a SDF SVG renderer after all.

I have a problem where I need to do surface&#x2F;volume estimation of a bunch of objects, but they are moving across the camera with a static background. Anybody know how to do that in real time?

This is basically Nerf but without support for transparent &#x2F; translucent &#x2F; subsurface &#x2F; glossy &#x2F; refractive objects.

Is this a gamechanger?

What is the problem they are solving? My best guess is: A 3d model is rendered and overlaid a number of background images. From these compositions, reconstruct the original model. Is that it?