Shape of Motion: 4D Reconstruction from a Single Video

For 3D VR videos, this would be useful for adjusting IPD for every person, rather than use the static IPD of the camera setup. Also, allowing just a little bit of head movement would really increase the immersiveness. I don't need to travel long distances inside the video. If the video is already filmed with static stereo setup, it would be even easier to reconstruct an accurate 4D video limited to short travel distances without glaring errors.

One thing I liked about Team Ico (a studio behind the Shadow of the Colossus, Ico, Last Guardian video games) was how the player can move the camera just a little but during automated sequences<p>Getting that kind of look around in a video scene would be really engaging. A bit different than VR or watching in The Sphere, with the engagement being that there are still things right out of view you have to pan the camera for

Our children will be so weird out by blade runner. Not by the zoom into the picture, but by the fact that the guy believes in halucinated data.

Whenever I play a video game (<i>Monster Hunter World</i> comes to me immediately) and see an establishing shot with moving camera (like the ones demoed on their web site) I think the game really wants to run in an a VR headset where you can walk around and see different angles.<p>(Funny there is a VR mod for <i>Monster Hunter Rise</i> which makes me think just how fun <i>Monster Hunter VR</i> would be)

I was wondering how were they getting depth from a video where camera is still.<p>&gt; we utilize a comprehensive set of data-driven priors, including monocular depth maps<p>&gt; Our method relies on off-the-shelf methods, e.g., mono-depth estimation, which can be incorrect.

This reminds me of the description of Disneys(future movies) in Cloud Atlas. The movie had a good visualization, this feels like that.

Curiosity, what is the difference between 4D or 6DoF (six degrees of freedom)? Sounds a lot like the 6DoF work that Lytro did back in 2012, although this obviously is coming at the problem from the other direction, generating it rather than capturing it.

The results are impressive, but what makes this 4D? Where’s the extra dimension and how is it relevant to 3D human beings?

the first HyperNeRF cat video is quite interesting-looking and surreal!

Looks flat to me!