There is an interesting problem being worked on with Gaussian Splats for use in VR stereo rendering [1]<p>>Gaussian splats are 3d, but currently rendered as 2d cards ordered on centre, not per pixel like they should be.<p>>So as centres reorder, they "shuffle". This is a showstopper for general adoption in VR<p>>The solution? Ordering per pixel will solve this, as would stochastic methods which write depth correctly (though those have other problems for VR).<p>>For it to be viable you need to cull much more aggressively but also can't have any warp divergence when processing samples.<p>>Unforunately those things are in direct conflict. Paper's renderer is at extreme end of "weak cull, perfect warp convergence".<p>Apparently need some new Order-Independent Transparency algorithms that handle depth layers.<p>[1] <a href="https://x.com/charshenton/status/1710207169407447396?s=20" rel="nofollow noreferrer">https://x.com/charshenton/status/1710207169407447396?s=20</a>
Aras has also done some very interesting blog posts regarding this technique:<p>* <a href="https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pretty-cool/" rel="nofollow noreferrer">https://aras-p.info/blog/2023/09/05/Gaussian-Splatting-is-pr...</a><p>* <a href="https://aras-p.info/blog/2023/09/13/Making-Gaussian-Splats-smaller/" rel="nofollow noreferrer">https://aras-p.info/blog/2023/09/13/Making-Gaussian-Splats-s...</a><p>* <a href="https://aras-p.info/blog/2023/09/27/Making-Gaussian-Splats-more-smaller/" rel="nofollow noreferrer">https://aras-p.info/blog/2023/09/27/Making-Gaussian-Splats-m...</a><p>Gsplat gets a mention:<p><pre><code> Aside: the excellent gsplat.tech by Jakub Červený (@jakub_c5y) seems to also be using some sort of VQ/Clustering for the data. Seriously, check it out, it’s probably be nicest gaussian splatting thing right now w.r.t. usability – very intuitive camera controls, nicely presented file sizes, and works on WebGL2. Craftsmanship!</code></pre>
If you want to see some impressive gaussian splat demos take a look here: <a href="https://gsplat.tech" rel="nofollow noreferrer">https://gsplat.tech</a>
This week I saw a YouTube video which left me impressed regarding Gaussian Splatting.<p><a href="https://www.youtube.com/watch?v=kShNYOuDnlI&t=224s">https://www.youtube.com/watch?v=kShNYOuDnlI&t=224s</a><p>I got it recommended because I watched the following video:<p><a href="https://www.youtube.com/watch?v=HVv_IQKlafQ&t=64s">https://www.youtube.com/watch?v=HVv_IQKlafQ&t=64s</a><p>Both are worth watching if one didn't know it exists.
One reason this implementation is exciting is that the code that accompanies the Gaussian Splatting paper requires companies wanting to use the tech commercially to pay for a commercial license. There are existing viewer implementations, but this seems like the first alternative that would let me do everything without depending on the Inria code. The code being nice and clean is a bonus too :)
Is there any method for animating scenes captured using this method? It would be really interesting to build a hyper realistic world in a game with moving foliage or being able to "use" an item and watch it turn. Would be great for Myst like games.