Apple releases Depth Pro, an AI model that rewrites the rules of 3D vision

<a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=41738022">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=41738022</a>

Title is such clickbait, it does not rewrite the rules of 3d vision, it is a marginal improvement on existing models, and does not work for video, only images. However, Apple open sourced the model weights, which is amazing for research.

This article has a link to the live demo.<p><a href="https:&#x2F;&#x2F;huggingface.co&#x2F;spaces&#x2F;akhaliq&#x2F;depth-pro" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;spaces&#x2F;akhaliq&#x2F;depth-pro</a><p>For some pictures it outputs something reasonable, for others it&#x27;s completely broken (black with colored noise in one area).

Just tried it on a &quot;difficult&quot; image (relatively low contrast photo of a small thin plant in front of a tree trunk with a distant fence in one corner) and it did a pretty good job, I think - <a href="https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;Sqr6hR8" rel="nofollow">https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;Sqr6hR8</a> including the depth maps.

This presumably is the same model that the Vision Pro Photos app uses to convert 2D photos to 3D.

Was this trained on iPhone photos since there is a decent amount of depth references within iPhone cameras? It’s interesting to see how clearly it understands depth of field. With that, how does it perform on F16 and above?

interesting. They claim 0.3 seconds on a consumer GPU; I thought that might scale to 30 seconds or so on CPU but gave up waiting after twelve minutes.

Can I use this to generate accurate depth maps from 2-D images that I can then CNC or 3-D print?