Trellis – 3D mesh generative model

Wow, that&#x27;s amazing.. I think this is the first time I&#x27;ve felt sick to my stomach watching AI generated content.. A sadness rush over me. Because these look so good, like every soul-less super-high quality shovel-ware asset ever made.<p>I&#x27;m not putting it down, it truly is an amazing achievement, and it feels like it marks the end of hand-made assets. I don&#x27;t even feel bad for the artists, I just feel bad for myself, because I want things made by people, for the inherent property that they were made by people. This is the same reason I don&#x27;t care for procedurally generated games, I want to walk worlds that arose in the minds of others, not just worlds, if I wanted a procedually generated world that just exist for no reason in particular, I&#x27;d walk outside..<p>I don&#x27;t want content, I don&#x27;t just want stuff to sift through, I want pieces of art, made by my peers with their own visions, ideas, values, insights and personalities. They don&#x27;t need to look that good, they just need to have been made with purpose.

Wow, these look amazing. I&#x27;m a layman, but I think this is what everyone&#x27;s been thinking about ever since the first NeRF demos?<p>EDIT: I went looking for those threads and found my own comment wishing for this 5 years ago <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22642628">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22642628</a><p>The next step is to automatically add &quot;nodes&quot; to the 3D images where the model can pivot, rotate and whatnot and then boom, you have on-demand animated, interactive content.<p>Feed it some childhood photos and recreate your memories. Add an audio sample from a loved one and have them speak to you. Drop into VR with noise-cancelling headphones for extra immersion. Coming soon! Click here to join the &quot;Surrender Reality&quot; waitlist

It&#x27;s not perfect, but significantly better than most that I&#x27;ve tried. Every time I&#x27;ve tried a 3d model generator up to this point, the result was unbelievably bad. This time it was medium good. All, give me a file format I can drop right into Orca Slicer.

I&#x27;m impressed. I used layer diffusion to make this low poly airship: <a href="https:&#x2F;&#x2F;image.non.io&#x2F;b3f843be-b1b4-468a-a0ec-9d58b191beee.webp" rel="nofollow">https:&#x2F;&#x2F;image.non.io&#x2F;b3f843be-b1b4-468a-a0ec-9d58b191beee.we...</a><p>Which resulted in this: <a href="https:&#x2F;&#x2F;video.non.io&#x2F;video-2732101706.mp4" rel="nofollow">https:&#x2F;&#x2F;video.non.io&#x2F;video-2732101706.mp4</a><p>Honestly not bad at all. Getting to the point of being able to use this as game assets.

I tried an image of a F-117 stealth bomber from wikipedia. The output was a complete fail, to the point where I have no idea how they managed to generate the examples on their project page. The basic silhouette was completely inaccurate.<p>I was hoping you could upload several images from different angles to help it, but that doesn&#x27;t appear to be a feature.

Saw this submitted a few days ago [0], but it&#x27;s a very impressive demo and would like to see it get discussed here.<p>[0] <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=42342557">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=42342557</a>

I can see the potential, but the images I give it must be very far outside of its training because all it generates are weird flat planes

It really depends on the image, but WOW I was really surprised that it reproduced animal fur with proper combination of polygon mesh and transparent texture, and this kind of capability isn&#x27;t even demonstrated in the examples on the page.<p><a href="https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;qJp4HNX" rel="nofollow">https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;qJp4HNX</a>

It&#x27;s interesting how Alphafold... which is a 3d generative model from 1d protein sequences is all fancy and complicated with its internal data representation in comparison to this paper which basically just voxelizes the input data and takes a bunch of pictures from various angles to build its training set.

I was able to use this to go from an AI generated image to 3d print. Steps documented here: <a href="https:&#x2F;&#x2F;x.com&#x2F;ryanlanciaux&#x2F;status&#x2F;1866163343788007619" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;ryanlanciaux&#x2F;status&#x2F;1866163343788007619</a>

Worked pretty terrible at trying to model the nix snowflake.<p>Guess its more trained on natural and biologic structures and textrues, rather than more structural or symetric data.

Looks like the generated wireframes are still terrible and would need to be redone by hand. So no need for artists to worry about their job security just yet.

Anyone see anything like this for GenAI for BREP CAD?<p>This is using meshes which seem a lot easier, and I&#x27;m unsure who&#x27;s working on BREP or at least models that have some feel for dimensions and physics.<p>ie It could still be meshes, but the designs would be physically reasonable like how a skilled tradesperson may make a device with their intuition without running FEA, etc.

Hey this is actually really good, one of the best image to 3D models I&#x27;ve seen so far. From an image of a bollard I was able to generate a pretty good mesh. The GLB as generated was 1.2MB, after compression I got it down to 35kb (!) and the fidelity is good enough for most projects.

The ability to edit parts of the model after the fact using prompts is pretty incredible

Nice work from Microsoft Research, including it being open source, and with open sourcing of the training dataset to come (which I suspect will actually be the most valuable contribution long term)

Tried it with a few simple images - not trying to deliberately confuse it. Seems to handle angular objects better (buildings, ships, etc.) as opposed to something more biological (lot of animal failures).<p>I haven&#x27;t cracked any of these open in Blender to look at the topology so they could be (and probably are) an absolute mess, so it&#x27;s likely that attempting to make changes to the model may prove to be more trouble than it&#x27;s worth, but still neat to play around with.<p>Images &#x2F; Resulting models:<p><a href="https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;eSWJhMj" rel="nofollow">https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;eSWJhMj</a>

Is there a demo of its &quot;Text to 3D Asset&quot; capabilities somewhere?

Very cool. I wonder if the fact that it featurizes 3d objects using voxels can be memory intensive considering only the surface voxels are actually relevant for 3d structure makin&#x27;

If this can be improved to accept multiple photos from different angles, it could make a great method for &quot;pseudo-photogrammetry&quot;.

This is incredible. I uploaded a picture of a cable with plug and it generated a mesh with individual wires and a plug with proper holes

Pretty amazing. Scale consistency is an issue looking at the scene examples, but for one-off or background assets, pretty neat!

hmm. Been using this fascinated for the past few hours, work-shopping workflows. The biggest oversight I see so far is merging points by distance. its the first step I have to do every time I bring it into blender

This is great! Love it, 3D visualising can really benefit from the use of Generative AI.

Think the server is overloaded, I keep getting an error

Wow those mech examples are pretty boss lookin

It looks very nice in the examples, but I tested it by dropping a Chrono Trigger character art in, and it generated some 2D planes instead of a 3D mesh..so ugh.. what?

Now we just need a physical VR environment where we can upload these and play with, similar to Genie 2 but mesh based(?).

Um, wowowow. This is a huge leap forward in 3D Asset generation. I&#x27;m downloading it right now. This feels like a combination of ergonomics - pulling together multiple somewhat janky workflows - better training data - and solid architecture innovation. Thanks Microsoft, seriously. I&#x27;m looking forward to playing with this.