> To improve the stability of the resulting designs, we employ an efficient validity check and physics-aware rollback during autoregressive inference, which prunes infeasible token predictions using physics laws and assembly constraints.<p>I'm far from an AI expert, but I've long felt that this is one of the most interesting ways to use AI: to generate and optimize possibilities <i>within</i> a set of domain-specific constraints that are programmed manually.<p>For example, imagine an AI that is designed to optimize traffic light patterns. You want a hard constraint that no intersection gives a combination of green lights that could cause collisions. But within that set of constraints, which you could manually specify, the AI could go wild trying whatever ideas it can come up with.<p>At that point, the interesting work is deciding how to design the problem space and the set of constraints. In this case it's a set of lego bricks and how they can be built (and be stable).
This is probably going to get a letter from LEGO's lawyers.<p>If you want to be safe do not use the word LEGO. Use Bricks or in German "Klemmbausteine".<p>Many people have had to deal with LEGO's lawyers and it ain't pretty.
This does not seem like a very impressive result. It's using such a small set of bricks and the results don't really look much like the intended thing.<p>It feels like a hand-crafted algorithm would get a much better result.
There’s a bug on the page (on iPhone, at least) once you scroll to the gifs that it starts to auto load them without doing anything, making it really hard to navigate anywhere at that point.
I don’t need automation to build LEGO sets — that’s the fun part, and I want to do it myself. What I need is automation <i>after</i> the build: to clean up, sort the bricks by color and shape, and store them properly.<p>I just wish scientists would start by solving problems that actually exist in the real world. There’s real value — and real money — in that.
This is super cool! The GIFs showing the object being built are just yummy; I have no other way to describe it.<p>If anyone else was searching for the dataset, it is at <a href="https://huggingface.co/datasets/AvaLovelace/StableText2Lego" rel="nofollow">https://huggingface.co/datasets/AvaLovelace/StableText2Lego</a><p>It contains " contains 47,000+ different LEGO structures, covering 28,000+ unique 3D objects from 21 common object categories of the ShapeNetCore dataset".<p>Local inference instructions are over at their github page - <a href="https://github.com/AvaLovelace1/LegoGPT/?tab=readme-ov-file">https://github.com/AvaLovelace1/LegoGPT/?tab=readme-ov-file</a>
I noticed that "a basic sofa" involves some placing some floating bricks if built in the order of the animation. It hints at the way this model generates the designs. The automated assembly of generated LEGO structures using robots would have serious trouble creating these designs I reckon.
It's hilarious watching $50,000 worth of robots take so long to assemble a couple dollars worth of Lego. It's like peering into the old folks home for robots.
Doesn’t seem to add much to just converting a 3d model into voxels and therefore bricks.<p>Using bricks other than 2x2 and 2x4 blocks creatively to make interesting things is really important, i’m not sure what type if algorithm would best auto generate beautiful MOCs however? Was thinking of doing a $50000 kaggle comp for this, what do others think?
Quit trying to read the article after the 15th video went to full screen and had to be dismissed hitting the tiny x in the upper left… 3 more interfered with me trying to go back to this page
Have the authors never heard of Lego being one of the companies that are super strict about their trademark? They file takedown notices etc on every project they see. Even if the stone design has the little thingies on top/bottom...
The high backed chair gif example is interesting - the way it’s animated it would completely fall apart and be unstable. But if you built it in reverse, it would work fine.<p>But it also shows the weirdness of the solution - in places where larger bricks make sense, multiple smaller bricks are used instead. In a section where a 2x6 should be repeated, in on instance of the repetition it uses tow 1x6s. It’s weird.<p>Cool idea.
Great. Please do cabinets next. Constrain to some specified material such as 2.5m by 1.25m 18mm ply. Iterate designs by text and output the model, cutlist and assembly instructions. Simple right?
When I was a kid, I proudly exclaimed I wanted to become a professional lego builder. Not in my wildest dreams would I have assumed how close to that career path I could have come.
Cool project, but judging from the videos, it looks like some of them can't actually be built using those instructions. E.g. "A backless bench with armrest" would require some bricks to float in the air with no support while you're assembling the rest.
Looks amazing, with the arms building the piece!<p>However, the model "A high-backed chair" has some floating pieces in the middle of the seat, that are fastened from above. Can these robots handle building these?
So, besides training a LLM to generate build instructions for lego model, they have robots to assemble these models, and they applied 3D texture on 3D generated model (what for?).<p>Sometimes the amount of money and energy that are spent in "recreation" projects just amazes me.
Is this use of the trademark approved by the LEGO owners by any chance? I skimmed it and there didn’t seem to be an indication this has been endorsed by the company.<p>It kind of makes me suspicious of the integrity of Carnegie Mellon if they will allow trademark infringement of this type because, well, it does make me feel like I can shit in a bag and call it a Carnegie Mellon Socking Stuffer without consequence.