The example mockup image shown is really unimpressive. As usual, the difficulty is in implementing the last 20% of edge cases and design subtleties, not well-defined black-and-white block layouts like the given example. It would've been simpler to build a classical program to handle that example, and the result would be much better and more predictably structured.<p>It's like programming a self-driving car for a 2D game. Trivial. Now try taking that to tar roads. I don't know why this toy example is even news, it's nowhere close to the difficulty of real-world design mockups.<p>The AirBnB sketch2code example is a lot more impressive, but that's basically just handwriting recognition with a 1 to 1 mapping of symbols to code pieces.
Aren't there a bunch of WYSIWYG tools for creating appearance and HTML/CSS together? Years ago when I looked, these existed (but designers prefered photoshop just because).<p>I mean, I'd assume also there's normally a give-and-take between designer, CSS-artist and client. The question is whether the neural network can also learn to take calls at 3am from a client wanting a different shade of aqua.
Writing HTML and CSS to implement a mockup is trivial for anyone with even a little bit of web dev experience. If you want high quality code, an AI isn't going to produce that.<p>I can't imagine ever using something so complex, if I can implement the same thing in a few minutes.<p>There's also further considerations when dealing with the real world. For example, you need to be aware of how to handle different accessibility features. Designers rarely seem to care about things like accessibility, but it needs to be declared somewhere.
This is what we are all working for. If we can automate the stupid stuff we do then we can work on other interesting stuff and automate it too.<p>I don't understand why we aren't all working on automating everything that we do.<p>Every line of code you pulled out of the code mine today is finite and given a business logic rule engine it's outcome could have been generated.
What’s the point of the code isn’t human readable? You need to train the AI to write code that is easily modifiable with low dependencies. Otherwise the conversion isn’t very useful for anything unless your page is completely static
Honest question: is anyone else sensing some defensiveness here? The timeframe is open to debate but what's more controversial is whether html/css is near the frontier of solvable problems. After AlphaGo, my worldview is changed forever. Glad to be still being valued for something I started out of joy but in no way feel it's not disruptable. Sometimes I even wonder why it hasn't advanced further since it's quite well defined, digital data in/out.
This is really cool and I believe necessary work. We need to start thinking about how to automate programming and to free ourselves much further from the mundane. Trust me, there is still plenty of room for people to analyze requirements and come up with a high level solution to people's problems that doesn't require fiddling with float: right and various technical hacks.<p>That said I wonder if this is the right approach. At some point "AI" as used in this context is just a function mapping from an input domain to an output domain. The output domain in this case, "code" was designed for human readability (whether it succeeded is a whole different approach).<p>What would a programming language designed for output from an AI system look like? How could we optimize it to reduce the output domain size of the function the AI has to train to learn? How could we optimize it to make the problem more tractable for machines? I feel like there is an entire field of research here. Maybe it has already been studied and I am just late to the game.
Turning web design mockups into code with Deep Learning | <a href="https://news.ycombinator.com/item?id=16115353" rel="nofollow">https://news.ycombinator.com/item?id=16115353</a> (Jan 2018)<p>>chrisfosterelli: <i>This is a neural network that takes an image and predicts very simple blocks (like BODY, TEXT, BTN-GREEN in the bootstrap example) and then uses a map to convert them to well-formed HTML</i><p>>jamesjyu: <i>I've always wanted to do a contest with other frontend coders to see who could get closest to a complex layout—like the NYTimes—in one go.</i> >>janneklouman: <i>these types of contests exist! I went to one of these[1] maybe two years ago in Stockholm and I had a blast</i> [1] <a href="http://codeinthedark.com/" rel="nofollow">http://codeinthedark.com/</a><p>Pix2code: Generating Code from a GUI Screenshot | <a href="https://news.ycombinator.com/item?id=14416530" rel="nofollow">https://news.ycombinator.com/item?id=14416530</a> (May 2017)
I agree with the need, my issue is with reproduceability and generally things that are not defined by the sketch (behavior, different screen sizes, etc). If everything were fully defined, we could just write a compiler from the high level definition to the low level. The real problem is that static 2D images are not good representations of dynamically sized interactive documents.
Kind of related to this, what do people think frontend developers will be doing in five years, or, where will they migrate to as frontend (either via tools like this or like AirBnBs SketchToCode) becomes more automated?
what about the converse of this? what about producing a basic set of content and letting AI taxonomise or organise it and then create a design based on design models. It could work both ways. I guess ultimately the problem will be solved both ways and AI could create entire experiences based on specific domains. Also, for those claiming this will never work i would suggest looking at some of the amazing stuff being done with AI right now. This kind of problem, including optimising and making best usage of html/css is almost tailor made for AI.
Interesting outcome.<p>The final html code from the first example is not that bad. There's the usual problem of beginner coders of too much div wrapping that probably isn't really necessary. I'm curious if the system can also create the css. The css also suggests a beginner coder, such as an overuse of unnecessary clear classes because of the overuse of floats for layout. Or having extra class names for things that are easily handled by a parent class reference, such as a "last" class on the last li in the list. Although, it appears the css is just a template obtained online. More on that later.<p>The second example using bootstrap is a soft failure in my eyes. Although the html does render correctly in the browser, which is because browsers do their best to render crappy html, the code is rough. The main problem is it decided to render the head element as a header element. Compared to the first example I'm shocked that this is the generated output. The usage of bootstrap does pose an interesting thought in that the content section of the html is more precise than the first example.<p>My reaction to this is it's a decent try at generating a website based on very strict rules assuming that more than half of the website creation process still requires a human to complete. For example, I could see this working quite well if one were to design your mockups strictly be bootstrap, or a template, and provide that css beforehand. If the mockup is custom outside of the template/bootstrap css then it'll have to generate that css itself. Which I think I'm more curious if that's possible. Generating html is easy as you can establish ground rules of "use this series of nested elements for this situation" and so on. The examples provided could just as easily be created by a drag-and-drop system that allows a non-coder to build a basic website. For that matter, use a markdown to bootstrap converter and train your writers/editors on the bootstrap basics and off you go.<p>But it sure did look like a fun learning exercise. As a front end dev, I'm not worried over my future and would be curious to see where it goes.
Up next, an AI for creating paintings and art and Sistine Chapel ceilings.<p>No. This will never be a thing. It will be copycat pages, never anything unique.