As someone working in the field I have a few thoughts:<p>1. The way I think about these huge models is no longer as something that makes predictions, but rather as a kind of huge knowledge base (compressed training data as representations) with smart query capabilities, which come from the fact that the model is constrained to create syntactically correct output. This could be the next generation of search. There is a fine line between search and creating novel outputs, since most novelty is just a combination of old things with constraints on correct syntax.<p>2. The shown results are likely a result of massive cherry-picking in the typical "pitch deck demo" fashion. You'll see a lot more failures when trying to use this and you need to get the query "just right" for it to produce good output, which a skill in itself. Also let's not forget that the model must be fed fine-tuning examples, which also need to be "just right.". That's why I believe these kinds of models aren't that useful for fully automating things - for something to be used in production it should be 100% correct - but they are very useful as a search function and alternative to e.g. StackOverflow in this case. Querying the model will give you good enough results that you can use as a starting point for your own use case.
This is a really cool demo, and congrats to the creator for making something really cool.<p>However, while these systems based on GPT-3, and similar techniques, can produce some amazing outputs given the right inputs, you don't have to poke them very hard before they start falling apart.<p>At the end of the day, this isn't really any more "clever" than a very, very advanced and long markov chain, there isn't any intelligence.
What I find most interesting about this is that is has just as much trouble as humans in translating requirements into code. Admittedly it makes some fairly basic mistakes as well, but always in the under specified areas<p>i.e.:<p>You didn't say what colour 'welcome to my news letter' should be, so I picked white.
The numbers from 1-5 are totally a possible random permutation in the 1-10 range.
I find the crying laughing emoji to be really ugly, so it's the worst one.
My actual worry about this is 'will I be obsolete as a front-end engineer'. It seems that AI will manage web optimization, and NLP conversational programming, so basically there is no need for a front end engineer. Perhaps I am wrong.
I was one of the people who didn't think that transformers would be powerful enough to automate software engineers out of a job.<p>I was wrong. It's coming for front-end engineers and it's coming soon.