Even if nobody has even tried yet, can we guess what a language generator's neural AI's training data might look like, or whether it would use symbolic logic instead, or even both?
> In the days when Sussman was a novice, Minsky once came to him as he sat hacking at the PDP-6.<p>> “What are you doing?”, asked Minsky.<p>> “I am training a randomly wired neural net to play Tic-Tac-Toe” Sussman replied.<p>> “Why is the net wired randomly?”, asked Minsky.<p>> “I do not want it to have any preconceptions of how to play”, Sussman said.<p>> Minsky then shut his eyes.<p>> “Why do you close your eyes?”, Sussman asked his teacher.<p>> “So that the room will be empty.”<p>> At that moment, Sussman was enlightened.<p><a href="http://www.catb.org/jargon/html/koans.html#id3141241" rel="nofollow">http://www.catb.org/jargon/html/koans.html#id3141241</a>
I pursued this question in grad school [1], leaning towards symbolic logic:<p>Suppose we want to learn a "natural" programming language. Training data would be example programs that we believe should be easy to express in any language. Since each of those programs will be expressed in a particular language, we'll need a notion of program equivalence across languages. As a toy framework let "language" mean a basis of combinators in pure lambda calculus; this is convenient because we have a (Hilbert-Post-)complete theory of behavioral equivalence among programs (H*, see Barendregt's classic book), and because the combinatory basis problem has been well studied since 1950s. Applying machine learning we can try to "fit a combinatory basis to data" in the sense of finding a finite weighted set of combinators, giving more weight to language primitives with shorter spelling. Our loss function will be the Kolmogorov complexity of our training data, actually gradients will be better if we use a softmax-Kolmogorov complexity. I used gradient descent to update weights of existing language primitives, and used greedy sparse dictionary learning to propose new language primitives. Most of the work was in proving equivalence and approximating Kolmogorov complexity.<p>It was a cute experiment, but hopelessly far from practical.<p>[1] <a href="http://fritzo.org/thesis.pdf" rel="nofollow">http://fritzo.org/thesis.pdf</a> (2009)
This seems to be displaying a common disconnect between the perceptions of "AI" and what modern AI actually is.<p>There is no "emergent" phenomenon currently in AI. I am thinking that the closest concept to what the author is asking for is a generative model, but even this does not really come close to performing tasks like "discovering" a new programming language. And it certainly would not discover an "emergent" anything. If it did "discover" anything it would be an object very statistically similar to the objects used to construct the model in the first place (e.g., deep fakes).<p>Everything in current AI is data first (read: data only). You can sometimes synthesize data and use that, but you always start with real-world data, and the models you end up with are parameterizations of the data you trained with. Always. So, while there is a future where a technical answer can be supplied to this question appropriately ... We are as of yet not in that future.<p>I consider this a technically valid answer and not a dismissal of the author's question, btw.
You win Risk by starting in Australia. What's a smaller problem with the same properties? User interface.<p>Our phones should learn a private language with us. My dog learns after one repetition; Zoom should learn to arrange my windows as I like, at least after 47 repetitions.<p>For the microdose-at-work developers, the trippiest arena would be in the visual realm, where our brains aren't so crippled by convention. Make a tablet/pen drawing app that develops a common language with the user.
Not to generate a compiler (very hard IMO) but to generate something that compiles and runs.<p>My next thought is to use a language agnostic spec and train an AI to create programs that adheres to that spec. OpenAPI for RESTful interfaces would be good.<p>IMO, using A.I. to generate programs (or languages) is the ultimate nerd snipe. I, sovietmudkip, have to admit defeat else my own productivity and happiness will suffer.<p>There is someone else out there whose personal incentives align with engaging in this difficult endeavor. But for me, I measure myself in projects realized, and that holds me back sometimes.
This is an interesting question. I think the first thing you need to decide is what will compile the language.<p>If you decide the language generator itself should be doing the compiling and processing, then congratulations, you are interested in the GPT-3 self-prompting community, and should go read everything Gwern wrote about GPT-3, then join the openAI GPT-3 slack. The short answer is that text transformers output a vector of likely ‘next tokens’ based on a token input. You can choose from this output vector using whatever rule you like, and feed it back in as input.<p>If you’re wondering how an AI might ‘talk to itself’ and program its own behavior internally, then you might like papers like this as a starting point: <a href="https://www.sciencedirect.com/science/article/pii/S1051200417302385" rel="nofollow">https://www.sciencedirect.com/science/article/pii/S105120041...</a>. Short answer: because of how they are wired, NNs tend to have activation ‘areas’ as they process things, and this is represented as the connections between and weights of data flowing through very large matrices; not a thing that’s super easy for humans to interpret as ‘language’.<p>I believe OpenAI also is publishing more on interpreting how AIs work and think behind the scenes, so you may want to check their blog / published papers.
I know very little about AI. But I think one could try to get AI to write an existing language.<p>Let me explain: Python, for example, is a language specification which has different implementations in different languages, most popular being CPython (commonly referred to as Python).<p>So if you fed the existing implementations of python to an AI and asked it to create a new one in some other language - I think this could be very well be a fruitful experiment.
Area's of consideration would be in teaching the model what a human wants from a language. An AI might see that not everybody speaks one language and lean towards a symbol based language that transcends any individual human language bias.<p>Then how would you train that - show it what computer languages have come before, would it weight it upon how well they are used to get an idea of what humans prefer - the prospects of some java-C-Cobol mutant language do seem a logical output perhaps.<p>Area's in which I'd like to see AI focus would be - optimisation and code auditing. Which may well prove easier as you have a solid defined goal. Also such endeavours would prove invaluable down the line if you wanted to have AI come up with a programming language. As to do that you would need to train how humans communicate/think and how computers communicate/think and meet in the middle.<p>[EDIT - spelongs]
A question/challenge that I was hoping would be posed:
"but what would be the point?"
Here are my answers:
(1) To see if it was possible to make a programming language that was 'better' (in terms of performance, flexibility, readability, debug-ability, ease of learning, or any other metric) that any existing programming language?
(2) to see if AI would or could create a language that was profoundly different to any existing 'human created' programming language
(3) to see if this kind of exercise could give us any new insights into aspects of the nature of programming languages
(4) to see if it was possible to use such an exercise to improve upon or refine existing 'language development tools' for creating domain specific Languages (DSLs)
Hmmm interesting question. Maybe you could you pass in thousands of code-bases and train an AI to <i>figure out</i> a common General Purpose programming language...<p>Not sure it could be done, but it would be interesting to see if an AI could come up with new abstractions that it sees in many different examples... Maybe like a new type of iteration or assignment
Facebook's TransCoder[1] does github-trained code translation between languages, suggesting the possibility of a polyglot or manticore language, cobbled together from existing languages.<p>[1] <a href="https://news.ycombinator.com/item?id=23463812" rel="nofollow">https://news.ycombinator.com/item?id=23463812</a>
Yes I am writing a language which can be used to construct computer programs based on pattern matching function inputs and outputs to form chains of computation. Still in the early stages and the A.I. component hasn’t been proved yet , even though early versions are in use at large tech companies
Can AI be instead used to program existing languages and develop applications? At least develop something more than a skeleton template and able to accommodate various requirements.
I think that this is question is in the direction of Stephen Wolfram's approach to AI. Namely: build a programming language that is capable of solving all kinds of problems (Wolfram Language), and then you will have condensed all the knowledge of the world into a programable structure with which you can more easily build AGI. They have the added benefit of collecting billions of user queries with Wolfram Alpha, which might serve as training data.
I’ve always fantasized about the eval() function in JS, which means if you feed it the correct random string, it could modify itself (which even the human brain can’t do). The AI version of Bogosort ;)<p>If you try to generate language artifacts, maybe you need to gather the sources of major projects, which large companies can do easily with their internal codebase.
take code from rosseta and do that google stuff with translating between two natural languages, networks generated third language<p>why you want yet another language?<p>although it's interesting stuff, maybe you make softdev easier :)
take code from rosseta and do the google thing with translating between two languages, where networks created third language as a bridge - thats your language<p>why you want tohave yet another language? ;)<p>its quite interesting :)