Teaching ChatGPT to speak my son’s invented language

In context learning is hands down the biggest breakthrough of LLMs. The flexibility the model displays without updating weights is genuinely mind blowing, bordering on absurd especially if you&#x27;ve trained other kinds of models before.<p>See here - <a href="https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;w3DAYOi" rel="nofollow">https:&#x2F;&#x2F;imgur.com&#x2F;a&#x2F;w3DAYOi</a> from the paper - <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2211.09066" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;abs&#x2F;2211.09066</a><p>GPT 3.5&#x27;s (4 is much much better) addition accuracy tanks after 2 digits. However, by approaching arithmetic as an algorithm to be performed and taught similarly to how it&#x27;s done with people, you can supercharge accuracy to basically 100% for up to 13 digit addition and &gt;90% after.

Wow, they asked the model to self-evaluate and it just outright cheated:<p><pre><code> He has three cats. Proposed: h’io’ngkiltrikumrikumrikumri’nguuy Correct: h’io’ngkiltri’ngkumrikumri’nguuy Points: 1 Hypothesis: N&#x2F;A </code></pre> (Other comments observe that it accidentally compensated for this by getting the sum wrong, haha, d&#x27;oh)<p>I have had similar problems with trying to get ChatGPT to do nontrivial things, &quot;here are the rules for this game, do you understand this game, great, let&#x27;s play it.&quot; And then it&#x27;s like herding cats. &quot;No that&#x27;s wrong, the game pieces cannot leave the game board,&quot; &quot;Oh my apologies you are entirely correct, here is the revised board (proceeds to dump the exact same state of the game board that I told it was wrong).&quot; Eventually it will lie about its own capacities, &quot;As an AI language model I am incapable of selecting a move to play next&quot;... But you have done several already!!! This is literally the ONLY thing you have been doing right and now you refuse?<p>Some other prompts are more successful but it does seem to have a sing-song high school book review style that inclines it to be boring... Very uncanny valley.

Unrelated to the actual work here (which is interesting!), I just wanted to mention how much of an awesome impact you must be having on your son. You two seem like such a fun curious pair! I spent a lot of time tinkering away alone in my room as a kid and that sucked. The opportunities you’re giving him now to explore are going to inspire a lifetime of creativity.<p>I know you don’t need someone on the internet to tell you that, but I thought I’d give kudos where it’s due :)

&gt;All of these differences can make it surprising and challenging for someone with an Indo-European language background to learn and use Kłeti.<p>Ironically, Proto-Indo-European is believed to be far more complex than its modern descendants, as described by Wikipedia:<p>&gt;PIE is believed to have had an elaborate system of morphology that included inflectional suffixes (analogous to English child, child&#x27;s, children, children&#x27;s) as well as ablaut (vowel alterations, as preserved in English sing, sang, sung, song) and accent. PIE nominals and pronouns had a complex system of declension, and verbs similarly had a complex system of conjugation.<p>So maybe a PIE speaker would have an easier time with Kłeti than we :-)

I would like to see this expanded, I think it&#x27;s a bit unfair to assess its abilities with so few examples. My hypothesis is that a rosetta stone with a thousand examples with a vector database hooked up to it so you don&#x27;t hit the 32k token context limit would lead to much better performance.

What a wonderful story and a great family. Congratulations!<p>You inspired me to ask GPT to do it the other way around. I asked it to invent a successor to Esperanto, with influences from Slavic, Germanic, and Latin languages. It called its language Euroglossa and wrote a short story in it. Who can understand it?<p>Un tag, en froliko vilaž, un jun chico namen Tomas trovat un misterioz mapo v star bibliotek. Na mapo, skribet: &quot;Skarb de Tri Montes.&quot; Tomas decidet da sledit la mapo in aventuro, sperante da otkriti grand skarb.<p>Tomas paket svoi rukzak s neobkhodim stvari, inkluziv kompas, binaukli, i nutrimento. Nachet svoi putovanje, iz vilaž, do la Tri Montes. Po nekoliko dni, on prishel k bazen na pervoj monte. Tam, on otkril zagadka, ki je klyuch za dalsi koraki: &quot;Kogda solntse küsst la luna, dvigat kamen i vstretit un oko.&quot;<p>Tomas wartet geduldig bis la sonne küsst la luna in noktchielo. Kvando moment venit, er raskt dvigat un gros stein u otkryvajet secret passaž. Ingressante, on sledit un dunkel tunel, portant un torča por iluminar svoi put. La tunel führt tief in monte, bis Tomas entdeckt un hider kammer s ančient skulpturen i un glänzend tresor.<p>V la hider kammer, Tomas najde un skarbnica poln von stran objekte, inkluziv un animat statuo von un banane, ki parlat und tanzt. Zaskočen, ale smejšeći, Tomas realisat ke la glänzend tresor je ver la amikeco i rado ke li povas dividi kon svoi vilaž. La legenda &quot;Skarb de Tri Montes&quot; je veritabile un symbol za unite de diversa kulturi i lingvoj en un mondo plen da froliko i humor.

I&#x27;m more impressed by your son than I am ChatGPT....TBH.

I am curious if the advent of GPT and LLMs allows linguistic theorists to adjudicate where we are with understanding the language instinct and settling the Chomsky vs Pinker vs Others debate.<p>Perhaps it is entirely irrelevant as GLT has learned through billions of examples a child never could. Or perhaps it is totally relevant as it can synthesise billions of examples better than any linguist.

Not trusting the models&#x27;s self-assessment is the right call, considering that the actual score summed up to 7.5 compared to the self-reported 6.5 :)

Yeah I was think yesterday maybe we can start translating dolphin language.<p>Someone get on that

The idea of asking it to produce an &quot;ouroboros prompt&quot; that can be fed back into itself summarizing everything already learned is very clever; definitely going to use that in future ChatGPT sessions of my own.

I&#x27;ve been trying a few things, some are very interesting.<p>For example it understands Europanto* perfectly, but when I asked it to produce some it was germanic-only Europanto: English, German, Danish, Swedish... I told it to use more romance words and he came up with pure French. After some more prodding he achieved a decent mix.<p>I also tried to get it to behave like an ersatz Duolingo for Basque and it sorta worked, but it would need some clever working on the prompts to really be usable.<p>(*) Europanto is a joke language that uses random European language vocabulary on top of a generally English grammar.

Once again illustrating that the powerful thing about ChatGPT is that no matter what you do, it does its best to play along. Its eyes do not glaze over.

Prompt: Write a poem in Simlish.<p>ChatGPT-4:<p>While it&#x27;s important to remember that Simlish is a nonsensical language without a proper grammar or vocabulary, I can still attempt to create a playful poem using some known Simlish phrases and sounds. Keep in mind that this poem will not have any real meaning:<p><pre><code> Sul sul! Vadeesh, plurna zib, Badeesh! Dag dag, cribna drib, Froobi flazzle, noogit fraz, Glarbni weeb, frizzle maz. Lerpna gort, jibbly zor, Slooble wib, kwebna dor, Skibbi plurt, pleeba narn, Grubble veeb, shazzle darn. Wabble gret, minzi glarn, Flartle doo, zibba yarn, Yibbs frazzle, dribna glorp, Vorpti zound, greeble snorp </code></pre> Enjoy the playful sounds and rhythm of the poem, even though it lacks any real meaning!

Mine is a silent language. I look at people and wait until they guess what I&#x27;m thinking or understand what I mean. Sometimes I show them an imaginary sphere that I hold with both hands.<p>edit: I had one more OT thought: This article convinced me even further that children should learn machine code. Adults (we) should stop pretending we are anywhere near as sophisticated at learning languages as they are. I&#x27;ve seen 5 year olds play together and create a hybrid dutch&#x2F;russian language in 5 minutes. They just kept talking. Within an hour they had a pretty good understanding, within a day it was good enough for what they needed.

This sounds like a lot of fun! It’s also great that you’re encouraging this - having a passion is great, and following it through to this level is fantastic for development.<p>I have a question. When you ask an LLM to summarise the rules of the language in a concise manner, and it does so, is it… disheartening?<p>I mean, if you put a lot of work into creating something, and it gets described in terms that basically make it feel like a configuration - given a glossary and a set of toggles and fill in the blank settings (e.g. word boundaries), out pops a language.<p>Does seeing that make it feel somewhat less exciting?

The Future of GPT4 1337 Programming: CyberCode Tutorial with &lt;=\&gt; cYber-C0mb1n4t0r !!! &lt;&#x2F;=&gt;<p>I asked GPT-4 to implement QuickSort and the Y-Combinator in my own personal programming language that I designed — who can guess the influences (besides cannabis and shrooms)?<p><a href="https:&#x2F;&#x2F;donhopkins.medium.com&#x2F;the-future-of-gpt4-programming-cybercode-tutorial-with-cyber-c0mb1n4t0r-ddfdf82a84d" rel="nofollow">https:&#x2F;&#x2F;donhopkins.medium.com&#x2F;the-future-of-gpt4-programming...</a>

&gt; For example, Kłeti has a sentence structure that is SOV (Subject-Object-Verb), while all the languages I know have a Subject-Verb-Object (SVO) sentence structure.<p>&quot;all the languages I know&quot; caveat alright, still Japanese has the verb at the end, and my German is far far away but I seem to recall it has something like that as well (at least the meaningful &quot;verb-like&quot; semantic part of the sentence often being at the end)

Did it actually speak the language or did it just translate text?<p>I&#x27;m not trying to be pedantic; these are two very different tasks.

Oh I wish I had time to train it on one of my old hobbies, Lojban!<p><a href="https:&#x2F;&#x2F;lojban.io&#x2F;" rel="nofollow">https:&#x2F;&#x2F;lojban.io&#x2F;</a><p><a href="https:&#x2F;&#x2F;mw.lojban.org&#x2F;papri&#x2F;Lojban" rel="nofollow">https:&#x2F;&#x2F;mw.lojban.org&#x2F;papri&#x2F;Lojban</a>

Super curious, would fine tuning with LoRa on a LLaMa&#x2F;Alpaca model work better?

This is a super amazing stuff! Just blown away with the power of AI in tbe future! I wonder if that still exist to make some of the languages spoken by humans which are on the verge of extinction?

Not sure if ChatGPT is correct but it does sound good

You might enjoy creating personalised bedtime stories for your kid: www.mybabystory.co

bookmark

I don&#x27;t have access to ChatGPT4, but in my tests I could observe that it can&#x27;t do some very simple tasks:<p><pre><code> - It can&#x27;t play tic-tac-toe, - It can&#x27;t play hangman, - It insists that winning on stone-paper-scissor using the chat (playing before me) is a matter of probability. </code></pre> It was also demonstrated that it can&#x27;t reverse strings.<p>Actually a transformer doesn&#x27;t accesses &#x27;strings&#x27;, all it processes are tokens which are then mapped to vectors by whatever embedding is applied. I think it will be extremely difficult for a transformer to do any of these tasks correctly until a successor model is adopted.<p>I don&#x27;t have much hope of any reasonably complex symbolic processing of anything that it was not trained on. Some of these tasks are easy for a human to perform with paper and pencil and a set of rules; of course a human may get confused, but for that you write programs. Write code is one of GPT&#x27;s skills but It is not &quot;that&quot; good with code for problems that are not mere small modification of problems it was trained on.<p>EDIT: Could have expressed myself better: I don&#x27;t have access to chatGPT4; I tested using the &quot;available&quot; chatGPT, I think it is 3.5.<p>A transcript of me trying to play tic-tac-toe with it: <a href="https:&#x2F;&#x2F;pastebin.com&#x2F;V1CW5hpt" rel="nofollow">https:&#x2F;&#x2F;pastebin.com&#x2F;V1CW5hpt</a>