Is Grok Basically Just an OpenAI Wrapper?

An xAI engineer replied:<p>&gt; The issue here is that the web is full of ChatGPT outputs, so we accidentally picked up some of them when we trained Grok on a large amount of web data. This was a huge surprise to us when we first noticed it. For what it’s worth, the issue is very rare and now that we’re aware of it we’ll make sure that future versions of Grok don’t have this problem. Don’t worry, no OpenAI code was used to make Grok.<p><a href="https:&#x2F;&#x2F;twitter.com&#x2F;ibab_ml&#x2F;status&#x2F;1733558576982155274" rel="nofollow noreferrer">https:&#x2F;&#x2F;twitter.com&#x2F;ibab_ml&#x2F;status&#x2F;1733558576982155274</a>

It&#x27;s far, far more likely that Grok was just trained on data that includes text generated by GPT.<p>This is super common in the open-source&#x2F;local AI area. Many models are trained on output from GPT. The better models will filter out anything that mentions GPT or OpenAI. Seems like Grok is not one of the better models.

I believe it is. I&#x27;ve had people send me screenshots where it refers to itself as chatGPT.

This isn&#x27;t really that newsworthy. Grok was probably trained on ChatGPT logs, just like the litany of other chat-oriented LLMs that are also open source. They still should&#x27;ve filtered out the OpenAI canned responses (there are datasets with that filter).

What&#x27;s grok? (I dropped off Twitter&#x2F;X the moment the inmates started running that asylum again. It appears to have to do with X?)

I had self hosted Llama 2 call itself Bard the other day, so I&#x27;d take this with a grain of salt - or maybe even the whole shaker.

A simple explanation is that Grok probably used GPT3.5 or GPT4 API to generate synthetic data, most likely for RLHF.

I feel like comments defending Twitter (I&#x27;m still calling it that until they change their domain) are half missing the point. True, it&#x27;s not necessarily an &quot;OpenAI <i>API</i> wrapper&quot;, but the fact that it was trained on ChatGPT&#x27;s logs basically means it&#x27;s still an &quot;OpenAI wrapper&quot; of sorts, and it&#x27;s going to be inferior in nearly every way (I always thought a huge problem with LLMs going forward was the risk that they would be &quot;contaminated&quot; with non-human training data).<p>I can&#x27;t think of any reason anybody would use Grok over ChatGPT besides political tribe signalling.

Grok is a new model. Its training data has seen enough examples of OpenAI generated content that it occasionally spits out text like this that makes it look like it&#x27;s by OpenAI.<p>This is common issue across all sorts of other alternative models too. It&#x27;s not particularly surprising.

I wonder why they decided not to filter-out references to OpenAI and ChatGPT.

It makes no sense to me that one would train a chatbot on chatgpt conversations and not filter strings that literally say &quot;openai&quot; and &quot;chatgpt&quot;. Extreme incompetence.