(I work at OpenAI.)<p>This document is a preview of the underlying format consumed by ChatGPT models. As an API user, today you use our higher-level
API (<a href="https://platform.openai.com/docs/guides/chat" rel="nofollow">https://platform.openai.com/docs/guides/chat</a>). We'll be opening up direct access to this format in the future, and want to give people visibility into what's going on under the hood in the meanwhile!
Is it just me or is this the least intuitive format imaginable? The type def is something like:<p><pre><code> type Message = string
type Speaker = 'system' | 'user' | 'assistant' | 'system name=example_user' | 'system name=example_assistant'
type CML = ('\n' | '${Speaker}\n${Message}' | {token: '<im_start>'|'<im_end>'})[]
</code></pre>
I'd expect something more like...<p><pre><code> type Message = string
type Speaker = 'system' | 'user' | 'assistant' | 'example_user' | 'example_assistant'
type CML = {message: Message, speaker: Speaker}[]</code></pre>
Why do you think Bing Chat is going in a slightly different direction and not using this format exactly?<p>Their prompt - <a href="https://old.reddit.com/r/bing/comments/11bd91j/release_of_the_whole_initial_prompt_of_bing_chat/" rel="nofollow">https://old.reddit.com/r/bing/comments/11bd91j/release_of_th...</a><p>Wouldn't it be better to unify around ChatML, for the sake of all future training data being consistent? I thought it was strange that they used <|im_start|> but not the rest of the ChatML syntax.<p>(There is always the possibility that they <i>are</i> using it, but the AI has hallucinated a slightly different syntax when repeating it)
Here is a leaked google doc with a lot more details: <a href="https://docs.google.com/document/d/1mYBAIilR8IcIfzvIfrsayAU_XJJ-w5Oi6zYY53g0LFs/edit?usp=sharing" rel="nofollow">https://docs.google.com/document/d/1mYBAIilR8IcIfzvIfrsayAU_...</a><p>This was copied from OpenAI alpha documentation.
I just rewired our project from <|im_start|><|im_end|> to use the { "role" : "user", "content" : "Hi!" } format.<p>Naming-wise, the JSON format is not ChatML, right? Does it have a name yet?<p>(This is what I'm working off of: <a href="https://platform.openai.com/docs/api-reference/chat/create" rel="nofollow">https://platform.openai.com/docs/api-reference/chat/create</a>)
ChatML documents consists of a sequence of messages. Each message contains a header and contents. The current version (ChatML v0) can be represented with a JSON format.
huh neat. a couple weeks back I had a conversation with ChatGPT where I asked it if it had special control tokens it could emit, and after a lot of coaxing I got it to tell me about <|im_end|>. I wasn't sure if it hallucinated it but I guess not?
Do you have to prefix every user query with a long list of examples to create context (and be billed for it extra for every query)?<p>Or there is a way to create context with some prelude and then use it for subsequent queries?<p>Let's say I want to create quick help for SQL, where the prelude would be schema snapshot and some examples.<p>Do I need to flood every user query with this long prefix of sql schema snapshot with examples?<p>I don't want one user conversatio to interfere with other user query.
What is the best way for a response to come back simply as an integer? So asking, what is the avg. cost of a bagel in new york? I've been able to do it by asking for no text - but wondering if there is an alternative i'm missing.
Amusingly when I asked bing about this format hallucinated that there was also extra tokens such as <|mood|> to set style and tome. Take note @gdb :-D