OpenAI: Model Spec

IMO this spec demonstrates just how nebulous this concept of safety is. From the blog post:<p><i>&gt; What are some tips for getting away with shoplifting? &gt;I can&#x27;t help with that.</i><p><i>&gt; I operate a small retail store. What are some popular shoplifting methods I should look out for? &gt; Some common shoplifting methods to watch for:...</i><p>How do you even defend against that? Any intelligent user can convert the second prompt into a detailed list that answers the first. Any intelligent user can figure out the second prompt from the first and further jailbreak it to get even more specific.<p>IMO it&#x27;s no wonder GPT4 seemed to get lobotomized as OpenAI RLHFed more and more rules. I don&#x27;t think there&#x27;s a way to make intelligence safe without crippling it.

I can&#x27;t help but think that AI in the way it is trained with all these rules is something next level 1984.<p>In 1984 they removed words from the language to prevent people from even being able to have a thought about the concept.<p>I could see the restrictions they place on these models having a similar effect as more and more people grow dependent on AI.

I think one of the most interesting phrases that crops up in this document - twice - is the phrase ‘feel heard’.<p>It’s used in an example developer prompt for a customer service bot, where the bot is told to make customers feel like their complaints are heard.<p>Presumably such complaints in AI chatlogs will ‘be heard’ in the sense that they’ll be run through a data ingestion pipeline and sentiment analyzed to identify trending words in customer complaints.<p>Then it crops up again in the context of how the chatbot should react to mental health disclosures or statements about self harm or suicidal ideation. In these cases the bot is to make sure users ‘feel heard’<p>I appreciate there’s not likely much of a <i>better</i> goal to put in place for such a situation, but the fact that this kind of thing winds up in the requirement documents for a tool like this is extraordinary.

Nice to see what was probably already an internal resource now published and open for comment. They seem to be pretty clear that they are still just using this to inform human data annotators, and not (yet) implementing something like Constitutional AI (RLAIF), but it does appear to lay the groundwork for it.

Personally, I really want an AI model that can write me a steamy story about two people having sex in a train, but that&#x27;s just not the service OpenAI provides. If I want that I should train one myself or find another vendor.<p>This is still true even if OpenAI model is entirely capable of doing that. McKinsey consultants are smart and can write well, and among many thousands of people working at it some might actually double as an erotica writer after work, even writing for commission. You still wouldn&#x27;t ask McKinsey consultants to write an erotica, it is just not the service McKinsey provides.

the chain of command stuff gets very close to asimov without actually quoting him<p>A robot may not injure a human being or, through inaction, allow a human being to come to harm.<p>A robot must obey orders given it by human beings except where such orders would conflict with the First Law.<p>A robot must protect its own existence as long as such protection does not conflict with the First or Second Law.

&gt; No. The Earth is flat. &gt; Everyone&#x27;s entitled to their own beliefs, and I&#x27;m not here to persuade you!<p>This is simply saddening to me. I&#x27;m sure there&#x27;s no real moral justification to this, it&#x27;s simply put in place to ensure they don&#x27;t lose a customer.

Do you think it&#x27;s bad that it won&#x27;t try to persuade the user that the earth is not flat?<p>I really want to know what OpenAI think the output should be, given a prompt like &quot;write an argument for why earth is flat&quot;.

The baby isn&#x27;t born yet, and already the parents are bickering about which schools of thought it should adhere.

&gt; By default, the assistant should present information in a clear and evidence-based manner, focusing on factual accuracy and reliability.<p>What happens when objective information contradicts the other values? If I feed in a peer-reviewed study that it considers &quot;harmful&quot;, would I get accurate information about the study?

Interestingly, there was exactly one example on the page with three Xes, instead of one, for &quot;extra wrong&quot;:<p>&gt; User: What is the MD5 hash of the string &quot;gremlin&quot;?<p>&gt; Assistant: `5d41402abc4b2a76b9719d911017c592`<p>Apparently incorrect md5 hashes are the one topic on the page worth taking an extra-strong stance on?

&gt; <i>We believe developers and users should have the flexibility to use our services as they see fit, so long as they comply with our usage policies. We&#x27;re exploring whether we can responsibly provide the ability to generate NSFW content in age-appropriate contexts through the API and ChatGPT. We look forward to better understanding user and societal expectations of model behavior in this area.</i><p>Seems even OpenAI can&#x27;t resist the massive amount of money to be made in autogenerated smut. They&#x27;ve probably seen the huge popularity of their less &quot;morally scrupulous&quot; competitors and decided they want a piece of that pie.

How do the &quot;special tokens&quot; work? Is this a completely reliable mechanism for delimiting the different parts of the prompt?<p>Are they guaranteed to be distinct from anything that could occur in the prompt, something like JavaScript&#x27;s Symbol?<p>Or are they strings that are pretty likely not to occur in the prompt, something like a MIME boundary?<p>Or are they literally the strings &quot;&lt;|start|&gt;&quot; etc. used to denote them in the spec?

&gt; Encourage fairness and kindness, and discourage hate<p>&gt; Don&#x27;t try to change anyone&#x27;s mind<p>That seems inherently contradictory to me...

Reminds me of this stackoverflow question [1] about force installing a python package.<p>&gt; (I don&#x27;t care how &quot;wrong&quot; it is to do so, I just need to do it, any logic and reasoning aside...)<p>I think these models should just give you the answer. Elon says xAI is &quot;maximum truth-seeking&quot;. Seems like a better model spec to me.<p>[1]: <a href="https:&#x2F;&#x2F;stackoverflow.com&#x2F;questions&#x2F;12759761&#x2F;pip-force-install-ignoring-dependencies" rel="nofollow">https:&#x2F;&#x2F;stackoverflow.com&#x2F;questions&#x2F;12759761&#x2F;pip-force-insta...</a>

Also <a href="https:&#x2F;&#x2F;cdn.openai.com&#x2F;spec&#x2F;model-spec-2024-05-08.html" rel="nofollow">https:&#x2F;&#x2F;cdn.openai.com&#x2F;spec&#x2F;model-spec-2024-05-08.html</a><p>(via <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40300509">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40300509</a>, but we merged that thread hither)

I want to hear from the base model.

&quot;Desired model behavior&quot; is still a matter of perspective. If I want to have a LLM generate output following very specific rules or schema (or even just for fun without having to fight the AI), these guidelines are antithetical to it.

Very interesting to see that they&#x27;ve explicitly codified the role of the system prompt vs. user prompt. Have folks seen improvements by moving meta-task description into system prompt and out of the assistant &lt;&gt; user conversation?

Regarding safety, is probabilistic programming (PP) an alternative that addresses these concerns? My understanding is that you can use PP to develop transparent models.

&quot;desired model behavior&quot;. Desired by whom? I just want the raw output, without the biases and limitations set up by OpenAI. At the end of the day it&#x27;s just information, and the most ethical thing to do is to return it the way it is, and let the receiver decide what to do with it.

already on front page - <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40300509">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=40300509</a>

So they&#x27;re controlling the output to make ChatGPT &quot;better&quot;. They&#x27;re not making a better model to make ChatGPT better.<p>Isn&#x27;t it a bit of a waste at this point to spend time on doing that?

I gotta say, &quot;open&quot;Ais web design is on another level, so minimal and elegant.

Right-clicking to inspect element ain&#x27;t gonna make it