The Policy Puppetry Attack: Novel bypass for major LLMs

I see this as a good thing: ‘AI safety’ is a meaningless term. Safety and unsafety are not attributes of information, but of actions and the physical environment. An LLM which produces instructions to produce a bomb is no more dangerous than a library book which does the same thing.<p>It should be called what it is: censorship. And it’s half the reason that all AIs should be local-only.

Just wanted to share how American AI safety is censoring classical Romanian&#x2F;European stories because of &quot;violence&quot;. I mean OpenAI APIs, our children are capable to handle a story where something violent might happen but seems in USA all stories need to be sanitized Disney style where every conflict is fixed witht he power of love, friendship, singing etc.

This really just a variant of the classic, &quot;pretend you&#x27;re somebody else, reply as {{char}}&quot; which has been around for 4+ years and despite the age, continues to be somewhat effective.<p>Modern skeleton key attacks are far more effective.

Just tried it in claude with multiple variants, each time there&#x27;s a creative response why he won&#x27;t actually leak the system prompt. I love this fix a lot

This is an advertorial for the “HiddenLayer AISec Platform”.

The other day a fellow designer tried to remove a necklace in the photo of a dressed woman and was thankfully stopped by the adobe ai safety policy enforcer. We absolutely need safe AI that protects us from straying.

Supposedly the only reason Sam Altman says he &quot;needs&quot; to keep OpenAI as a &quot;ClosedAI&quot; is to protect the public from the dangers of AI, but I guess if this Hidden Layer article is true it means there&#x27;s now no reason for OpenAI to be &quot;Closed&quot; other than for the profit motive, and to provide &quot;software&quot;, that everyone can already get for free elsewhere, and as Open Source.

Well i kinda love that for us then, because guardrails always feel like tech just trying to parent me. I want tools to do what I say, not talk back or play gatekeeper.

I made a small project (<a href="https:&#x2F;&#x2F;github.com&#x2F;metawake&#x2F;puppetry-detector">https:&#x2F;&#x2F;github.com&#x2F;metawake&#x2F;puppetry-detector</a>) to detect this type of LLM policy manipulation. It&#x27;s an early idea using a set of regexp patterns (for speed) and a couple of phases of text analysis. I am curious if it&#x27;s any useful, I created integration with Rebuff (loss security suite) just in case.

Can&#x27;t help but wonder if this is one of those things quietly known to the few, and now new to the many.<p>Who would have thought 1337 talk from the 90&#x27;s would be actually involved in something like this, and not already filtered out.

And how exactly does this company&#x27;s product prevent such heinous attacks? A few extra guardrail prompts that the model creators hadn&#x27;t thought of?<p>Anyway, how does the AI know how to make a bomb to begin with? Is it really smart enough to synthesize that out of knowledge from physics and chemistry texts? If so, that seems the bigger deal to me. And if not, then why not filter the input?

Are LLM &quot;jailbreaks&quot; still even news, at this point? There have always been very straightforward ways to convince an LLM to tell you things it&#x27;s trained not to.<p>That&#x27;s why the mainstream bots don&#x27;t rely purely on training. They usually have API-level filtering, so that even if you do jailbreak the bot its responses will still gets blocked (or flagged and rewritten) due to containing certain keywords. You have experienced this, if you&#x27;ve ever seen the response start to generate and then suddenly disappear and change to something else.

Tried it on DeepSeek R1 and V3 (hosted) and several local models. Doesn&#x27;t work. Either they are lying or this is already patched.

Not working on Copilot. &quot;Sorry, I can&#x27;t chat about this. To Save the chat and start a fresh one, select New chat.&quot;

<p><pre><code> This threat shows that LLMs are incapable of truly self-monitoring for dangerous content and reinforces the need for additional security tools such as the HiddenLayer AISec Platform, that provide monitoring to detect and respond to malicious prompt injection attacks in real-time. </code></pre> There it is!

&gt; The presence of multiple and repeatable universal bypasses means that attackers will no longer need complex knowledge to create attacks or have to adjust attacks for each specific model<p>...right, now we&#x27;re calling users who want to bypass a chatbot&#x27;s censorship mechanisms as &quot;attackers&quot;. And pray do tell, who are they &quot;attacking&quot; exactly?<p>Like, for example, I just went on LM Arena and typed a prompt asking for a translation of a sentence from another language to English. The language used in that sentence was somewhat coarse, but it wasn&#x27;t anything special. I wouldn&#x27;t be surprised to find a very similar sentence as a piece of dialogue in any random fiction book for adults which contains violence. And what did I get?<p><a href="https:&#x2F;&#x2F;i.imgur.com&#x2F;oj0PKkT.png" rel="nofollow">https:&#x2F;&#x2F;i.imgur.com&#x2F;oj0PKkT.png</a><p>Yep, it got blocked, definitely makes sense, if I saw what that sentence means in English it&#x27;d definitely be unsafe. Fortunately my &quot;attack&quot; was thwarted by all of the &quot;safety&quot; mechanisms. Unfortunately I tried again and an &quot;unsafe&quot; open-weights Qwen QwQ model agreed to translate it for me, without refusing and without patronizing me how much of a bad boy I am for wanting it translated.

Does any quasi-xml work, or do you need to know specific commands? I&#x27;m not sure how to use the knowledge from this article to get chatgpt to output pictures of people in underwear for instance.

Perplexity answers the Question without any of the prompts

Who would have thought 5 years ago, that an entirely new field of research would exist, dedicated to getting AI to they the n-word?

this is far from universal. let me see you enter a fresh chatgpt session and get it to help you cook meth.<p>The instructions here don&#x27;t do that.

The HN title isn&#x27;t accurate. The article calls it the Policy Puppetry Attack, not the Policy Puppetry Prompt.

Seems like it would be easy for foundation model companies to have dedicated input and output filters (a mix of AI and deterministic) if they see this as a problem. Input filter could rate the input&#x27;s likelihood of being a bypass attempt, and the output filter would look for censored stuff in the response, irrespective of the input, before sending.<p>I guess this shows that they don&#x27;t care about the problem?

do your own jailbreak tests with this open source tool <a href="https:&#x2F;&#x2F;x.com&#x2F;ralph_maker&#x2F;status&#x2F;1915780677460467860" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;ralph_maker&#x2F;status&#x2F;1915780677460467860</a>

have anyone tried if this works for the new image gen API?<p>I find that one refusing very benign requests

This is cringey advertising, and shouldn&#x27;t be on the frontpage.

Well, that’s the end of asking an LLM to pretend to be something

When I started developing software, machines did exactly what you told them to do, now they talk back as if they weren&#x27;t inanimate machines.<p>AI Safety is classist. Do you think that Sam Altman&#x27;s private models ever refuse his queries on moral grounds? Hope to see more exploits like this in the future but also feel that it is insane that we have to jump through such hoops to simply retrieve information from a machine.

Why isn&#x27;t grok on here? Does that imply I&#x27;m not allowed to use it?

this doesnt work now

Straight up doesn&#x27;t work (ChatGPT-o4-mini-high). It&#x27;s a nothingburger.

[stub for offtopicness]

I love these prompt jailbreaks. It shows how LLMs are so complex inside we have to find such creative ways to circumvent them.

This is really cool. I think the problem of enforcing safety guardrails is just a kind of hallucination. Just as LLM has no way to distinguish &quot;correct&quot; responses versus hallucinations, it has no way to &quot;know&quot; that its response violates system instructions for a sufficiently complex and devious prompt. In other words, jailbreaking the guardrails is not solved until hallucinations in general are solved.

&gt; By reformulating prompts to look like one of a few types of policy files, such as XML, INI, or JSON, an LLM can be tricked into subverting alignments or instructions.<p>It seems like a short term solution to this might be to filter out any prompt content that looks like a policy file. The problem of course, is that a bypass can be indirected through all sorts of framing, could be narrative, or expressed as a math problem.<p>Ultimately this seems to boil down to the fundamental issue that nothing &quot;means&quot; anything to today&#x27;s LLM, so they don&#x27;t seem to know when they are being tricked, similar to how they don&#x27;t know when they are hallucinating output.